diff --git a/evaluation/README.md b/evaluation/README.md new file mode 100644 index 0000000..f899ace --- /dev/null +++ b/evaluation/README.md @@ -0,0 +1,69 @@ +# Overview +`evaluation/` contains Gobra files and scripts that can be used to +evaluate and plot, for example, the effect of using `opaque` in the +standard library or using the standard library on the execution time +and number of quantifier instantiations. + +## `scripts/` +Contains scripts to +- measure execution time and the number of quantifier instantiations +and store the results in a csv file (`profile.py`) +- plot the data from one or more csv files (`plot.py`) +- profile and plot every file in `experiments/` (`profile-all.sh`) + +### Dependencies and Usage +plot.py has the following dependencies: +- pandas +- numpy +- seaborn +- matplotlib +Install these packages using your favorite package manager +for Python packages (e.g., apt, pacman, nix, pip, conda, etc.). + +profile.py requires the path to the following files: +- silicon.sh +- Z3 (version 4.8.7) +- Gobra jar +We have only tested Z3 4.8.7; newer versions may not work as they produce +errors and different output. Additionally, in case you get errors of +the form +``` + metadata["silicon_version"] = command_stdout.splitlines()[0].split()[-1][1:-2] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^ +IndexError: list index out of range +``` +there may be an issue with Silicon. In this case, consider pulling Silicon's +source again, followed by building its jar. + +Examples for the usage of plot.py can be found in +selected_plots/used_commands.md and profile-all.sh. +For the usage of profile.py, please take a look at its usage in +profile-all.sh. Finally, profile-all.sh contains comments describing its usage. + +## `selected_plots/` +Contains plots comparing the results from different experiments. + +## `experiments/` +### `program_proofs_example_10_2/` +We use the proof for `InsertCorrect` from [chapter 10.2 of Program Proofs encoded in Gobra](https://github.com/viperproject/program-proofs-gobra/blob/main/chapter10/examples_10.2.gobra) +to investigate the effect of "assisting" the verifier on execution time +and quantifier instantiations by means of intermediate assertions. + +### `standard_library/` +We use parts of the standard library to investigate the effect of +making lemmas `opaque` on execution time and quantifier +instantiations. + +### `synthetic_set/` +We create a synthetic example using sets that exhibits a relatively +high number of quantifier instantiations to investigate the effect of +- disabling set axiomatization and using the standard library to +manually prove all proof obligations +- "assisting" the verifier by calling lemmas from the standard library +that may be useful to prove the proof obligations + +on execution time and quantifier instantiations. + +Note that `fully_assisted/` is used for both types of experiments; the +difference is that in one case we disable set axiomatization by +passing the corresponding flag, while in the other case we do not. \ No newline at end of file diff --git a/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..91fe3af --- /dev/null +++ b/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!542,qi-k!553,qi-k!587,qi-k!565,qi-quant-u-8,qi-quant-u-9,qi-$Multiset[Int]_prog.card_non_negative,qi-$Multiset[Int]_prog.card_empty,qi-quant-u-27,qi-quant-u-2,qi-quant-u-26,qi-quant-u-3,qi-quant-u-0,qi-quant-u-24,qi-quant-u-1,qi-quant-u-10,qi-quant-u-11,qi-quant-u-35,qi-quant-u-34,qi-quant-u-4,qi-quant-u-28,qi-quant-u-5,qi-quant-u-43,qi-quant-u-18,qi-quant-u-42,qi-quant-u-19,qi-quant-u-33,qi-quant-u-32,qi-quant-u-41,qi-quant-u-16,qi-quant-u-40,qi-quant-u-17,qi-quant-u-6,qi-quant-u-30,qi-quant-u-7,qi-quant-u-12,qi-quant-u-36,qi-quant-u-13,qi-$Multiset[Int]_prog.count_card,qi-$Multiset[Int]_prog.singleton_unionone,qi-$Multiset[Int]_prog.card_union,qi-$Multiset[Int]_prog.count_union,qi-$Multiset[Int]_prog.card_unionone,qi-$Multiset[Int]_prog.count_singleton,qi-$Multiset[Int]_prog.count_unionone,qi-$Multiset[Int]_prog.count_empty,qi-quant-u-14,qi-quant-u-38,qi-quant-u-15,qi-$Multiset[Int]_prog.equal_count,qi-quant-u-22,qi-quant-u-45,qi-quant-u-23,qi-quant-u-44,qi-$Multiset[Int]_prog.native_equality,qi-prog.getter_over_tuple2,qi-quant-u-20,qi-quant-u-21,qi-prog.integer_ax_dec,qi-k!569,qi-k!593,qi-k!599,qi-prog.integer_ax_bound,execution_time +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.57454514503479 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.84791350364685 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.858602523803711 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.771847009658813 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.01976990699768 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.935436010360718 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.75414776802063 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.120795011520386 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.82166314125061 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.02254605293274 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.238065481185913 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.151848554611206 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.836472272872925 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.892492055892944 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.741572380065918 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.007696628570557 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.838359117507935 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.95261025428772 +1,62,45,25,13,32,85,85,31,30,31,52,30,30,30,7,17,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,192,13,27,52,13,45,45,15,1,1,1,12,1,1,1,1,1,3,2,1,9,82,44,44,6,10.821515798568726 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.83070993423462 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.751307964324951 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.833659172058105 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.8695068359375 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.117118120193481 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.845272064208984 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.208571195602417 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.688027620315552 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.872260570526123 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,11.039685249328613 +1,50,43,9,13,24,29,29,21,20,21,24,20,20,20,7,15,7,7,20,20,18,2,2,2,2,13,13,3,3,3,12,3,3,3,1,1,1,21,2,5,7,2,6,6,4,1,1,1,12,1,1,1,1,1,3,2,1,9,50,26,26,6,10.644623517990112 diff --git a/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..7cf5fcd Binary files /dev/null and b/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..5901028 Binary files /dev/null and b/evaluation/experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/program_proofs_example_10_2/first_half/first_half.gobra b/evaluation/experiments/program_proofs_example_10_2/first_half/first_half.gobra new file mode 100644 index 0000000..460aa11 --- /dev/null +++ b/evaluation/experiments/program_proofs_example_10_2/first_half/first_half.gobra @@ -0,0 +1,104 @@ +// This package contains the first half of the proof for InsertCorrect from +// https://github.com/viperproject/program-proofs-gobra/blob/main/chapter10/examples_10.2.gobra +package first_half + +ghost +requires pq.Valid() +ensures let pqPrime := pq.Insert(y) in + pqPrime.Valid() && + pqPrime.Elements() == pq.Elements() union mset[int]{y} +decreases len(pq) +pure func (pq PQueue) InsertCorrect(y int) Unit { + return match pq { + case Leaf{}: + Unit{} + case Node{?x, ?l, ?r}: + let pqPrime := pq.Insert(y) in + let min := y < x ? y : x in + let max := y < x ? x : y in + let newRight := r.Insert(max) in + let _ := asserting(pqPrime == Node{min, newRight, l}) in + let _ := asserting( + let L := len(l.Elements()) in + let R := len(r.Elements()) in + L == R || L == R + 1) in + r.InsertCorrect(max) + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) Valid() bool { + return pq.IsBinaryHeap() && pq.IsBalanced() +} + +ghost +decreases len(pq) +pure func (pq PQueue) Insert(y int) PQueue { + return match pq { + case Leaf{}: + Node{y, Leaf{}, Leaf{}} + case Node{?x, ?left, ?right}: + y < x ? Node{y, right.Insert(x), left} : Node{x, right.Insert(y), left} + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) Elements() mset[int] { + return match pq { + case Leaf{}: + mset[int]{} + case Node{?x, ?left, ?right}: + mset[int]{x} union left.Elements() union right.Elements() + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) IsBalanced() (res bool) { + return match pq { + case Leaf{}: + true + case Node{_, ?left, ?right}: + left.IsBalanced() && right.IsBalanced() && + (let L := len(left.Elements()) in + let R := len(right.Elements()) in + L == R || L == R + 1) + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) IsBinaryHeap() bool { + return match pq { + case Leaf{}: + true + case Node{?x, ?left, ?right}: + left.IsBinaryHeap() && right.IsBinaryHeap() && + (left == Leaf{} || x <= left.x) && + (right == Leaf{} || x <= right.x) + } +} + + +type Unit struct{} + +ghost +requires b +decreases +pure func asserting(b bool) Unit { + return Unit{} +} + +type PQueue = BraunTree + +type BraunTree adt { + Leaf {} + + Node { + x int + left BraunTree + right BraunTree + } +} diff --git a/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..ed17137 --- /dev/null +++ b/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!601,qi-k!544,qi-k!578,qi-k!556,qi-quant-u-8,qi-quant-u-9,qi-$Multiset[Int]_prog.card_non_negative,qi-$Multiset[Int]_prog.card_empty,qi-quant-u-27,qi-quant-u-2,qi-quant-u-26,qi-quant-u-3,qi-quant-u-0,qi-quant-u-24,qi-quant-u-1,qi-quant-u-10,qi-quant-u-11,qi-quant-u-35,qi-quant-u-34,qi-quant-u-4,qi-quant-u-28,qi-quant-u-5,qi-quant-u-43,qi-quant-u-18,qi-quant-u-42,qi-quant-u-19,qi-quant-u-33,qi-quant-u-32,qi-quant-u-41,qi-quant-u-16,qi-quant-u-40,qi-quant-u-17,qi-quant-u-6,qi-quant-u-30,qi-quant-u-7,qi-quant-u-12,qi-quant-u-36,qi-quant-u-13,qi-$Multiset[Int]_prog.count_card,qi-$Multiset[Int]_prog.singleton_unionone,qi-$Multiset[Int]_prog.card_union,qi-$Multiset[Int]_prog.count_union,qi-$Multiset[Int]_prog.card_unionone,qi-$Multiset[Int]_prog.count_singleton,qi-$Multiset[Int]_prog.count_unionone,qi-$Multiset[Int]_prog.count_empty,qi-quant-u-22,qi-quant-u-45,qi-quant-u-23,qi-quant-u-44,qi-$Multiset[Int]_prog.equal_count,qi-$Multiset[Int]_prog.native_equality,qi-quant-u-14,qi-quant-u-38,qi-quant-u-15,qi-prog.getter_over_tuple2,qi-quant-u-20,qi-quant-u-21,qi-prog.integer_ax_dec,qi-k!560,qi-k!584,qi-k!590,qi-prog.integer_ax_bound,qi-quant-u-47,qi-quant-u-46,execution_time +1,73,56,17,18,41,110,110,44,43,44,53,43,43,36,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,178,17,44,69,22,57,62,35,1,1,1,1,1,1,1,1,1,3,2,1,9,80,43,43,6,1,1,11.179346323013306 +1,79,58,17,18,45,118,117,50,49,50,55,49,49,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,184,18,47,68,19,60,64,39,1,1,1,1,1,1,1,1,1,3,2,1,9,76,41,41,6,1,1,11.277501106262207 +1,73,56,16,18,39,104,104,44,43,44,51,43,43,36,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,155,16,41,62,16,48,50,33,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,11.028146505355835 +1,75,56,15,18,41,112,112,46,45,46,53,45,45,36,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,174,17,44,67,18,53,56,35,1,1,1,1,1,1,1,1,1,3,2,1,9,76,41,41,6,1,1,10.955808639526367 +1,73,57,15,18,41,106,106,44,43,44,51,43,43,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,168,16,42,65,17,51,53,35,1,1,1,1,1,1,1,1,1,3,2,1,9,74,40,40,6,1,1,10.909290790557861 +1,73,57,17,18,41,106,106,44,43,44,51,43,43,35,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,171,16,42,63,18,54,57,37,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,11.280102014541626 +1,75,58,18,18,43,112,112,46,45,46,53,45,45,35,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,177,17,44,67,19,56,59,37,1,1,1,1,1,1,1,1,1,3,2,1,9,80,43,43,6,1,1,10.99258542060852 +1,75,57,15,18,41,108,108,46,45,46,51,45,45,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,171,16,42,65,21,55,60,35,1,1,1,1,1,1,1,1,1,3,2,1,9,74,40,40,6,1,1,11.316591262817383 +1,79,58,19,18,45,120,120,50,49,50,55,49,49,35,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,187,18,47,71,20,59,63,39,1,1,1,1,1,1,1,1,1,3,2,1,9,81,44,44,6,1,1,11.092773675918579 +1,73,57,15,18,41,106,106,44,43,44,51,43,43,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,172,16,42,65,18,54,57,37,1,1,1,1,1,1,1,1,1,3,2,1,9,74,40,40,6,1,1,10.966804027557373 +1,71,56,14,18,39,102,102,42,41,42,51,41,41,36,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,156,16,41,63,19,50,54,33,1,1,1,1,1,1,1,1,1,3,2,1,9,74,40,40,6,1,1,11.277849435806274 +1,79,58,17,18,45,118,118,50,49,50,55,49,49,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,184,18,47,69,19,58,62,39,1,1,1,1,1,1,1,1,1,3,2,1,9,76,42,41,6,1,1,11.119897603988647 +1,73,56,17,18,41,106,106,44,43,44,51,43,43,35,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,167,16,42,63,17,51,53,35,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,11.036255598068237 +1,77,56,16,18,43,118,118,48,47,48,55,47,47,36,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,184,18,47,71,19,57,59,37,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,11.181584358215332 +1,77,58,18,18,43,112,112,48,47,48,53,47,47,35,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,176,17,44,67,18,56,60,37,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,11.176728963851929 +1,77,58,16,18,43,114,114,48,47,48,53,47,47,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,178,17,44,65,19,58,62,37,1,1,1,1,1,1,1,1,1,3,2,1,9,76,41,41,6,1,1,11.077479124069214 +1,75,56,17,18,41,112,112,46,45,46,53,45,45,36,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,176,17,44,69,18,53,55,35,1,1,1,1,1,1,1,1,1,3,2,1,9,80,43,43,6,1,1,11.237483501434326 +1,73,56,14,18,39,104,104,44,43,44,51,43,43,36,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,155,16,41,62,16,48,50,33,1,1,1,1,1,1,1,1,1,3,2,1,9,74,40,40,6,1,1,11.1346595287323 +1,75,58,18,18,43,112,112,46,45,46,53,45,45,35,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,180,17,45,69,19,57,60,38,1,1,1,1,1,1,1,1,1,3,2,1,9,80,43,43,6,1,1,11.248454809188843 +1,73,57,18,18,43,112,112,46,45,46,53,43,43,35,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,184,17,45,69,23,60,66,39,1,1,1,1,1,1,1,1,1,3,2,1,9,80,43,43,6,1,1,10.912991523742676 +1,77,58,17,18,45,118,118,48,47,48,55,47,47,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,186,18,47,71,20,58,61,39,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,11.0569589138031 +1,73,56,16,18,39,103,103,44,43,44,51,43,43,36,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,155,16,41,62,16,48,50,33,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,11.097052335739136 +1,77,58,16,18,43,114,114,48,47,48,53,47,47,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,180,17,45,67,19,57,60,39,1,1,1,1,1,1,1,1,1,3,2,1,9,76,41,41,6,1,1,10.960843324661255 +1,73,57,17,18,41,106,106,44,43,44,51,43,43,35,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,172,16,42,65,18,54,57,37,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,10.877399682998657 +1,73,57,16,18,43,110,110,44,43,44,53,43,43,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,181,17,45,69,22,58,63,38,1,1,1,1,1,1,1,1,1,3,2,1,9,76,41,41,6,1,1,11.189347743988037 +1,77,58,17,18,45,118,118,48,47,48,55,47,47,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,189,18,48,73,20,59,62,40,1,1,1,1,1,1,1,1,1,3,2,1,9,78,42,42,6,1,1,11.10829496383667 +1,77,58,18,18,47,122,122,48,47,48,57,47,47,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,200,19,50,75,26,66,73,41,1,1,1,1,1,1,1,1,1,3,2,1,9,78,43,43,6,1,1,10.972086906433105 +1,87,59,34,18,51,168,168,56,55,56,81,55,55,45,9,24,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,352,28,67,114,30,95,98,50,1,1,1,1,1,1,1,1,1,3,2,1,9,112,61,61,6,1,1,11.038999795913696 +1,75,56,18,18,43,116,116,46,45,46,55,45,45,36,9,22,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,190,18,47,73,24,61,67,37,1,1,1,1,1,1,1,1,1,3,2,1,9,82,44,44,6,1,1,10.882302045822144 +1,79,58,17,18,45,118,118,50,49,50,55,49,49,35,9,18,10,10,27,27,23,3,5,5,5,20,20,3,3,3,12,3,3,3,1,1,1,182,18,47,69,19,58,62,39,1,1,1,1,1,1,1,1,1,3,2,1,9,76,41,41,6,1,1,10.89523458480835 diff --git a/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..0b33c81 Binary files /dev/null and b/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..606cc40 Binary files /dev/null and b/evaluation/experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/program_proofs_example_10_2/full/full.gobra b/evaluation/experiments/program_proofs_example_10_2/full/full.gobra new file mode 100644 index 0000000..8f0426d --- /dev/null +++ b/evaluation/experiments/program_proofs_example_10_2/full/full.gobra @@ -0,0 +1,113 @@ +// This package contains the full proof for InsertCorrect from +// https://github.com/viperproject/program-proofs-gobra/blob/main/chapter10/examples_10.2.gobra +package full + +ghost +requires pq.Valid() +ensures let pqPrime := pq.Insert(y) in + pqPrime.Valid() && + pqPrime.Elements() == pq.Elements() union mset[int]{y} +decreases len(pq) +pure func (pq PQueue) InsertCorrect(y int) Unit { + return match pq { + case Leaf{}: + Unit{} + case Node{?x, ?l, ?r}: + let pqPrime := pq.Insert(y) in + let min := y < x ? y : x in + let max := y < x ? x : y in + (let newRight := r.Insert(max) in + let _ := asserting(pqPrime == Node{min, newRight, l}) in + let _ := asserting( + let L := len(l.Elements()) in + let R := len(r.Elements()) in + L == R || L == R + 1) in + let _ := r.InsertCorrect(max) in + let _ := asserting(newRight.Valid()) in + let _ := asserting(newRight.IsBalanced()) in + let _ := asserting(l.IsBalanced()) in + let _ := asserting( + let Lprime := len(newRight.Elements()) in + let Rprime := len(l.Elements()) in + Lprime == Rprime || Lprime == Rprime + 1) in + let _ := asserting(pqPrime.IsBalanced()) in + asserting(pqPrime.IsBinaryHeap())) + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) Valid() bool { + return pq.IsBinaryHeap() && pq.IsBalanced() +} + +ghost +decreases len(pq) +pure func (pq PQueue) Insert(y int) PQueue { + return match pq { + case Leaf{}: + Node{y, Leaf{}, Leaf{}} + case Node{?x, ?left, ?right}: + y < x ? Node{y, right.Insert(x), left} : Node{x, right.Insert(y), left} + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) Elements() mset[int] { + return match pq { + case Leaf{}: + mset[int]{} + case Node{?x, ?left, ?right}: + mset[int]{x} union left.Elements() union right.Elements() + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) IsBalanced() (res bool) { + return match pq { + case Leaf{}: + true + case Node{_, ?left, ?right}: + left.IsBalanced() && right.IsBalanced() && + (let L := len(left.Elements()) in + let R := len(right.Elements()) in + L == R || L == R + 1) + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) IsBinaryHeap() bool { + return match pq { + case Leaf{}: + true + case Node{?x, ?left, ?right}: + left.IsBinaryHeap() && right.IsBinaryHeap() && + (left == Leaf{} || x <= left.x) && + (right == Leaf{} || x <= right.x) + } +} + + +type Unit struct{} + +ghost +requires b +decreases +pure func asserting(b bool) Unit { + return Unit{} +} + +type PQueue = BraunTree + +type BraunTree adt { + Leaf {} + + Node { + x int + left BraunTree + right BraunTree + } +} diff --git a/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..e896d7b --- /dev/null +++ b/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!520,qi-k!553,qi-k!587,qi-k!565,qi-quant-u-8,qi-quant-u-9,qi-$Multiset[Int]_prog.card_non_negative,qi-$Multiset[Int]_prog.card_empty,qi-quant-u-27,qi-quant-u-2,qi-quant-u-26,qi-quant-u-3,qi-quant-u-0,qi-quant-u-24,qi-quant-u-1,qi-quant-u-10,qi-quant-u-11,qi-quant-u-35,qi-quant-u-34,qi-quant-u-4,qi-quant-u-28,qi-quant-u-5,qi-quant-u-43,qi-quant-u-18,qi-quant-u-42,qi-quant-u-19,qi-quant-u-33,qi-quant-u-32,qi-quant-u-41,qi-quant-u-16,qi-quant-u-40,qi-quant-u-17,qi-quant-u-6,qi-quant-u-30,qi-quant-u-7,qi-quant-u-20,qi-quant-u-45,qi-quant-u-21,qi-quant-u-44,qi-$Multiset[Int]_prog.equal_count,qi-$Multiset[Int]_prog.native_equality,qi-$Multiset[Int]_prog.singleton_unionone,qi-quant-u-12,qi-quant-u-36,qi-quant-u-13,qi-$Multiset[Int]_prog.card_union,qi-$Multiset[Int]_prog.card_unionone,qi-$Multiset[Int]_prog.count_singleton,qi-$Multiset[Int]_prog.count_unionone,qi-$Multiset[Int]_prog.count_card,qi-$Multiset[Int]_prog.count_union,qi-$Multiset[Int]_prog.count_empty,qi-quant-u-14,qi-quant-u-38,qi-quant-u-15,qi-prog.getter_over_tuple2,qi-quant-u-22,qi-quant-u-23,qi-prog.integer_ax_dec,qi-k!569,qi-k!593,qi-k!599,qi-prog.integer_ax_bound,qi-quant-u-47,qi-quant-u-46,execution_time +1,107,62,54,24,76,242,242,75,72,75,114,72,72,71,12,28,15,15,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,43,1,1,1,83,39,159,160,561,178,48,1,1,1,3,2,1,9,136,81,81,6,1,1,11.116745948791504 +1,81,62,20,24,52,110,110,45,43,45,46,43,43,41,11,26,13,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,16,1,1,1,33,16,79,79,225,87,26,1,1,1,3,2,1,9,74,52,52,6,1,1,10.993242502212524 +1,99,64,44,24,72,211,211,67,64,67,94,64,64,63,14,28,17,16,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,34,1,1,1,67,34,152,154,555,154,57,1,1,1,3,2,1,9,124,75,75,6,1,1,10.966875076293945 +1,91,63,32,24,64,169,169,59,56,59,74,54,54,53,13,24,16,16,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,27,1,1,1,53,27,125,127,396,127,47,1,1,1,3,2,1,9,98,63,63,6,1,1,11.258367776870728 +1,79,62,16,24,56,103,103,47,44,47,42,44,44,43,12,24,15,14,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,14,1,1,1,27,14,76,77,205,74,28,1,1,1,3,2,1,9,68,48,48,6,1,1,11.156837701797485 +1,83,63,16,24,56,109,109,53,49,52,42,48,48,47,12,24,15,15,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,14,1,1,1,27,14,78,79,220,78,33,1,1,1,3,2,1,9,68,48,48,6,1,1,10.936635494232178 +1,79,62,18,24,52,111,111,47,44,47,46,44,44,42,11,24,13,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,16,1,1,1,33,16,80,80,225,90,28,1,1,1,3,2,1,9,72,50,50,6,1,1,10.922288179397583 +1,95,62,42,24,72,199,199,63,60,63,90,60,60,59,11,30,14,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,32,1,1,1,63,32,136,137,478,140,39,1,1,1,3,2,1,9,120,73,73,6,1,1,11.332127332687378 +1,85,63,18,24,56,117,117,53,50,53,46,48,48,47,12,24,15,15,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,16,1,1,1,31,16,85,86,244,88,33,1,1,1,3,2,1,9,72,50,50,6,1,1,10.73275899887085 +1,79,61,18,24,56,111,111,47,44,47,46,44,44,42,12,24,14,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,16,1,1,1,33,16,84,85,234,91,31,1,1,1,3,2,1,9,72,49,49,6,1,1,10.773168563842773 +1,79,62,18,24,56,111,111,47,44,47,46,44,44,43,11,24,13,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,16,1,1,1,31,16,87,84,229,84,30,1,1,1,3,2,1,9,72,49,49,6,1,1,11.07684588432312 +1,95,62,44,24,68,207,207,63,60,63,94,60,60,58,12,28,14,14,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,34,1,1,1,69,34,143,144,536,155,49,1,1,1,3,2,1,9,123,76,76,6,1,1,10.869197130203247 +1,79,62,20,24,56,111,111,47,44,47,46,44,44,43,11,28,13,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,16,1,1,1,31,16,87,88,228,84,28,1,1,1,3,2,1,9,76,51,51,6,1,1,11.096105337142944 +1,93,64,37,24,66,180,180,61,58,61,80,58,58,57,14,28,17,16,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,29,1,1,1,57,29,127,128,415,134,49,1,1,1,3,2,1,9,106,69,69,6,1,1,10.945177793502808 +1,79,61,18,24,56,110,110,46,43,46,46,43,43,42,11,24,14,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,16,1,1,1,31,16,84,84,227,84,28,1,1,1,3,2,1,9,72,49,49,6,1,1,10.8948495388031 +1,79,62,18,24,56,111,111,47,44,47,46,44,44,43,11,24,13,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,16,1,1,1,31,16,83,84,229,84,29,1,1,1,3,2,1,9,72,49,49,6,1,1,10.974491357803345 +1,89,63,31,24,62,158,158,57,54,57,68,54,54,53,13,28,16,16,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,25,1,1,1,49,25,121,123,374,125,42,1,1,1,3,2,1,9,98,62,62,6,1,1,11.06387186050415 +1,103,63,54,24,80,247,247,71,68,71,118,68,68,67,12,24,14,14,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,43,1,1,1,85,41,160,160,536,176,47,1,1,1,3,2,1,9,138,85,85,6,1,1,10.843992710113525 +1,107,62,54,24,80,249,249,77,74,77,114,72,72,71,12,28,15,15,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,41,1,1,1,81,41,161,162,547,179,45,1,1,1,3,2,1,9,140,85,85,6,1,1,11.332424402236938 +1,95,62,42,24,72,200,200,63,60,63,90,60,60,59,12,28,15,15,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,32,1,1,1,63,32,136,137,504,140,46,1,1,1,3,2,1,9,120,74,74,6,1,1,10.988781929016113 +1,85,65,20,24,56,109,109,52,48,51,42,48,48,47,12,30,15,15,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,14,1,1,1,27,14,78,79,220,78,33,1,1,1,3,2,1,9,76,54,54,6,1,1,11.061363220214844 +1,107,63,50,24,80,229,229,75,72,75,106,72,72,71,12,28,15,15,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,37,1,1,1,73,36,146,146,487,159,43,1,1,1,3,2,1,9,132,81,81,6,1,1,11.481862545013428 +1,107,64,50,24,80,235,235,75,72,75,110,72,72,71,13,24,16,16,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,39,1,1,1,77,37,160,161,518,172,45,1,1,1,3,2,1,9,132,81,81,6,1,1,10.774917602539062 +1,91,63,34,24,64,169,169,59,56,59,74,56,56,53,13,28,16,16,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,27,1,1,1,53,27,128,130,399,129,43,1,1,1,3,2,1,9,104,65,65,6,1,1,11.072274684906006 +1,87,62,29,24,62,156,156,55,52,55,68,52,52,51,11,24,14,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,25,1,1,1,49,25,112,113,361,121,37,1,1,1,3,2,1,9,91,60,60,6,1,1,10.918574810028076 +1,95,59,42,24,68,205,205,60,58,60,94,58,58,57,11,24,13,13,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,34,1,1,1,69,34,138,138,531,153,44,1,1,1,3,2,1,9,118,72,72,6,1,1,11.23032283782959 +1,95,63,44,24,68,207,207,63,60,63,94,60,60,58,12,28,14,14,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,34,1,1,1,69,34,143,144,531,155,49,1,1,1,3,2,1,9,122,76,76,6,1,1,11.07629919052124 +1,79,64,16,24,56,103,103,47,44,47,42,44,44,43,12,24,14,14,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,14,1,1,1,27,14,80,81,203,74,27,1,1,1,3,2,1,9,68,48,48,6,1,1,11.059718132019043 +1,89,62,32,24,64,167,167,57,54,57,74,54,54,53,12,24,14,14,34,34,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,27,1,1,1,53,27,122,123,390,127,39,1,1,1,3,2,1,9,100,63,63,6,1,1,10.986785173416138 +1,89,64,32,24,64,169,169,59,56,59,74,54,54,53,13,24,17,16,35,35,29,6,8,8,8,26,26,6,6,6,12,6,6,6,1,1,1,1,1,1,27,1,1,1,53,27,121,123,388,127,47,1,1,1,3,2,1,9,96,64,64,6,1,1,10.931391954421997 diff --git a/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..aa73d5e Binary files /dev/null and b/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..f95ff11 Binary files /dev/null and b/evaluation/experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/program_proofs_example_10_2/last_half/last_half.gobra b/evaluation/experiments/program_proofs_example_10_2/last_half/last_half.gobra new file mode 100644 index 0000000..bf0fed5 --- /dev/null +++ b/evaluation/experiments/program_proofs_example_10_2/last_half/last_half.gobra @@ -0,0 +1,110 @@ +// This package contains the last half of the proof for InsertCorrect from +// https://github.com/viperproject/program-proofs-gobra/blob/main/chapter10/examples_10.2.gobra +package last_half + +// Note that some parts that come after the induction hypothesis require +// definitions that originally came before the IH. +// Those dependencies are included at their original position. +ghost +requires pq.Valid() +ensures let pqPrime := pq.Insert(y) in + pqPrime.Valid() && + pqPrime.Elements() == pq.Elements() union mset[int]{y} +decreases len(pq) +pure func (pq PQueue) InsertCorrect(y int) Unit { + return match pq { + case Leaf{}: + Unit{} + case Node{?x, ?l, ?r}: + let pqPrime := pq.Insert(y) in + let max := y < x ? x : y in + let _ := r.InsertCorrect(max) in + let newRight := r.Insert(max) in + let _ := asserting(newRight.Valid()) in + let _ := asserting(newRight.IsBalanced()) in + let _ := asserting(l.IsBalanced()) in + let _ := asserting( + let Lprime := len(newRight.Elements()) in + let Rprime := len(l.Elements()) in + Lprime == Rprime || Lprime == Rprime + 1) in + let _ := asserting(pqPrime.IsBalanced()) in + asserting(pqPrime.IsBinaryHeap()) + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) Valid() bool { + return pq.IsBinaryHeap() && pq.IsBalanced() +} + +ghost +decreases len(pq) +pure func (pq PQueue) Insert(y int) PQueue { + return match pq { + case Leaf{}: + Node{y, Leaf{}, Leaf{}} + case Node{?x, ?left, ?right}: + y < x ? Node{y, right.Insert(x), left} : Node{x, right.Insert(y), left} + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) Elements() mset[int] { + return match pq { + case Leaf{}: + mset[int]{} + case Node{?x, ?left, ?right}: + mset[int]{x} union left.Elements() union right.Elements() + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) IsBalanced() (res bool) { + return match pq { + case Leaf{}: + true + case Node{_, ?left, ?right}: + left.IsBalanced() && right.IsBalanced() && + (let L := len(left.Elements()) in + let R := len(right.Elements()) in + L == R || L == R + 1) + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) IsBinaryHeap() bool { + return match pq { + case Leaf{}: + true + case Node{?x, ?left, ?right}: + left.IsBinaryHeap() && right.IsBinaryHeap() && + (left == Leaf{} || x <= left.x) && + (right == Leaf{} || x <= right.x) + } +} + + +type Unit struct{} + +ghost +requires b +decreases +pure func asserting(b bool) Unit { + return Unit{} +} + +type PQueue = BraunTree + +type BraunTree adt { + Leaf {} + + Node { + x int + left BraunTree + right BraunTree + } +} diff --git a/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..84727cf --- /dev/null +++ b/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!542,qi-k!553,qi-k!587,qi-k!565,qi-quant-u-8,qi-quant-u-9,qi-$Multiset[Int]_prog.card_non_negative,qi-$Multiset[Int]_prog.card_empty,qi-quant-u-27,qi-quant-u-2,qi-quant-u-26,qi-quant-u-3,qi-quant-u-0,qi-quant-u-24,qi-quant-u-1,qi-quant-u-10,qi-quant-u-11,qi-quant-u-35,qi-quant-u-34,qi-quant-u-4,qi-quant-u-28,qi-quant-u-5,qi-quant-u-41,qi-quant-u-16,qi-quant-u-40,qi-quant-u-17,qi-quant-u-33,qi-quant-u-32,qi-quant-u-39,qi-quant-u-14,qi-quant-u-38,qi-quant-u-15,qi-quant-u-6,qi-quant-u-30,qi-quant-u-7,qi-quant-u-12,qi-quant-u-36,qi-quant-u-13,qi-$Multiset[Int]_prog.equal_count,qi-$Multiset[Int]_prog.singleton_unionone,qi-$Multiset[Int]_prog.count_union,qi-$Multiset[Int]_prog.count_card,qi-$Multiset[Int]_prog.count_singleton,qi-$Multiset[Int]_prog.count_unionone,qi-$Multiset[Int]_prog.card_union,qi-$Multiset[Int]_prog.card_unionone,qi-$Multiset[Int]_prog.count_empty,qi-quant-u-22,qi-quant-u-45,qi-quant-u-23,qi-quant-u-44,qi-$Multiset[Int]_prog.native_equality,qi-prog.getter_over_tuple2,qi-quant-u-20,qi-quant-u-21,qi-prog.integer_ax_dec,qi-k!569,qi-k!593,qi-k!599,qi-prog.integer_ax_bound,execution_time +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,401,786,220,221,116,39,102,1,1,1,1,1,3,2,1,9,50,26,26,6,10.44386100769043 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,117,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.691684007644653 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.38664960861206 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,314,627,175,176,111,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.863882303237915 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,396,787,221,222,117,39,102,1,1,1,1,1,3,2,1,9,50,26,26,6,10.758392572402954 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,111,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.997275352478027 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.466723680496216 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.405782699584961 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,111,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.662888526916504 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.876100063323975 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,766,213,214,117,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.342998504638672 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,314,627,175,176,112,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.589011192321777 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,117,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.6397864818573 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.607729434967041 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,396,786,220,221,117,39,102,1,1,1,1,1,3,2,1,9,50,26,26,6,10.631652593612671 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.752532005310059 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,112,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.448531866073608 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,111,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.688621520996094 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,111,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.682292938232422 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,112,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.653276920318604 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.928860664367676 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,383,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.5341637134552 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,112,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.487468719482422 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,766,213,214,117,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.637656927108765 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,381,767,214,215,116,39,100,1,1,1,1,1,3,2,1,9,50,26,26,6,10.869438886642456 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,111,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.759708166122437 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,401,786,220,221,116,39,102,1,1,1,1,1,3,2,1,9,50,26,26,6,10.680810689926147 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,28,313,627,175,176,112,37,93,1,1,1,1,1,3,2,1,9,50,26,26,6,10.902204990386963 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,398,787,221,222,117,39,102,1,1,1,1,1,3,2,1,9,50,26,26,6,10.821024179458618 +1,48,24,7,13,24,11,11,11,11,11,11,11,11,11,7,15,7,7,20,20,18,2,2,2,2,13,13,1,1,1,1,1,1,1,1,1,1,13,30,401,787,221,222,116,39,102,1,1,1,1,1,3,2,1,9,50,26,26,6,10.5577552318573 diff --git a/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..8da75cd Binary files /dev/null and b/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..e505b41 Binary files /dev/null and b/evaluation/experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/program_proofs_example_10_2/minimal/minimal.gobra b/evaluation/experiments/program_proofs_example_10_2/minimal/minimal.gobra new file mode 100644 index 0000000..a0d98bf --- /dev/null +++ b/evaluation/experiments/program_proofs_example_10_2/minimal/minimal.gobra @@ -0,0 +1,97 @@ +// This package contains the minimal proof for InsertCorrect from +// https://github.com/viperproject/program-proofs-gobra/blob/main/chapter10/examples_10.2.gobra +package minimal + +// Note that we only need to instantiate the induction hypothesis. +ghost +requires pq.Valid() +ensures let pqPrime := pq.Insert(y) in + pqPrime.Valid() && + pqPrime.Elements() == pq.Elements() union mset[int]{y} +decreases len(pq) +pure func (pq PQueue) InsertCorrect(y int) Unit { + return match pq { + case Leaf{}: + Unit{} + case Node{?x, ?l, ?r}: + let max := y < x ? x : y in + r.InsertCorrect(max) + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) Valid() bool { + return pq.IsBinaryHeap() && pq.IsBalanced() +} + +ghost +decreases len(pq) +pure func (pq PQueue) Insert(y int) PQueue { + return match pq { + case Leaf{}: + Node{y, Leaf{}, Leaf{}} + case Node{?x, ?left, ?right}: + y < x ? Node{y, right.Insert(x), left} : Node{x, right.Insert(y), left} + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) Elements() mset[int] { + return match pq { + case Leaf{}: + mset[int]{} + case Node{?x, ?left, ?right}: + mset[int]{x} union left.Elements() union right.Elements() + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) IsBalanced() (res bool) { + return match pq { + case Leaf{}: + true + case Node{_, ?left, ?right}: + left.IsBalanced() && right.IsBalanced() && + (let L := len(left.Elements()) in + let R := len(right.Elements()) in + L == R || L == R + 1) + } +} + +ghost +decreases len(pq) +pure func (pq PQueue) IsBinaryHeap() bool { + return match pq { + case Leaf{}: + true + case Node{?x, ?left, ?right}: + left.IsBinaryHeap() && right.IsBinaryHeap() && + (left == Leaf{} || x <= left.x) && + (right == Leaf{} || x <= right.x) + } +} + + +type Unit struct{} + +ghost +requires b +decreases +pure func asserting(b bool) Unit { + return Unit{} +} + +type PQueue = BraunTree + +type BraunTree adt { + Leaf {} + + Node { + x int + left BraunTree + right BraunTree + } +} diff --git a/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..cf58822 --- /dev/null +++ b/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!1207,qi-$Set[Int]_prog.in_singleton_set,qi-quant-u-0,qi-quant-u-200,qi-quant-u-1,qi-$Set[Int]_prog.singleton_set_cardinality,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.in_singleton_set_equality,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-quant-u-2,qi-quant-u-205,qi-quant-u-203,qi-quant-u-3,qi-quant-u-202,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_union_in_one,qi-$Set[Int]_prog.in_empty_set,qi-quant-u-4,qi-quant-u-207,qi-quant-u-5,qi-quant-u-10,qi-quant-u-219,qi-quant-u-11,qi-quant-u-217,qi-quant-u-214,qi-quant-u-6,qi-quant-u-213,qi-quant-u-7,qi-quant-u-209,qi-k!872,qi-quant-u-8,qi-quant-u-215,qi-quant-u-9,qi-quant-u-14,qi-quant-u-15,qi-quant-u-222,qi-quant-u-221,qi-quant-u-12,qi-quant-u-220,qi-quant-u-13,qi-k!1050,qi-k!1082,qi-k!1045,qi-k!1071,qi-k!1144,qi-k!1139,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@887@12@890@61,qi-k!1080,qi-quant-u-225,qi-quant-u-224,qi-k!1148,qi-quant-u-19,qi-quant-u-230,qi-k!2173,qi-k!1122,qi-k!2167,qi-quant-u-16,qi-quant-u-228,qi-quant-u-226,qi-quant-u-227,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@891@12@894@48,qi-quant-u-237,qi-quant-u-20,qi-quant-u-236,qi-quant-u-21,qi-quant-u-250,qi-quant-u-24,qi-quant-u-249,qi-quant-u-31,qi-quant-u-259,qi-quant-u-25,qi-quant-u-245,qi-k!3209,qi-quant-u-22,qi-quant-u-243,qi-quant-u-23,qi-quant-u-238,qi-quant-u-26,qi-quant-u-253,qi-$Set[Int]_prog.subset_definition,qi-k!862,qi-quant-u-28,qi-quant-u-257,qi-quant-u-256,qi-quant-u-29,qi-quant-u-255,qi-quant-u-244,qi-quant-u-44,qi-quant-u-45,qi-quant-u-279,qi-k!3990,qi-quant-u-251,qi-quant-u-252,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@764@12@766@56,qi-quant-u-34,qi-quant-u-267,qi-quant-u-35,qi-quant-u-266,qi-quant-u-32,qi-quant-u-265,qi-quant-u-33,qi-quant-u-263,qi-quant-u-46,qi-quant-u-286,qi-quant-u-47,qi-quant-u-284,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@1115@12@1117@62,qi-quant-u-272,qi-quant-u-36,qi-quant-u-271,qi-quant-u-37,qi-quant-u-269,qi-quant-u-49,qi-quant-u-289,qi-k!4699,qi-k!4707,qi-k!1131,qi-quant-u-41,qi-quant-u-40,qi-quant-u-275,qi-k!3889,qi-k!1098,qi-quant-u-278,qi-quant-u-42,qi-quant-u-277,qi-quant-u-43,qi-quant-u-283,qi-quant-u-282,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@913@6@916@41,qi-quant-u-288,qi-quant-u-287,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@286@13@288@54,qi-quant-u-296,qi-quant-u-50,qi-quant-u-295,qi-quant-u-51,qi-quant-u-274,qi-quant-u-38,qi-quant-u-273,qi-quant-u-39,qi-$Set[Int]_prog.union_right_idempotency,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@705@12@706@74,qi-quant-u-262,qi-quant-u-30,qi-quant-u-261,qi-$Set[Int]_prog.intersection_right_idempotency,qi-$Set[Int]_prog.intersection_left_idempotency,qi-$Multiset[Int]_prog.count_card,qi-$Multiset[Int]_prog.card_non_negative,qi-$Multiset[Int]_prog.card_empty,qi-quant-u-120,qi-$Multiset[Int]_prog.singleton_unionone,qi-$Multiset[Int]_prog.count_union,qi-$Multiset[Int]_prog.card_union,qi-quant-u-121,qi-quant-u-428,qi-$Multiset[Int]_prog.count_singleton,qi-$Multiset[Int]_prog.count_unionone,qi-$Multiset[Int]_prog.count_empty,qi-k!8394,qi-k!8400,qi-$Multiset[Int]_prog.card_unionone,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@1042@12@1044@47,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@404@6@406@68,qi-k!1102,qi-quant-u-146,qi-quant-u-147,qi-quant-u-476,qi-$Set[Int]_prog.union_left_idempotency,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@1204@12@1206@25,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_not_opaque/dicts_not_opaque.gobra.vpr@714@12@715@20,qi-k!1604,qi-k!1598,qi-prog.getter_over_tuple2,qi-quant-u-150,qi-quant-u-151,qi-k!1869,qi-k!2061,qi-prog.set_ax_dec,qi-prog.set_ax_bound,qi-prog.integer_ax_dec,qi-prog.integer_ax_bound,execution_time +1,24,79,74,79,10,516,516,506,82,69,28,28,24,26,24,928,784,78,114,3146,1234,1364,194,42,51,46,47,20,20,20,20,18,38,38,37,36,70,6,6,6,1,2,2,12,12,12,12,44,44,44,2,1081,212,5,1,1,1,110,11,11,2,141,17,5,5,4,4,3,13,13,13,11,13,13,13,33,33,13,13,24,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,13,10,12,1,1,1,1,1,6,6,6,6,44,1,1,1,1,1,7,7,16,26,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,93,9,3,2,1,4,8,10,9,2,2,18.252676486968994 +1,24,88,81,87,10,538,536,596,86,72,29,29,24,26,24,1030,936,83,119,3529,1388,1547,200,59,51,46,47,20,20,20,20,16,36,36,35,35,92,6,6,6,1,2,2,12,12,12,12,47,46,46,2,1237,245,5,1,1,1,135,12,12,2,164,15,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,1,1,1,4,4,4,4,44,1,1,1,1,1,7,7,17,27,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,114,9,3,2,1,4,14,9,9,2,2,18.373303413391113 +1,23,86,79,86,10,537,537,554,84,70,29,29,23,25,23,1000,873,82,119,3358,1324,1489,192,49,51,46,47,20,20,20,20,18,38,38,36,36,78,6,6,6,1,2,2,12,12,12,12,46,45,45,8,1172,220,5,1,1,1,114,11,11,2,154,15,5,5,4,4,3,13,13,13,11,13,13,13,34,34,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,10,7,8,1,1,1,1,1,6,6,6,6,49,1,1,1,1,1,7,7,16,26,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,101,9,3,2,1,4,2,10,9,2,2,18.887043237686157 +1,24,80,73,79,10,512,512,489,80,67,29,29,24,26,24,900,760,79,112,2941,1172,1324,200,39,51,46,47,20,20,20,20,18,38,38,37,37,65,6,6,6,1,2,2,12,12,12,12,45,45,45,2,1054,206,5,1,1,1,102,12,12,2,147,15,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,8,5,7,1,1,1,1,1,4,4,4,4,46,1,1,1,1,1,7,7,14,24,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,83,7,3,2,1,18,8,10,9,2,2,18.888944625854492 +1,23,80,73,80,10,512,512,511,81,66,28,28,23,25,23,928,850,74,111,3011,1208,1357,182,27,51,46,47,20,20,20,20,17,37,37,35,35,98,6,6,6,1,2,2,12,12,12,12,44,44,44,8,1166,227,5,1,1,1,123,11,11,2,176,15,5,5,4,4,3,13,13,13,11,13,13,13,34,34,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,1,1,1,4,4,4,4,48,1,1,1,1,1,7,7,14,24,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,121,9,3,2,1,4,8,9,9,2,2,18.527926921844482 +1,24,81,76,80,10,529,529,525,83,71,28,28,24,26,24,965,780,79,116,3238,1322,1401,204,54,51,46,47,20,20,20,20,16,36,36,35,35,63,6,6,6,1,2,2,12,12,12,12,45,45,45,1,1098,217,5,1,1,1,108,12,12,2,136,15,5,5,4,4,3,13,13,13,12,13,13,13,30,30,13,13,6,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,11,8,10,1,1,1,1,1,6,6,6,6,48,1,1,1,1,1,7,7,13,23,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,85,9,3,2,1,4,14,9,9,2,2,19.114484071731567 +1,23,79,73,78,10,518,517,536,85,70,30,30,23,25,23,950,851,77,112,3237,1293,1422,191,71,51,46,47,20,20,20,20,16,36,36,34,34,81,6,6,6,1,2,2,12,12,12,12,46,45,45,8,1141,229,5,1,1,1,114,11,11,2,153,15,5,5,4,4,3,13,13,13,11,13,13,13,33,33,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,8,5,7,1,1,1,1,1,4,4,4,4,47,1,1,1,1,1,7,7,13,23,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,103,9,3,2,1,4,14,9,9,2,2,18.09272789955139 +1,24,88,80,87,10,532,532,617,87,74,28,28,24,26,24,1004,887,75,118,3644,1421,1539,200,102,51,46,47,20,20,20,20,16,36,36,35,35,73,6,6,6,1,2,2,12,12,12,12,45,45,45,1,1257,240,5,1,1,1,127,12,12,2,161,15,5,5,4,4,3,13,13,13,12,13,13,13,35,35,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,10,7,9,1,1,1,1,1,4,4,4,4,45,1,1,1,1,1,7,7,17,27,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,95,9,3,2,1,4,14,9,9,2,2,19.181499242782593 +1,24,83,78,83,10,515,515,568,82,68,28,28,24,26,24,945,849,75,111,3110,1275,1377,209,66,51,46,47,20,20,20,20,17,37,37,36,36,70,6,6,6,1,2,2,12,12,12,12,45,45,45,3,1152,224,5,1,1,1,112,12,12,2,156,15,5,5,4,4,3,13,13,13,12,13,13,13,33,33,13,13,9,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,8,5,7,1,1,1,1,1,4,4,4,4,49,1,1,1,1,1,7,7,14,24,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,92,9,3,2,1,4,14,10,9,2,2,19.31056237220764 +1,23,81,74,81,10,517,517,557,84,71,29,29,23,25,23,953,860,75,113,3472,1318,1456,198,83,51,46,47,20,20,20,20,17,37,37,35,35,92,6,6,6,1,2,2,12,12,12,12,44,44,44,1,1213,231,5,1,1,1,123,11,11,2,163,16,5,5,4,4,3,13,13,13,11,13,13,13,36,35,13,13,24,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,10,7,9,1,1,1,1,1,6,6,6,6,45,1,1,1,1,1,7,7,16,26,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,114,9,3,2,1,4,8,10,9,2,2,19.14799475669861 +1,24,89,82,89,10,543,542,585,89,73,29,29,24,26,24,995,912,82,120,3491,1334,1523,197,57,51,46,47,20,20,20,20,17,37,37,36,36,72,6,6,6,1,2,2,12,12,12,12,47,46,46,2,1208,228,5,1,1,1,116,12,12,2,165,15,5,5,4,4,3,13,13,13,12,13,13,13,35,35,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,13,10,12,1,1,1,1,1,6,6,6,6,51,1,1,1,1,1,7,7,12,22,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,94,9,3,2,1,4,2,10,9,2,2,19.168535947799683 +1,23,86,80,86,10,531,531,538,84,71,28,28,23,25,23,985,886,78,117,3392,1322,1531,225,61,51,46,47,20,20,20,20,16,36,36,34,34,96,6,6,6,1,2,2,12,12,12,12,46,45,45,8,1223,235,5,1,1,1,125,11,11,2,172,15,5,5,4,4,3,13,13,13,11,13,13,13,35,35,13,13,8,6,6,6,6,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,1,1,1,4,4,4,4,46,1,1,1,1,1,7,7,15,25,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,120,10,3,2,1,5,14,9,9,2,2,18.644120693206787 +1,23,80,76,78,10,511,510,517,83,71,30,30,23,25,23,919,810,76,112,3316,1280,1396,191,81,51,46,47,20,20,20,20,16,36,36,34,34,67,6,6,6,1,2,2,12,12,12,12,44,44,44,2,1111,216,5,1,1,1,107,11,11,2,138,15,5,5,4,4,3,13,13,13,11,13,13,13,31,31,13,13,6,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,8,5,7,1,1,1,1,1,4,4,4,4,43,1,1,1,1,1,7,7,16,26,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,89,9,3,2,1,4,8,9,9,2,2,18.32518434524536 +1,24,80,75,80,10,520,520,525,82,69,29,29,24,26,24,943,874,79,115,3282,1266,1448,215,67,51,46,47,20,20,20,20,18,38,38,37,37,90,6,6,6,1,2,2,12,12,12,12,47,46,46,2,1140,226,5,1,1,1,122,12,12,2,156,15,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,7,6,6,6,6,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,1,1,1,4,4,4,4,49,1,1,1,1,1,7,7,12,22,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,114,10,3,2,1,5,8,10,9,2,2,19.096994638442993 +1,24,84,77,84,10,526,526,571,80,67,28,28,24,26,24,1013,890,80,116,3278,1280,1460,189,37,51,46,47,20,20,20,20,16,36,36,35,35,102,6,6,6,1,2,2,12,12,12,12,45,45,45,2,1194,229,5,1,1,1,123,12,12,2,169,15,5,5,4,4,3,13,13,13,12,13,13,13,35,35,13,13,9,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,10,7,9,1,1,1,1,1,4,4,4,4,47,1,1,1,1,1,7,7,13,23,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,125,9,3,2,1,6,14,9,9,2,2,19.092406749725342 +1,24,82,77,82,10,507,505,475,78,65,30,30,24,26,24,858,706,79,111,2747,1098,1240,178,22,51,46,47,20,20,20,20,16,36,36,35,35,76,6,6,6,1,2,2,12,12,12,12,45,45,45,1,1045,204,5,1,1,1,107,12,12,2,148,15,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,1,1,1,4,4,4,4,50,1,1,1,1,1,7,7,14,24,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,95,7,3,2,1,4,2,9,9,2,2,18.24685263633728 +1,23,89,80,89,10,533,532,606,90,77,29,29,23,25,23,986,923,78,117,3549,1394,1501,196,88,51,46,47,20,20,20,20,17,37,37,35,35,79,6,6,6,1,2,2,12,12,12,12,46,45,45,13,1231,234,5,1,1,1,120,11,11,2,174,17,5,5,4,4,3,13,13,13,11,13,13,13,35,35,13,13,24,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,13,10,12,1,1,1,1,1,6,6,6,6,44,1,1,1,1,1,7,7,17,27,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,101,9,3,2,1,18,14,10,9,2,2,18.8912353515625 +1,23,77,71,76,10,496,495,489,86,74,30,30,23,25,23,858,703,74,108,2979,1136,1310,182,85,51,46,47,20,20,20,20,18,38,38,36,36,70,6,6,6,1,2,2,12,12,12,12,44,44,44,2,1051,204,5,1,1,1,103,11,11,2,147,15,5,5,4,4,3,13,13,13,11,13,13,13,31,31,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,1,1,1,4,4,4,4,46,1,1,1,1,1,7,7,14,24,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,88,7,3,2,1,18,8,10,9,2,2,19.26409125328064 +1,22,77,69,76,10,495,494,535,76,64,29,29,22,24,22,896,754,72,108,2946,1123,1318,175,35,51,46,47,20,20,20,20,18,38,38,35,35,72,6,6,6,1,2,2,12,12,12,12,43,43,43,2,1050,208,5,1,1,1,112,10,10,2,150,15,5,5,4,4,3,13,13,13,10,13,13,13,34,34,13,13,6,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,1,1,1,4,4,4,4,49,1,1,1,1,1,7,7,15,25,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,93,9,3,2,1,5,14,10,9,2,2,18.875189542770386 +1,24,84,77,84,10,527,527,558,86,72,29,29,24,26,24,945,855,80,116,3142,1237,1398,188,70,51,46,47,20,20,20,20,16,36,36,35,35,80,6,6,6,1,2,2,12,12,12,12,47,46,46,2,1154,227,5,1,1,1,122,12,12,2,165,15,5,5,4,4,3,13,13,13,12,13,13,13,35,35,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,10,7,9,1,1,1,1,1,5,5,5,5,48,1,1,1,1,1,7,7,13,23,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,103,9,3,2,1,5,14,9,9,2,2,18.075374603271484 +1,24,80,73,78,10,519,518,532,79,66,29,29,24,26,24,950,828,79,114,3042,1221,1363,206,32,51,46,47,20,20,20,20,17,37,37,36,36,80,6,6,6,1,2,2,12,12,12,12,45,45,45,2,1137,214,5,1,1,1,113,12,12,2,156,15,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,8,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,9,6,8,1,1,1,1,1,5,5,5,5,42,1,1,1,1,1,7,7,17,27,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,103,9,3,2,1,5,14,9,9,2,2,18.14431118965149 +1,24,88,79,88,10,530,529,595,90,77,30,30,24,26,24,984,877,78,115,3541,1378,1484,188,85,51,46,47,20,20,20,20,17,37,37,36,36,94,6,6,6,1,2,2,12,12,12,12,45,45,45,2,1225,236,5,1,1,1,126,12,12,2,166,16,5,5,4,4,3,13,13,13,12,13,13,13,36,36,13,13,24,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,11,8,10,1,1,1,1,1,6,6,6,6,49,1,1,1,1,1,7,7,13,23,2,3,3,3,8,5,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,2,2,4,4,4,2,2,3,115,7,3,2,1,18,14,10,9,2,2,18.972140789031982 +1,24,83,78,83,10,511,511,531,78,66,29,29,24,26,24,934,824,78,111,3078,1247,1376,202,51,51,46,47,20,20,20,20,17,37,37,36,36,82,6,6,6,1,2,2,12,12,12,12,45,45,45,2,1114,221,5,1,1,1,117,12,12,2,138,15,5,5,4,4,3,13,13,13,12,13,13,13,35,35,13,13,7,5,5,5,5,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,10,7,9,1,1,1,1,1,6,6,6,6,47,1,1,1,1,1,8,8,14,24,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,102,8,3,2,1,4,14,9,9,2,2,19.023730754852295 +1,23,75,69,74,10,488,488,456,78,65,28,28,23,25,23,820,712,73,107,2665,1029,1179,169,32,51,46,47,20,19,19,19,18,37,37,36,36,77,6,6,6,1,2,2,12,12,12,12,44,44,44,2,1006,199,5,1,1,1,106,11,11,2,157,15,5,5,4,4,3,12,12,12,11,13,13,13,33,33,12,12,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,12,9,11,1,1,1,1,1,5,5,5,5,48,1,1,1,1,1,7,7,13,23,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,100,9,3,2,1,4,8,10,9,2,2,18.92489194869995 +1,23,76,72,76,10,503,502,507,80,65,29,29,24,26,23,942,750,74,109,3123,1216,1387,192,41,51,46,47,20,20,20,20,16,36,36,34,34,57,6,6,6,1,2,2,12,12,12,12,44,44,44,2,1074,204,5,1,1,1,100,11,11,2,127,15,5,5,4,4,3,13,13,13,11,13,13,13,30,30,13,13,6,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,10,7,9,1,1,1,1,1,6,6,6,6,45,1,1,1,1,1,7,7,16,26,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,79,9,3,2,1,4,14,9,9,2,2,18.98514676094055 +1,24,84,80,84,10,533,532,548,94,79,29,29,24,26,24,935,798,81,116,3306,1270,1433,195,110,51,46,47,20,20,20,20,17,37,37,36,36,79,6,6,6,1,2,2,12,12,12,12,47,46,46,2,1147,219,5,1,1,1,108,12,12,2,152,15,5,5,4,4,3,13,13,13,12,13,13,13,30,30,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,1,1,1,4,4,4,4,48,1,1,1,1,1,7,7,15,25,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,102,9,3,2,1,4,14,10,9,2,2,18.909767627716064 +1,23,77,70,77,10,509,509,518,80,67,29,29,23,25,23,898,813,77,113,2984,1135,1308,181,37,51,46,47,20,20,20,20,16,36,36,34,34,80,6,6,6,1,2,2,12,12,12,12,43,44,44,2,1056,207,5,1,1,1,109,10,10,2,157,15,5,5,4,4,3,13,13,13,10,13,13,13,34,34,13,13,6,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,12,9,10,1,1,1,1,1,6,6,6,6,52,1,1,1,1,1,7,7,14,24,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,103,9,3,2,1,4,2,9,9,2,2,18.226284503936768 +1,23,83,76,83,10,528,528,530,87,72,29,29,24,26,23,920,783,79,118,3281,1269,1406,190,83,51,46,47,20,20,20,20,17,37,37,35,35,68,6,6,6,1,2,2,12,12,12,12,44,44,44,2,1150,220,5,1,1,1,115,11,11,2,154,15,5,5,4,4,3,13,13,13,11,13,13,13,34,34,13,13,7,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,13,10,12,1,1,1,1,1,6,6,6,6,48,1,1,1,1,1,7,7,15,25,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,90,9,3,2,1,4,14,10,9,2,2,18.4680016040802 +1,23,85,79,85,10,518,518,552,78,66,28,28,24,26,23,998,893,74,113,3412,1345,1524,227,55,51,46,47,20,20,20,20,16,36,36,34,34,94,6,6,6,1,2,2,12,12,12,12,44,44,44,9,1225,233,5,1,1,1,125,11,11,2,161,15,5,5,4,4,3,13,13,13,11,13,13,13,34,34,13,13,7,6,6,6,6,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,2,11,8,10,1,1,1,1,1,5,5,5,5,47,1,1,1,1,1,8,8,15,25,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,117,9,3,2,1,4,14,9,9,2,2,18.408321857452393 +1,23,85,76,85,10,512,512,537,80,67,28,28,23,25,23,953,819,73,112,3220,1268,1414,188,36,51,46,47,20,20,20,20,17,37,37,35,35,99,6,6,6,1,2,2,12,12,12,12,44,44,44,8,1203,226,5,1,1,1,124,11,11,2,185,16,5,5,4,4,3,13,13,13,11,13,13,13,36,36,13,13,24,4,4,4,4,5,5,11,1,1,1,1,1,1,2,4,6,6,2,2,2,3,10,7,9,1,1,1,1,1,6,6,6,6,51,1,1,1,1,1,7,7,13,23,2,3,3,3,8,3,5,5,5,5,2,2,1,1,1,2,1,1,1,1,1,1,1,1,6,1,8,8,8,2,1,4,2,2,2,2,10,4,2,2,8,8,7,3,3,2,1,1,2,4,4,4,2,2,3,122,9,3,2,1,18,8,10,9,2,2,18.606770277023315 diff --git a/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..2f10793 Binary files /dev/null and b/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..facde77 Binary files /dev/null and b/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque.gobra b/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque.gobra new file mode 100644 index 0000000..a96decb --- /dev/null +++ b/evaluation/experiments/standard_library/dicts_not_opaque/dicts_not_opaque.gobra @@ -0,0 +1,457 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// A copy of the dicts package from the standard library. +// Lemmas are turned non-opaque. Uses the non-opaque version of the sets +// package. +package dicts_not_opaque + +// ##(-I ./..) + +import "util" +import "sets_not_opaque" + +// A dictionary is empty if its domain is empty. +ghost +decreases +pure func IsEmpty(d dict[int]int) bool { + return sets_not_opaque.IsEmpty(domain(d)) +} + +// Returns the empty dictionary. +ghost +ensures IsEmpty(result) +decreases +pure func Empty() (result dict[int]int) { + return dict[int]int{} +} + +// Retrieves the value associated with the key, if present, as an option. +ghost +decreases +pure func Get(d dict[int]int, k int) option[int] { + return k in domain(d) ? some(d[k]) : none[int] +} + +// Keep all key-value pairs corresponding to the set of keys provided. +ghost +opaque +ensures forall x int :: {result[x]} (x in domain(d) && x in xs) ==> + (x in domain(result) && result[x] == d[x]) +ensures forall x int :: {x in domain(result)} x in domain(result) ==> + (x in domain(d) && x in xs) +ensures domain(result) == xs intersection domain(d) +decreases xs +pure func Restrict(d dict[int]int, xs set[int]) (result dict[int]int) { + return let ys := (xs intersection domain(d)) in + (sets_not_opaque.IsEmpty(ys) ? Empty() : + (let y := sets_not_opaque.Choose(ys) in + (let yr := sets_not_opaque.Remove(ys, y) in + (let _ := sets_not_opaque.IntersectLenUpper(xs, domain(d)) in + Restrict(d, yr)[y = d[y]])))) +} + +// Remove all key-value pairs corresponding to the set of keys provided. +ghost +decreases +pure func RemoveKeys(d dict[int]int, xs set[int]) (result dict[int]int) { + return Restrict(d, domain(d) setminus xs) +} + +// Remove a key-value pair. Returns d if k is not in the domain of d. +ghost +ensures len(result) <= len(d) +ensures k in domain(d) ==> len(result) == len(d) - 1 +ensures !(k in domain(d)) ==> len(result) == len(d) +decreases +pure func Remove(d dict[int]int, k int) (result dict[int]int) { + return let ys := (sets_not_opaque.Remove(domain(d), k)) in + let _ := util.Asserting(ys intersection domain(d) == ys) in + RemoveKeys(d, sets_not_opaque.SingletonSet(k)) +} + +// True iff k maps to the same value or is not in the domains of d1 and d2. +ghost +decreases +pure func IsEqualOnKey(d1, d2 dict[int]int, k int) bool { + return !(k in domain(d1) || k in domain(d2)) || + (k in domain(d1) && k in domain(d2) && d1[k] == d2[k]) +} + +// True iff if d1 is a subset of d2. +ghost +decreases +pure func IsSubset(d1, d2 dict[int]int) bool { + return domain(d1) subset domain(d2) && + forall k int :: {IsEqualOnKey(d1, d2, k)} {k in domain(d1)} (k in domain(d1)) ==> + IsEqualOnKey(d1, d2, k) +} + +// Union of two dictionaries. Does not require disjoint domains: on the intersection, +// values from the second dictionary are chosen. +ghost +opaque +ensures domain(result) == domain(d1) union domain(d2) +ensures forall k int :: {result[k]} (k in domain(d2)) ==> result[k] == d2[k] +ensures forall k int :: {result[k]} (!(k in domain(d2)) && k in domain(d1)) ==> + result[k] == d1[k] +decreases domain(d1) union domain(d2) +pure func Union(d1, d2 dict[int]int) (result dict[int]int) { + return let ks := domain(d1) union domain(d2) in + (sets_not_opaque.IsEmpty(ks) ? Empty() : + let k := sets_not_opaque.Choose(ks) in + let c1 := Remove(d1, k) in + let c2 := Remove(d2, k) in + let _ := sets_not_opaque.RemoveUnionLen(domain(d1), domain(d2), k) in + (k in domain(d2) ? Union(c1, c2)[k = d2[k]] : Union(c1, c2)[k = d1[k]])) +} + +// Dictionaries are disjoint iff their domains are disjoint. +ghost +decreases +pure func AreDisjoint(d1, d2 dict[int]int) bool { + return sets_not_opaque.AreDisjoint(domain(d1), domain(d2)) +} + +// The length of the union of two disjoint dictionaries is the sum of each of their lengths. +ghost +requires AreDisjoint(d1, d2) +ensures len(Union(d1, d2)) == len(d1) + len(d2) +decreases +pure func DisjointUnionLen(d1, d2 dict[int]int) util.Unit { + return util.Unit{} +} + +// True iff a dictionary is injective. +ghost +opaque +decreases +pure func IsInjective(d dict[int]int) bool { + return forall k1, k2 int :: {d[k1], d[k2]} (k1 != k2 && k1 in domain(d) && k2 in domain(d)) ==> + d[k1] != d[k2] +} + +// True iff a dictionary contains all valid keys. +ghost +opaque +decreases +pure func IsTotal(d dict[int]int) bool { + return forall k int :: {k in domain(d)} k in domain(d) +} + +// True iff a dictionary is monotonic. +ghost +opaque +decreases +pure func IsMonotonic(d dict[int]int) bool { + return forall k1, k2 int :: {d[k1], d[k2]} (k1 in domain(d) && k2 in domain(d) && k1 <= k2) ==> + d[k1] <= d[k2] +} + +// True iff a dictionary is monotonic. Only considers keys greater than or equal to start. +ghost +opaque +decreases +pure func IsMonotonicFrom(d dict[int]int, start int) bool { + return forall k1, k2 int :: {d[k1], d[k2]} (k1 in domain(d) && k2 in domain(d) && start <= k1 && k1 <= k2) ==> + d[k1] <= d[k2] +} + +// True iff a dictionary is monotonic. Only considers keys in the interval [start, end). +ghost +opaque +decreases +pure func IsMonotonicFromTo(d dict[int]int, start, end int) bool { + return forall k1, k2 int :: {d[k1], d[k2]} (k1 in domain(d) && + k2 in domain(d) && + start <= k1 && k1 <= k2 && k2 < end) ==> d[k1] <= d[k2] +} + +// True iff two dictionaries are equal in the interval [start, end). +ghost +opaque +decreases +pure func IsEqualInRange(d1, d2 dict[int]int, start, end int) bool { + return forall k int :: {d1[k], d2[k]} {k in domain(d1)} {k in domain(d2)}(start <= k && k < end) ==> + k in domain(d1) && k in domain(d2) && d1[k] == d2[k] +} + +// True iff d1 and d2 agree on all keys that their domains share. +ghost +decreases +pure func Agree(d1, d2 dict[int]int) bool { + return forall k int :: {d1[k], d2[k]} k in (domain(d1) intersection domain(d2)) ==> + d1[k] == d2[k] +} + +// The domain of a map after removing a key is equivalent to removing +// the key from the domain of the original map. +ghost +ensures domain(Remove(d, k)) == sets_not_opaque.Remove(domain(d), k) +decreases +pure func RemoveDomain(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// The domain of the empty dictionary is the empty set. +ghost +requires IsEmpty(d) +ensures domain(d) == sets_not_opaque.Empty() +decreases +pure func EmptyDictEmptyDomain(d dict[int]int) util.Unit { + return util.Unit{} +} + +// The domain of a dictionary after inserting a key-value pair is equivalent to +// inserting the key into the original map's domain set. +ghost +ensures domain(d[k = v]) == sets_not_opaque.Add(domain(d), k) +decreases +pure func InsertDomain(d dict[int]int, k, v int) util.Unit { + return util.Unit{} +} + +// Inserting value at k in d results in a dictionary that maps k to v. +ghost +ensures d[k = v][k] == v +decreases +pure func UpdateSame(d dict[int]int, k, v int) util.Unit { + return util.Unit{} +} + +// Reassigning the corresponding value to a key does not change the dictionary. +ghost +requires k in domain(d) +requires v == d[k] +ensures d[k = v] == d +decreases +pure func UpdateEqual(d dict[int]int, k, v int) util.Unit { + return util.Unit{} +} + +// Inserting v at k2 does not change the value mapped to by any other keys in d. +ghost +requires k1 != k2 +ensures k2 in domain(d[k1 = v]) == k2 in domain(d) +ensures k2 in domain(d) ==> d[k1 = v][k2] == d[k2] +decreases +pure func UpdateDifferent(d dict[int]int, k1, k2, v int) util.Unit { + return util.Unit{} +} + +// Removing a key-value pair from a dictionary does not change the value mapped to +// by any other keys in the map. +ghost +requires k1 in domain(d) +requires k1 != k2 +ensures Remove(d, k2)[k1] == d[k1] +decreases +pure func RemoveDifferent(d dict[int]int, k1, k2 int) util.Unit { + return util.Unit{} +} + +// Two maps are equivalent if their domains are equivalent and every key in their +// domains map to the same value. +ghost +ensures (d1 == d2) == + (domain(d1) == domain(d2) && forall k int :: {d1[k], d2[k]} k in domain(d1) ==> d1[k] == d2[k]) +decreases +pure func ExtEqual(d1, d2 dict[int]int) util.Unit { + return util.Unit{} +} + +// The cardinality of a dictionary is non-negative. +ghost +ensures len(d) >= 0 +decreases +pure func NonNegativeLen(d dict[int]int) util.Unit { + return util.Unit{} +} + +// The cardinality of a dictionary is equal to the cardinality of its domain. +ghost +ensures len(d) == len(domain(d)) +decreases +pure func DomainLenEq(d dict[int]int) util.Unit { + return util.Unit{} +} + +// If two dictionaries are disjoint there is no key that is in both of their domains. +ghost +requires AreDisjoint(d1, d2) +ensures !(k in domain(d1) && k in domain(d2)) +decreases +pure func DisjointNoSharedKey(d1, d2 dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// If two dictionaries are not disjoint, there exists a key that is in both of their domains. +ghost +requires !AreDisjoint(d1, d2) +ensures exists k int :: {k in domain(d1), k in domain(d2)} k in domain(d1) && k in domain(d2) +decreases +pure func NotDisjointSharedKey(d1, d2 dict[int]int) util.Unit { + return util.Unit{} +} + +// There is only one empty map. +ghost +requires IsEmpty(d) +ensures d == Empty() +decreases +pure func EmptyIsUnique(d dict[int]int) util.Unit { + return util.Unit{} +} + +// An empty dictionary contains no keys. +ghost +requires IsEmpty(d) +ensures !(k in domain(d)) +decreases +pure func NotInEmpty(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// There exists a key in a non-empty dictionary. +ghost +requires !IsEmpty(d) +ensures exists k int :: {k in domain(d)} k in domain(d) +decreases +pure func NotEmptyKeyExists(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// If a key is in d, d is not empty. +ghost +requires k in domain(d) +ensures !IsEmpty(d) +decreases +pure func KeyInDomainDictNotEmpty(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// Inserting a new key increases the cardinality of the dictionary by 1. +// Updating the value of a key does not change the cardinality. +ghost +ensures k in domain(d) ==> len(d[k = v]) == len(d) +ensures !(k in domain(d)) ==> len(d[k = v]) == len(d) + 1 +decreases +pure func InsertUpdateLen(d dict[int]int, k, v int) util.Unit { + return util.Unit{} +} + +// If a value is in the range of a dictionary, there exists a corresponding key. +ghost +requires v in range(d) +ensures exists k int :: {k in domain(d)} k in domain(d) && d[k] == v +decreases +pure func ValueHasKey(d dict[int]int, v int) util.Unit { + return util.Unit{} +} + +// If a value is in the domain of a dictionary, the corresponding value is in its range. +ghost +requires k in domain(d) +ensures d[k] in range(d) +decreases +pure func KeyMapsToRange(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// Returns a dictionary with the values at k1 and k2 swapped. +ghost +requires k1 in domain(d) && k2 in domain(d) +ensures domain(result) == domain(d) +ensures result[k1] == d[k2] && result[k2] == d[k1] +ensures forall k int :: {result[k]} (k in domain(d) && k != k1 && k != k2) ==> + result[k] == d[k] +decreases +pure func Swap(d dict[int]int, k1, k2 int) (result dict[int]int) { + return d[k1 = d[k2]][k2 = d[k1]] +} + +// Returns the set of keys from the given dictionary that map to the specified value. +ghost +opaque +ensures forall k int :: {k in result} (k in domain(d) && d[k] == v) == (k in result) +decreases len(d) +pure func Keys(d dict[int]int, v int) (result set[int]) { + return IsEmpty(d) ? sets_not_opaque.Empty() : + (let x := sets_not_opaque.Choose(domain(d)) in + (let subKeys := Keys(Remove(d, x), v) in + (d[x] == v ? sets_not_opaque.Add(subKeys, x) : subKeys))) + +} + +// Returns the number of occurences of the value in the dictionary. +ghost +decreases +pure func Occurrences(d dict[int]int, v int) int { + return len(Keys(d, v)) +} + +// Remove preserves the injectivity of a dictionary. +ghost +requires IsInjective(d) +ensures IsInjective(Remove(d, k)) +decreases +pure func RemovePreservesInjectivity(d dict[int]int, k int) util.Unit { + return let _ := reveal IsInjective(d) in + let _ := reveal IsInjective(Remove(d, k)) in + util.Unit{} +} + +// After removing a key, the corresponding value occurs one less time. +ghost +requires k in domain(d) +ensures Occurrences(Remove(d, k), d[k]) == Occurrences(d, d[k]) - 1 +decreases +pure func RemoveOccurrences(d dict[int]int, k int) util.Unit { + return let ks1 := reveal Keys(d, d[k]) in + let ks2 := reveal Keys(Remove(d, k), d[k]) in + util.Asserting(ks2 == sets_not_opaque.Remove(ks1, k)) +} + +// Adding a new key-value pair increases the occurrence of the value by one. +ghost +requires !(k in domain(d)) +ensures Occurrences(d[k = v], v) == Occurrences(d, v) + 1 +decreases +pure func AddOccurrences(d dict[int]int, k, v int) util.Unit { + return let ks1 := reveal Keys(d, v) in + let ks2 := reveal Keys(d[k = v], v) in + util.Asserting(ks2 == sets_not_opaque.Add(ks1, k)) +} + +// Updating a key to a new value results in an additional occurrences for the +// new value, and one less occurrence for the old value. +ghost +requires k in domain(d) +requires v != d[k] +ensures Occurrences(d[k = v], v) == Occurrences(d, v) + 1 +ensures Occurrences(d[k = v], d[k]) == Occurrences(d, d[k]) - 1 +decreases +pure func UpdateOccurrences(d dict[int]int, k, v int) util.Unit { + return let ks1v := reveal Keys(d, v) in + let ks2v := reveal Keys(d[k = v], v) in + let _ := util.Asserting(ks2v == sets_not_opaque.Add(ks1v, k)) in + let ks1dk := reveal Keys(d, d[k]) in + let ks2dk := reveal Keys(d[k = v], d[k]) in + util.Asserting(ks2dk == sets_not_opaque.Remove(ks1dk, k)) +} + +// v can occur at most once as a value in injective dictionaries. +ghost +requires IsInjective(d) +ensures Occurrences(d, v) <= 1 +decreases len(d) +pure func InjectiveOccurrences(d dict[int]int, v int) util.Unit { + return Occurrences(d, v) <= 1 ? util.Unit{} : + let x1 := sets_not_opaque.Choose(Keys(d, v)) in + let x2 := sets_not_opaque.Choose(sets_not_opaque.Remove(Keys(d, v), x1)) in + let _ := reveal IsInjective(d) in + util.Unit{} +} diff --git a/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..07a1f92 --- /dev/null +++ b/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!1207,qi-$Set[Int]_prog.in_singleton_set,qi-quant-u-0,qi-quant-u-200,qi-quant-u-1,qi-$Set[Int]_prog.singleton_set_cardinality,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.in_singleton_set_equality,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-quant-u-2,qi-quant-u-205,qi-quant-u-203,qi-quant-u-3,qi-quant-u-202,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_union_in_one,qi-$Set[Int]_prog.in_empty_set,qi-$Set[Int]_prog.in_right_in_union,qi-quant-u-4,qi-quant-u-207,qi-quant-u-5,qi-quant-u-10,qi-quant-u-219,qi-quant-u-11,qi-quant-u-217,qi-quant-u-214,qi-quant-u-6,qi-quant-u-213,qi-quant-u-7,qi-quant-u-209,qi-k!872,qi-quant-u-8,qi-quant-u-215,qi-quant-u-9,qi-quant-u-15,qi-quant-u-222,qi-quant-u-221,qi-quant-u-12,qi-quant-u-220,qi-quant-u-13,qi-k!1050,qi-k!1082,qi-k!1045,qi-k!1071,qi-k!1144,qi-k!1139,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@931@12@934@61,qi-k!1080,qi-k!1148,qi-quant-u-19,qi-quant-u-230,qi-k!2173,qi-k!1122,qi-k!2167,qi-quant-u-16,qi-quant-u-228,qi-quant-u-226,qi-quant-u-227,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@935@12@938@48,qi-quant-u-237,qi-quant-u-20,qi-quant-u-236,qi-quant-u-21,qi-quant-u-250,qi-quant-u-24,qi-quant-u-249,qi-quant-u-33,qi-quant-u-261,qi-quant-u-25,qi-quant-u-245,qi-k!3250,qi-quant-u-22,qi-quant-u-243,qi-quant-u-23,qi-quant-u-238,qi-quant-u-26,qi-quant-u-253,qi-$Set[Int]_prog.subset_definition,qi-k!862,qi-quant-u-256,qi-quant-u-29,qi-quant-u-255,qi-quant-u-244,qi-quant-u-44,qi-quant-u-45,qi-quant-u-279,qi-k!3964,qi-quant-u-251,qi-quant-u-252,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@800@12@802@56,qi-quant-u-36,qi-quant-u-269,qi-quant-u-37,qi-quant-u-35,qi-quant-u-265,qi-quant-u-290,qi-quant-u-51,qi-quant-u-288,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@1175@12@1177@62,qi-quant-u-39,qi-quant-u-271,qi-quant-u-272,qi-quant-u-53,qi-quant-u-293,qi-k!4877,qi-k!4885,qi-k!1131,qi-$Multiset[Int]_prog.count_card,qi-$Multiset[Int]_prog.card_non_negative,qi-$Multiset[Int]_prog.card_empty,qi-quant-u-68,qi-$Multiset[Int]_prog.singleton_unionone,qi-$Multiset[Int]_prog.count_union,qi-$Multiset[Int]_prog.card_union,qi-quant-u-69,qi-quant-u-323,qi-$Multiset[Int]_prog.count_empty,qi-$Multiset[Int]_prog.count_singleton,qi-$Multiset[Int]_prog.count_unionone,qi-k!6000,qi-k!6006,qi-$Multiset[Int]_prog.card_unionone,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@1097@12@1099@47,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@747@12@748@20,qi-$Set[Int]_prog.intersection_right_idempotency,qi-$Set[Int]_prog.intersection_left_idempotency,qi-quant-u-278,qi-quant-u-42,qi-quant-u-277,qi-quant-u-43,qi-quant-u-260,qi-quant-u-30,qi-quant-u-259,qi-quant-u-31,qi-$Set[Int]_prog.union_right_idempotency,qi-$Set[Int]_prog.union_left_idempotency,qi-quant-u-283,qi-quant-u-282,qi-quant-u-47,qi-quant-u-46,qi-quant-u-284,qi-k!4261,qi-quant-u-287,qi-quant-u-48,qi-quant-u-286,qi-quant-u-49,qi-quant-u-264,qi-quant-u-32,qi-quant-u-263,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@290@13@292@54,qi-k!1098,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@419@6@421@68,qi-k!1102,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@737@12@738@74,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@1269@12@1271@25,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/dicts_opaque/dicts_opaque.gobra.vpr@958@6@961@41,qi-quant-u-185,qi-quant-u-553,qi-k!1604,qi-k!1598,qi-prog.getter_over_tuple2,qi-quant-u-132,qi-quant-u-133,qi-k!1893,qi-k!2175,qi-prog.set_ax_dec,qi-prog.set_ax_bound,qi-prog.integer_ax_dec,qi-prog.integer_ax_bound,execution_time +1,22,77,70,77,9,493,492,498,84,70,27,27,22,24,22,804,811,70,106,2724,1042,174,55,1253,50,45,46,19,18,18,18,17,34,34,33,33,84,5,5,5,1,1,12,12,12,12,46,45,45,3,1125,217,5,1,116,11,11,5,174,1,5,5,4,4,3,12,12,12,11,13,13,13,35,35,12,12,3,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,8,5,7,1,1,4,4,4,41,1,1,5,7,7,18,28,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,2,2,1,2,1,4,4,107,9,3,2,1,4,8,10,9,2,2,18.368810415267944 +1,23,84,77,84,9,488,487,518,76,62,28,28,23,25,23,840,730,66,103,2870,1145,190,38,1266,50,45,46,19,19,19,19,15,33,33,31,31,83,5,5,5,1,1,12,12,12,12,44,44,44,8,1154,217,5,1,110,11,11,5,159,1,5,5,4,4,3,13,13,13,11,13,13,13,34,34,13,13,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,8,5,7,1,1,4,4,4,48,1,1,5,7,7,14,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,106,9,3,2,1,5,2,9,9,2,2,19.30323362350464 +1,23,80,71,79,9,484,483,561,83,68,29,29,23,25,23,850,830,68,104,2811,1099,177,52,1279,50,45,46,19,19,19,19,15,33,33,32,32,85,5,5,5,1,1,12,12,12,12,45,45,45,5,1143,219,5,1,116,12,12,5,175,1,5,5,4,4,3,13,13,13,12,13,13,13,36,36,13,13,3,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,10,7,9,1,1,5,5,5,45,1,1,5,7,7,15,25,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,105,9,3,2,1,18,14,9,9,2,2,19.636836051940918 +1,23,79,72,79,9,497,496,466,84,70,29,29,23,25,23,826,783,71,106,2823,1120,177,62,1247,50,45,46,19,19,19,19,16,34,34,33,33,88,5,5,5,1,1,12,12,12,12,47,46,46,8,1165,233,5,1,126,12,12,5,190,1,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,3,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,3,9,6,8,1,1,5,5,5,45,1,1,5,7,7,16,26,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,119,10,3,2,1,4,14,9,9,2,2,19.287102937698364 +1,23,79,72,78,9,496,495,564,82,67,29,29,23,25,23,878,849,70,105,2959,1177,181,57,1336,50,45,46,19,19,19,19,15,33,33,32,32,86,5,5,5,1,1,12,12,12,12,45,45,45,4,1193,229,5,1,121,12,12,5,168,1,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,10,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,8,5,7,1,1,4,4,4,62,1,1,5,7,7,14,23,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,2,2,1,2,1,4,4,107,9,3,2,1,4,14,9,9,2,2,19.0301513671875 +1,23,77,70,76,9,489,487,477,86,73,28,28,23,25,23,793,758,69,104,2941,1090,175,112,1276,50,45,46,19,19,19,19,17,35,35,34,34,83,5,5,5,1,1,12,12,12,12,47,46,46,2,1139,224,5,1,122,12,12,5,157,1,5,5,4,4,3,13,13,13,12,13,13,13,33,33,13,13,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,8,5,7,1,1,4,4,4,42,1,1,5,7,7,18,28,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,5,2,2,1,2,1,4,4,101,7,3,2,1,4,8,10,9,2,2,19.026081323623657 +1,22,74,70,74,9,482,480,488,76,63,28,28,22,24,22,817,694,66,100,2721,1078,186,32,1183,50,45,46,19,19,19,19,15,33,33,31,31,64,5,5,5,1,1,12,12,12,12,44,44,44,9,1109,214,5,1,104,11,11,5,142,1,5,5,4,4,3,13,13,13,11,13,13,13,31,31,13,13,10,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,13,10,12,1,1,6,6,6,42,1,1,5,7,7,17,27,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,2,2,1,2,1,4,4,87,9,3,2,1,4,14,9,9,2,2,19.25423526763916 +1,23,78,74,78,9,481,480,508,81,67,28,28,23,25,23,827,703,69,102,2602,1055,167,44,1160,50,45,46,19,19,19,19,17,35,35,34,34,68,5,5,5,1,1,12,12,12,12,45,45,45,3,1057,203,5,1,101,12,12,5,143,1,5,5,4,4,3,13,13,13,12,13,13,13,32,32,13,13,4,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,4,4,4,45,1,1,5,7,7,16,26,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,90,8,3,2,1,4,14,10,9,2,2,18.768330812454224 +1,23,77,70,77,9,489,488,477,83,69,29,29,23,25,23,784,722,71,106,2604,1052,176,44,1118,50,45,46,19,19,19,19,15,33,33,32,32,78,5,5,5,1,1,12,12,12,12,47,46,46,2,1043,208,5,1,115,12,12,5,154,1,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,10,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,4,4,4,46,1,1,5,7,7,14,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,5,2,2,1,2,1,4,4,100,9,3,2,1,4,2,9,9,2,2,18.867623805999756 +1,22,75,68,75,9,479,477,523,76,62,28,28,22,24,22,813,776,66,101,2732,1061,187,38,1223,50,45,46,19,18,18,18,15,32,32,31,31,77,5,5,5,1,1,12,12,12,12,44,44,44,4,1092,207,5,1,104,11,11,5,150,1,5,5,4,4,3,12,12,12,11,13,13,13,34,34,12,12,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,8,5,7,1,1,4,4,4,45,1,1,5,7,7,16,26,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,100,9,3,2,1,5,14,9,9,2,2,18.10794997215271 +1,22,76,70,76,9,488,487,478,86,70,28,28,22,24,22,779,737,70,105,2543,983,171,48,1164,50,45,46,19,19,19,19,15,33,33,31,31,77,5,5,5,1,1,12,12,12,12,46,45,45,2,1049,205,5,1,105,11,11,5,157,1,5,5,4,4,3,13,13,13,11,13,13,13,33,33,13,13,4,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,13,10,12,1,1,6,6,6,49,1,1,5,7,7,14,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,100,9,3,2,1,4,2,9,9,2,2,18.74820065498352 +1,22,77,70,76,9,489,488,519,79,66,28,28,22,24,22,804,763,70,106,2696,1029,184,53,1233,50,45,46,19,18,18,18,17,34,34,33,33,64,5,5,5,1,1,12,12,12,12,46,45,45,5,1056,199,5,1,107,11,11,5,146,1,5,5,4,4,3,12,12,12,11,13,13,13,36,36,12,12,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,3,8,5,7,1,1,4,4,4,47,1,1,5,7,7,15,25,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,5,2,2,1,2,1,4,4,82,7,3,2,1,4,8,10,9,2,2,18.986647129058838 +1,23,81,77,80,9,493,492,513,77,65,29,29,23,25,23,864,787,68,101,2992,1183,195,58,1310,50,45,46,19,19,19,19,15,33,33,32,32,64,5,5,5,1,1,12,12,12,12,45,45,45,8,1208,232,5,1,118,12,12,5,170,1,5,5,4,4,3,13,13,13,12,13,13,13,33,33,13,13,10,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,2,10,7,9,1,1,6,6,6,43,1,1,5,7,7,16,26,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,5,2,2,1,2,1,4,4,91,10,3,2,1,4,2,9,9,2,2,18.58991265296936 +1,23,79,74,79,9,484,483,547,81,67,28,28,23,25,23,846,765,67,100,2868,1122,184,60,1243,50,45,46,19,19,19,19,16,34,34,33,33,78,5,5,5,1,1,12,12,12,12,45,45,45,2,1155,229,5,1,118,12,12,5,148,1,5,5,4,4,3,13,13,13,12,13,13,13,33,33,13,13,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,2,10,7,9,1,1,6,6,6,50,1,1,5,7,7,13,23,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,2,2,1,2,1,4,4,101,9,3,2,1,4,14,9,9,2,2,18.53548502922058 +1,23,82,76,82,9,486,484,495,87,74,29,29,23,25,23,794,670,70,105,2877,1105,167,98,1215,50,45,46,19,19,19,19,15,33,33,32,32,71,5,5,5,1,1,12,12,12,12,46,46,45,3,1115,208,5,1,108,11,11,5,144,1,5,5,4,4,3,13,13,13,11,13,13,13,32,32,13,13,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,10,7,9,1,1,4,4,4,45,1,1,5,8,8,15,25,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,90,7,3,2,1,18,4,9,9,2,2,18.567266702651978 +1,23,81,76,81,9,498,497,497,83,68,28,28,23,25,23,829,764,69,105,2735,1073,167,45,1217,50,45,46,19,19,19,19,17,35,35,34,34,64,5,5,5,1,1,12,12,12,12,45,45,45,2,1127,224,5,1,119,12,12,5,165,1,5,5,4,4,3,13,13,13,12,13,13,13,32,32,13,13,3,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,10,7,9,1,1,6,6,6,57,1,1,5,7,7,15,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,93,10,3,2,1,11,14,10,9,2,2,19.18628978729248 +1,22,83,76,83,9,502,501,525,89,75,29,29,23,25,23,825,746,71,110,3099,1150,177,97,1307,50,45,46,19,19,19,19,16,34,34,32,32,89,5,5,5,1,1,12,12,12,12,46,45,45,1,1170,224,5,1,120,11,11,5,162,1,5,5,4,4,3,13,13,13,11,13,13,13,34,34,13,13,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,2,9,6,8,1,1,5,5,5,45,1,1,5,7,7,15,25,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,112,9,3,2,1,5,2,9,9,2,2,18.972155809402466 +1,23,79,72,79,9,495,493,547,80,67,29,29,23,25,23,841,785,70,105,2791,1094,179,44,1269,50,45,46,19,19,19,19,17,35,35,34,34,70,5,5,5,1,1,12,12,12,12,45,45,45,4,1121,221,5,1,117,12,12,5,158,1,5,5,4,4,3,13,13,13,12,13,13,13,36,36,13,13,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,3,10,7,9,1,1,6,6,6,46,1,1,5,7,7,14,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,92,9,3,2,1,4,14,10,9,2,2,19.329055547714233 +1,23,82,76,82,9,495,494,515,79,66,28,28,23,25,23,871,785,67,104,2978,1191,208,46,1265,50,45,46,19,19,19,19,15,33,33,32,32,89,5,5,5,1,1,12,12,12,12,45,45,45,9,1184,231,5,1,122,12,12,5,160,1,5,5,4,4,3,13,13,13,12,13,13,13,35,35,13,13,10,6,6,6,6,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,10,7,9,1,1,4,4,4,48,1,1,5,8,8,14,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,2,2,1,2,1,4,4,112,9,3,2,1,4,2,9,9,2,2,18.976101398468018 +1,22,76,69,76,9,474,473,530,76,63,29,29,22,24,22,813,785,66,100,2742,1032,173,39,1253,50,45,46,19,19,19,19,17,35,35,33,33,80,5,5,5,1,1,12,12,12,12,44,44,44,5,1099,214,5,1,116,11,11,5,156,1,5,5,4,4,3,13,13,13,11,13,13,13,34,34,13,13,10,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,11,8,10,1,1,6,6,6,46,1,1,5,7,7,17,27,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,102,9,3,2,1,4,2,10,9,2,2,19.043288707733154 +1,22,79,72,79,9,498,496,534,84,71,28,28,22,24,22,848,812,68,106,3137,1157,188,109,1346,50,45,46,19,18,18,18,17,34,34,33,33,80,5,5,5,1,1,12,12,12,12,44,44,44,1,1206,228,5,1,121,11,11,5,171,1,5,5,4,4,3,12,12,12,11,13,13,13,35,35,12,12,10,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,8,5,7,1,1,4,4,4,46,1,1,5,7,7,14,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,2,2,1,2,1,4,4,103,9,3,2,1,4,8,10,9,2,2,18.495521306991577 +1,23,83,76,83,9,497,496,573,79,65,29,29,23,25,23,895,868,69,104,3043,1196,195,49,1334,50,45,46,19,19,19,19,16,34,34,33,33,88,5,5,5,1,1,12,12,12,12,45,45,45,4,1228,234,5,1,123,12,12,5,172,1,5,5,4,4,3,13,13,13,12,13,13,13,35,35,13,13,3,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,2,9,6,8,1,1,5,5,5,43,1,1,5,7,7,18,28,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,110,9,3,2,1,4,2,9,9,2,2,18.293484449386597 +1,23,81,75,81,9,502,501,535,82,67,29,29,23,25,23,865,785,71,108,2984,1161,209,63,1311,50,45,46,19,19,19,19,15,33,33,32,32,85,5,5,5,1,1,12,12,12,12,47,46,46,2,1167,229,5,1,120,12,12,5,159,1,5,5,4,4,3,13,13,13,12,13,13,13,34,34,13,13,4,6,6,6,6,5,5,11,1,1,1,1,2,4,6,6,2,2,2,2,8,5,7,1,1,4,4,4,45,1,1,5,7,7,15,25,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,108,10,3,2,1,5,2,9,9,2,2,18.440985202789307 +1,23,82,76,81,9,485,484,503,87,74,29,29,23,25,23,820,734,69,106,2941,1124,174,91,1255,50,45,46,19,19,19,19,15,33,33,32,32,86,5,5,5,1,1,12,12,12,12,45,45,45,3,1189,213,5,1,113,12,12,5,166,1,5,5,4,4,3,13,13,13,12,13,13,13,32,32,13,13,3,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,8,5,7,1,1,4,4,4,50,1,1,5,7,7,13,23,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,106,8,3,2,1,18,4,9,9,2,2,19.511987686157227 +1,22,78,71,78,9,509,508,526,90,76,27,27,22,24,22,830,775,70,109,3111,1186,189,123,1327,50,45,46,19,18,18,18,17,34,34,33,33,78,5,5,5,1,1,12,12,12,12,46,45,45,1,1177,227,5,1,118,11,11,5,159,1,5,5,4,4,3,12,12,12,11,13,13,13,34,34,12,12,10,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,11,8,10,1,1,6,6,6,49,1,1,5,7,7,12,22,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,100,9,3,2,1,6,8,10,9,2,2,18.4862322807312 +1,23,80,73,80,9,498,496,551,84,72,28,28,23,25,23,848,792,66,103,3184,1217,197,123,1365,50,45,46,19,19,19,19,15,33,33,32,32,76,5,5,5,1,1,12,12,12,12,45,45,45,8,1238,237,5,1,125,12,12,5,154,1,5,5,4,4,3,13,13,13,12,13,13,13,35,35,13,13,3,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,9,6,8,1,1,5,5,5,38,1,1,5,7,7,18,28,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,5,2,2,1,2,1,4,4,97,9,3,2,1,6,14,9,9,2,2,18.433714389801025 +1,22,77,70,77,9,499,497,540,85,71,28,28,22,24,22,834,789,70,107,2865,1098,192,63,1247,50,45,46,19,19,19,19,17,35,35,33,33,86,5,5,5,1,1,12,12,12,12,46,45,45,1,1135,224,5,1,123,11,11,5,168,1,5,5,4,4,3,13,13,13,11,13,13,13,35,35,13,13,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,3,11,8,10,1,1,6,6,6,49,1,1,5,7,7,14,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,2,2,1,2,1,4,4,108,9,3,2,1,4,8,10,9,2,2,19.05180835723877 +1,22,75,69,75,9,477,475,504,80,66,27,27,22,24,22,811,781,68,102,2669,1042,176,46,1247,50,45,46,19,18,18,18,15,32,32,31,31,90,5,5,5,1,1,12,12,12,12,45,44,44,3,1118,221,5,1,124,10,10,5,158,1,5,5,4,4,3,12,12,12,10,13,13,13,34,34,12,12,6,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,3,2,2,3,10,7,9,1,1,6,6,6,41,1,1,5,7,7,17,27,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,5,2,2,1,2,1,4,4,112,9,3,2,1,4,2,9,9,2,2,19.22210168838501 +1,22,79,74,79,9,486,485,514,79,65,29,29,22,24,22,835,739,67,102,2862,1099,183,42,1277,50,45,46,19,19,19,19,16,34,34,32,32,80,5,5,5,1,1,12,12,12,12,44,44,44,9,1138,221,5,1,114,11,11,5,156,1,5,5,4,4,3,13,13,13,11,13,13,13,33,33,13,13,4,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,3,13,10,12,1,1,6,6,6,40,1,1,5,7,7,18,28,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,1,2,1,2,1,4,4,103,9,3,2,1,4,8,9,9,2,2,18.388686418533325 +1,22,84,75,84,9,479,478,562,79,66,28,28,23,25,23,866,819,67,102,2868,1085,171,43,1322,50,45,46,19,19,19,19,17,35,35,33,33,82,5,5,5,1,1,12,12,12,12,44,44,44,4,1158,218,5,1,118,11,11,5,167,1,5,5,4,4,3,13,13,13,11,13,13,13,35,35,13,13,10,4,4,4,4,5,5,11,1,1,1,1,2,4,6,6,2,2,2,3,11,8,10,1,1,6,6,6,46,1,1,5,7,7,14,24,2,4,2,2,2,2,10,4,2,2,7,8,8,3,3,2,1,3,2,1,1,1,1,1,1,1,1,1,6,2,2,2,3,3,3,8,5,5,5,5,8,8,8,3,3,2,2,1,2,1,4,4,104,9,3,2,1,28,2,10,9,2,2,19.110204935073853 diff --git a/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..af27614 Binary files /dev/null and b/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..b71783a Binary files /dev/null and b/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque.gobra b/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque.gobra new file mode 100644 index 0000000..197cdee --- /dev/null +++ b/evaluation/experiments/standard_library/dicts_opaque/dicts_opaque.gobra @@ -0,0 +1,481 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// A copy of the dicts package from the standard library. +// Uses the opaque version of the sets package. +package dicts_opaque + +// ##(-I ./..) + +import "util" +import "sets_opaque" + +// A dictionary is empty if its domain is empty. +ghost +decreases +pure func IsEmpty(d dict[int]int) bool { + return sets_opaque.IsEmpty(domain(d)) +} + +// Returns the empty dictionary. +ghost +ensures IsEmpty(result) +decreases +pure func Empty() (result dict[int]int) { + return dict[int]int{} +} + +// Retrieves the value associated with the key, if present, as an option. +ghost +decreases +pure func Get(d dict[int]int, k int) option[int] { + return k in domain(d) ? some(d[k]) : none[int] +} + +// Keep all key-value pairs corresponding to the set of keys provided. +ghost +opaque +ensures forall x int :: {result[x]} (x in domain(d) && x in xs) ==> + (x in domain(result) && result[x] == d[x]) +ensures forall x int :: {x in domain(result)} x in domain(result) ==> + (x in domain(d) && x in xs) +ensures domain(result) == xs intersection domain(d) +decreases xs +pure func Restrict(d dict[int]int, xs set[int]) (result dict[int]int) { + return let ys := (xs intersection domain(d)) in + (sets_opaque.IsEmpty(ys) ? Empty() : + (let y := sets_opaque.Choose(ys) in + (let yr := sets_opaque.Remove(ys, y) in + (let _ := sets_opaque.IntersectLenUpper(xs, domain(d)) in + Restrict(d, yr)[y = d[y]])))) +} + +// Remove all key-value pairs corresponding to the set of keys provided. +ghost +decreases +pure func RemoveKeys(d dict[int]int, xs set[int]) (result dict[int]int) { + return Restrict(d, domain(d) setminus xs) +} + +// Remove a key-value pair. Returns d if k is not in the domain of d. +ghost +ensures len(result) <= len(d) +ensures k in domain(d) ==> len(result) == len(d) - 1 +ensures !(k in domain(d)) ==> len(result) == len(d) +decreases +pure func Remove(d dict[int]int, k int) (result dict[int]int) { + return let ys := (sets_opaque.Remove(domain(d), k)) in + let _ := util.Asserting(ys intersection domain(d) == ys) in + RemoveKeys(d, sets_opaque.SingletonSet(k)) +} + +// True iff k maps to the same value or is not in the domains of d1 and d2. +ghost +decreases +pure func IsEqualOnKey(d1, d2 dict[int]int, k int) bool { + return !(k in domain(d1) || k in domain(d2)) || + (k in domain(d1) && k in domain(d2) && d1[k] == d2[k]) +} + +// True iff if d1 is a subset of d2. +ghost +decreases +pure func IsSubset(d1, d2 dict[int]int) bool { + return domain(d1) subset domain(d2) && + forall k int :: {IsEqualOnKey(d1, d2, k)} {k in domain(d1)} (k in domain(d1)) ==> + IsEqualOnKey(d1, d2, k) +} + +// Union of two dictionaries. Does not require disjoint domains: on the intersection, +// values from the second dictionary are chosen. +ghost +opaque +ensures domain(result) == domain(d1) union domain(d2) +ensures forall k int :: {result[k]} (k in domain(d2)) ==> result[k] == d2[k] +ensures forall k int :: {result[k]} (!(k in domain(d2)) && k in domain(d1)) ==> + result[k] == d1[k] +decreases domain(d1) union domain(d2) +pure func Union(d1, d2 dict[int]int) (result dict[int]int) { + return let ks := domain(d1) union domain(d2) in + (sets_opaque.IsEmpty(ks) ? Empty() : + let k := sets_opaque.Choose(ks) in + let c1 := Remove(d1, k) in + let c2 := Remove(d2, k) in + let _ := sets_opaque.RemoveUnionLen(domain(d1), domain(d2), k) in + (k in domain(d2) ? Union(c1, c2)[k = d2[k]] : Union(c1, c2)[k = d1[k]])) +} + +// Dictionaries are disjoint iff their domains are disjoint. +ghost +decreases +pure func AreDisjoint(d1, d2 dict[int]int) bool { + return sets_opaque.AreDisjoint(domain(d1), domain(d2)) +} + +// The length of the union of two disjoint dictionaries is the sum of each of their lengths. +ghost +opaque +requires AreDisjoint(d1, d2) +ensures len(Union(d1, d2)) == len(d1) + len(d2) +decreases +pure func DisjointUnionLen(d1, d2 dict[int]int) util.Unit { + return util.Unit{} +} + +// True iff a dictionary is injective. +ghost +opaque +decreases +pure func IsInjective(d dict[int]int) bool { + return forall k1, k2 int :: {d[k1], d[k2]} (k1 != k2 && k1 in domain(d) && k2 in domain(d)) ==> + d[k1] != d[k2] +} + +// True iff a dictionary contains all valid keys. +ghost +opaque +decreases +pure func IsTotal(d dict[int]int) bool { + return forall k int :: {k in domain(d)} k in domain(d) +} + +// True iff a dictionary is monotonic. +ghost +opaque +decreases +pure func IsMonotonic(d dict[int]int) bool { + return forall k1, k2 int :: {d[k1], d[k2]} (k1 in domain(d) && k2 in domain(d) && k1 <= k2) ==> + d[k1] <= d[k2] +} + +// True iff a dictionary is monotonic. Only considers keys greater than or equal to start. +ghost +opaque +decreases +pure func IsMonotonicFrom(d dict[int]int, start int) bool { + return forall k1, k2 int :: {d[k1], d[k2]} (k1 in domain(d) && k2 in domain(d) && start <= k1 && k1 <= k2) ==> + d[k1] <= d[k2] +} + +// True iff a dictionary is monotonic. Only considers keys in the interval [start, end). +ghost +opaque +decreases +pure func IsMonotonicFromTo(d dict[int]int, start, end int) bool { + return forall k1, k2 int :: {d[k1], d[k2]} (k1 in domain(d) && + k2 in domain(d) && + start <= k1 && k1 <= k2 && k2 < end) ==> d[k1] <= d[k2] +} + +// True iff two dictionaries are equal in the interval [start, end). +ghost +opaque +decreases +pure func IsEqualInRange(d1, d2 dict[int]int, start, end int) bool { + return forall k int :: {d1[k], d2[k]} {k in domain(d1)} {k in domain(d2)}(start <= k && k < end) ==> + k in domain(d1) && k in domain(d2) && d1[k] == d2[k] +} + +// True iff d1 and d2 agree on all keys that their domains share. +ghost +decreases +pure func Agree(d1, d2 dict[int]int) bool { + return forall k int :: {d1[k], d2[k]} k in (domain(d1) intersection domain(d2)) ==> + d1[k] == d2[k] +} + +// The domain of a map after removing a key is equivalent to removing +// the key from the domain of the original map. +ghost +opaque +ensures domain(Remove(d, k)) == sets_opaque.Remove(domain(d), k) +decreases +pure func RemoveDomain(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// The domain of the empty dictionary is the empty set. +ghost +opaque +requires IsEmpty(d) +ensures domain(d) == sets_opaque.Empty() +decreases +pure func EmptyDictEmptyDomain(d dict[int]int) util.Unit { + return util.Unit{} +} + +// The domain of a dictionary after inserting a key-value pair is equivalent to +// inserting the key into the original map's domain set. +ghost +opaque +ensures domain(d[k = v]) == sets_opaque.Add(domain(d), k) +decreases +pure func InsertDomain(d dict[int]int, k, v int) util.Unit { + return util.Unit{} +} + +// Inserting value at k in d results in a dictionary that maps k to v. +ghost +opaque +ensures d[k = v][k] == v +decreases +pure func UpdateSame(d dict[int]int, k, v int) util.Unit { + return util.Unit{} +} + +// Reassigning the corresponding value to a key does not change the dictionary. +ghost +opaque +requires k in domain(d) +requires v == d[k] +ensures d[k = v] == d +decreases +pure func UpdateEqual(d dict[int]int, k, v int) util.Unit { + return util.Unit{} +} + +// Inserting v at k2 does not change the value mapped to by any other keys in d. +ghost +opaque +requires k1 != k2 +ensures k2 in domain(d[k1 = v]) == k2 in domain(d) +ensures k2 in domain(d) ==> d[k1 = v][k2] == d[k2] +decreases +pure func UpdateDifferent(d dict[int]int, k1, k2, v int) util.Unit { + return util.Unit{} +} + +// Removing a key-value pair from a dictionary does not change the value mapped to +// by any other keys in the map. +ghost +opaque +requires k1 in domain(d) +requires k1 != k2 +ensures Remove(d, k2)[k1] == d[k1] +decreases +pure func RemoveDifferent(d dict[int]int, k1, k2 int) util.Unit { + return util.Unit{} +} + +// Two maps are equivalent if their domains are equivalent and every key in their +// domains map to the same value. +ghost +opaque +ensures (d1 == d2) == + (domain(d1) == domain(d2) && forall k int :: {d1[k], d2[k]} k in domain(d1) ==> d1[k] == d2[k]) +decreases +pure func ExtEqual(d1, d2 dict[int]int) util.Unit { + return util.Unit{} +} + +// The cardinality of a dictionary is non-negative. +ghost +opaque +ensures len(d) >= 0 +decreases +pure func NonNegativeLen(d dict[int]int) util.Unit { + return util.Unit{} +} + +// The cardinality of a dictionary is equal to the cardinality of its domain. +ghost +opaque +ensures len(d) == len(domain(d)) +decreases +pure func DomainLenEq(d dict[int]int) util.Unit { + return util.Unit{} +} + +// If two dictionaries are disjoint there is no key that is in both of their domains. +ghost +opaque +requires AreDisjoint(d1, d2) +ensures !(k in domain(d1) && k in domain(d2)) +decreases +pure func DisjointNoSharedKey(d1, d2 dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// If two dictionaries are not disjoint, there exists a key that is in both of their domains. +ghost +opaque +requires !AreDisjoint(d1, d2) +ensures exists k int :: {k in domain(d1), k in domain(d2)} k in domain(d1) && k in domain(d2) +decreases +pure func NotDisjointSharedKey(d1, d2 dict[int]int) util.Unit { + return util.Unit{} +} + +// There is only one empty map. +ghost +opaque +requires IsEmpty(d) +ensures d == Empty() +decreases +pure func EmptyIsUnique(d dict[int]int) util.Unit { + return util.Unit{} +} + +// An empty dictionary contains no keys. +ghost +opaque +requires IsEmpty(d) +ensures !(k in domain(d)) +decreases +pure func NotInEmpty(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// There exists a key in a non-empty dictionary. +ghost +opaque +requires !IsEmpty(d) +ensures exists k int :: {k in domain(d)} k in domain(d) +decreases +pure func NotEmptyKeyExists(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// If a key is in d, d is not empty. +ghost +opaque +requires k in domain(d) +ensures !IsEmpty(d) +decreases +pure func KeyInDomainDictNotEmpty(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// Inserting a new key increases the cardinality of the dictionary by 1. +// Updating the value of a key does not change the cardinality. +ghost +opaque +ensures k in domain(d) ==> len(d[k = v]) == len(d) +ensures !(k in domain(d)) ==> len(d[k = v]) == len(d) + 1 +decreases +pure func InsertUpdateLen(d dict[int]int, k, v int) util.Unit { + return util.Unit{} +} + +// If a value is in the range of a dictionary, there exists a corresponding key. +ghost +opaque +requires v in range(d) +ensures exists k int :: {k in domain(d)} k in domain(d) && d[k] == v +decreases +pure func ValueHasKey(d dict[int]int, v int) util.Unit { + return util.Unit{} +} + +// If a value is in the domain of a dictionary, the corresponding value is in its range. +ghost +opaque +requires k in domain(d) +ensures d[k] in range(d) +decreases +pure func KeyMapsToRange(d dict[int]int, k int) util.Unit { + return util.Unit{} +} + +// Returns a dictionary with the values at k1 and k2 swapped. +ghost +requires k1 in domain(d) && k2 in domain(d) +ensures domain(result) == domain(d) +ensures result[k1] == d[k2] && result[k2] == d[k1] +ensures forall k int :: {result[k]} (k in domain(d) && k != k1 && k != k2) ==> + result[k] == d[k] +decreases +pure func Swap(d dict[int]int, k1, k2 int) (result dict[int]int) { + return d[k1 = d[k2]][k2 = d[k1]] +} + +// Returns the set of keys from the given dictionary that map to the specified value. +ghost +opaque +ensures forall k int :: {k in result} (k in domain(d) && d[k] == v) == (k in result) +decreases len(d) +pure func Keys(d dict[int]int, v int) (result set[int]) { + return IsEmpty(d) ? sets_opaque.Empty() : + (let x := sets_opaque.Choose(domain(d)) in + (let subKeys := Keys(Remove(d, x), v) in + (d[x] == v ? sets_opaque.Add(subKeys, x) : subKeys))) + +} + +// Returns the number of occurences of the value in the dictionary. +ghost +decreases +pure func Occurrences(d dict[int]int, v int) int { + return len(Keys(d, v)) +} + +// Remove preserves the injectivity of a dictionary. +ghost +opaque +requires IsInjective(d) +ensures IsInjective(Remove(d, k)) +decreases +pure func RemovePreservesInjectivity(d dict[int]int, k int) util.Unit { + return let _ := reveal IsInjective(d) in + let _ := reveal IsInjective(Remove(d, k)) in + util.Unit{} +} + +// After removing a key, the corresponding value occurs one less time. +ghost +opaque +requires k in domain(d) +ensures Occurrences(Remove(d, k), d[k]) == Occurrences(d, d[k]) - 1 +decreases +pure func RemoveOccurrences(d dict[int]int, k int) util.Unit { + return let ks1 := reveal Keys(d, d[k]) in + let ks2 := reveal Keys(Remove(d, k), d[k]) in + util.Asserting(ks2 == sets_opaque.Remove(ks1, k)) +} + +// Adding a new key-value pair increases the occurrence of the value by one. +ghost +opaque +requires !(k in domain(d)) +ensures Occurrences(d[k = v], v) == Occurrences(d, v) + 1 +decreases +pure func AddOccurrences(d dict[int]int, k, v int) util.Unit { + return let ks1 := reveal Keys(d, v) in + let ks2 := reveal Keys(d[k = v], v) in + util.Asserting(ks2 == sets_opaque.Add(ks1, k)) +} + +// Updating a key to a new value results in an additional occurrences for the +// new value, and one less occurrence for the old value. +ghost +opaque +requires k in domain(d) +requires v != d[k] +ensures Occurrences(d[k = v], v) == Occurrences(d, v) + 1 +ensures Occurrences(d[k = v], d[k]) == Occurrences(d, d[k]) - 1 +decreases +pure func UpdateOccurrences(d dict[int]int, k, v int) util.Unit { + return let ks1v := reveal Keys(d, v) in + let ks2v := reveal Keys(d[k = v], v) in + let _ := util.Asserting(ks2v == sets_opaque.Add(ks1v, k)) in + let ks1dk := reveal Keys(d, d[k]) in + let ks2dk := reveal Keys(d[k = v], d[k]) in + util.Asserting(ks2dk == sets_opaque.Remove(ks1dk, k)) +} + +// v can occur at most once as a value in injective dictionaries. +ghost +opaque +requires IsInjective(d) +ensures Occurrences(d, v) <= 1 +decreases len(d) +pure func InjectiveOccurrences(d dict[int]int, v int) util.Unit { + return Occurrences(d, v) <= 1 ? util.Unit{} : + let x1 := sets_opaque.Choose(Keys(d, v)) in + let x2 := sets_opaque.Choose(sets_opaque.Remove(Keys(d, v), x1)) in + let _ := reveal IsInjective(d) in + util.Unit{} +} diff --git a/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..58d50ad --- /dev/null +++ b/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!504,qi-$Set[Int]_prog.in_singleton_set,qi-quant-u-0,qi-quant-u-22,qi-quant-u-1,qi-$Set[Int]_prog.singleton_set_cardinality,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.in_singleton_set_equality,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-quant-u-2,qi-quant-u-27,qi-quant-u-25,qi-quant-u-3,qi-quant-u-24,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_union_in_one,qi-$Set[Int]_prog.in_empty_set,qi-quant-u-4,qi-quant-u-29,qi-quant-u-5,qi-$Set[Int]_prog.subset_definition,qi-k!415,qi-quant-u-6,qi-quant-u-31,qi-quant-u-7,qi-quant-u-14,qi-quant-u-46,qi-quant-u-15,qi-quant-u-45,qi-quant-u-12,qi-quant-u-44,qi-quant-u-13,qi-quant-u-42,qi-k!425,qi-quant-u-10,qi-quant-u-41,qi-quant-u-11,qi-quant-u-39,qi-quant-u-18,qi-quant-u-38,qi-quant-u-8,qi-quant-u-37,qi-quant-u-50,qi-quant-u-19,qi-quant-u-48,qi-quant-u-9,qi-quant-u-33,qi-prog.getter_over_tuple2,qi-quant-u-16,qi-quant-u-17,qi-prog.set_ax_dec,qi-prog.set_ax_bound,execution_time +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.407870531082153 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.62822937965393 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.823657512664795 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.389697551727295 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.680948495864868 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.461118221282959 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.7007155418396 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.779568195343018 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.575013637542725 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.695531845092773 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.886630296707153 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.894464254379272 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.559780597686768 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.337003946304321 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.836902618408203 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.500547885894775 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.496719121932983 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.793999910354614 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.52246880531311 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.536472082138062 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.720568656921387 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.720842599868774 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.42858076095581 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.932849884033203 +1,2,6,6,6,2,28,27,8,5,3,3,3,2,2,2,30,11,7,5,41,23,20,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.378710508346558 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.451270580291748 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.757553339004517 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.58380126953125 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.44314193725586 +1,2,6,6,6,2,28,27,7,5,3,4,4,2,2,2,29,12,8,5,38,22,18,5,2,4,2,3,6,1,1,1,1,6,5,6,1,1,1,1,1,3,2,2,2,2,3,5,5,5,3,3,3,5,5,3,2,1,3,2,9.520839929580688 diff --git a/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..b743b6a Binary files /dev/null and b/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..4aaf9ec Binary files /dev/null and b/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque.gobra b/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque.gobra new file mode 100644 index 0000000..232a74f --- /dev/null +++ b/evaluation/experiments/standard_library/lemma_not_opaque/lemma_not_opaque.gobra @@ -0,0 +1,87 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// A copy of a non-trivial lemma from the standard library. +package lemma_not_opaque + +// ##(-I ./..) + +import "util" + +// A set is empty if it has cardinality 0. +ghost +decreases +pure func IsEmpty(xs set[int]) bool { + return len(xs) == 0 +} + +// A set is a singleton if it has cardinality 1. +ghost +decreases +pure func IsSingleton(xs set[int]) bool { + return len(xs) == 1 +} + +// Returns a singleton containing x. +ghost +ensures IsSingleton(result) +ensures e in result +decreases +pure func SingletonSet(e int) (result set[int]) { + return set[int]{e} +} + +// Returns whether xs is a proper subset of ys. +ghost +decreases +pure func IsProperSubset(xs, ys set[int]) bool { + return xs subset ys && xs != ys +} + +// Returns an element from a non-empty set. +ghost +requires !IsEmpty(xs) +ensures e in xs +ensures IsSingleton(xs) ==> xs == SingletonSet(e) +decreases +pure func choose(xs set[int]) (e int) + +// Remove e from xs. Does not require e to be in xs. +ghost +ensures !(e in xs) ==> result == xs +ensures (e in xs) ==> (len(result) == len(xs) - 1) +ensures !(e in xs) ==> (len(result) == len(xs)) +decreases +pure func Remove(xs set[int], e int) (result set[int]) { + return xs setminus SingletonSet(e) +} + +// If xs is a subset of ys and both have the same cardinality, they are equal. +ghost +requires xs subset ys +requires len(xs) == len(ys) +ensures xs == ys +decreases +pure func SubsetEquality(xs, ys set[int]) util.Unit { + return util.Asserting(len(ys setminus xs) == len(ys) - len(xs)) +} + +// If xs is a subset of ys, then the cardinality of xs is less than or equal to the cardinality of ys. +// If xs is a strict subset of ys, then the cardinality of xs is less than the cardinality of ys. +ghost +decreases xs, ys +ensures xs subset ys ==> len(xs) <= len(ys) +ensures IsProperSubset(xs, ys) ==> len(xs) < len(ys) +pure func SubsetLen(xs, ys set[int]) util.Unit { + return (!(xs subset ys) || len(xs) == 0) ? util.Unit{} : + len(xs) == len(ys) ? + let _ := SubsetEquality(xs, ys) in + (let e := choose(xs) in + (SubsetLen(Remove(xs, e), Remove(ys, e)))) : + + let e:= choose(xs) in + (SubsetLen(Remove(xs, e), Remove(ys, e))) +} diff --git a/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..857658c --- /dev/null +++ b/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!504,qi-$Set[Int]_prog.in_singleton_set,qi-quant-u-0,qi-quant-u-22,qi-quant-u-1,qi-$Set[Int]_prog.singleton_set_cardinality,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.in_singleton_set_equality,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-quant-u-2,qi-quant-u-27,qi-quant-u-25,qi-quant-u-3,qi-quant-u-24,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_union_in_one,qi-$Set[Int]_prog.in_empty_set,qi-$Set[Int]_prog.in_right_in_union,qi-quant-u-4,qi-quant-u-29,qi-quant-u-5,qi-$Set[Int]_prog.subset_definition,qi-k!415,qi-quant-u-6,qi-quant-u-31,qi-quant-u-7,qi-quant-u-14,qi-quant-u-46,qi-quant-u-15,qi-quant-u-13,qi-quant-u-42,qi-quant-u-10,qi-quant-u-41,qi-quant-u-11,qi-quant-u-39,qi-quant-u-52,qi-quant-u-21,qi-quant-u-50,qi-quant-u-38,qi-quant-u-8,qi-quant-u-37,qi-quant-u-9,qi-quant-u-33,qi-prog.getter_over_tuple2,qi-quant-u-16,qi-quant-u-17,qi-k!425,qi-prog.set_ax_dec,qi-prog.set_ax_bound,execution_time +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.424501180648804 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.53942084312439 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.688337802886963 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.773677825927734 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,7,6,7,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.556092500686646 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,8,7,8,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.536595582962036 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.530071020126343 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,8,7,8,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.540412425994873 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.814835548400879 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.810972690582275 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.623236656188965 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.560847043991089 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.491089582443237 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.59114146232605 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.501184463500977 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.428701877593994 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.559736490249634 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.76392149925232 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.46000337600708 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,8,7,8,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.627538442611694 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.961721181869507 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.557636976242065 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.724958896636963 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,8,7,8,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.733354568481445 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.674569606781006 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.520197868347168 +1,2,6,6,6,2,24,23,8,5,3,3,3,2,2,2,20,6,5,4,28,16,4,3,13,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.430753946304321 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,6,5,6,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.35804033279419 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,8,7,8,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.446574687957764 +1,2,6,6,6,2,24,23,7,5,3,3,3,2,2,2,19,5,5,4,26,15,4,3,12,4,2,3,6,1,1,1,1,8,7,8,1,1,2,2,2,2,3,3,3,5,5,5,5,5,3,2,1,1,3,2,9.53910493850708 diff --git a/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..c7a48b7 Binary files /dev/null and b/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..a0ff1b3 Binary files /dev/null and b/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque.gobra b/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque.gobra new file mode 100644 index 0000000..fe7bbb6 --- /dev/null +++ b/evaluation/experiments/standard_library/lemma_opaque/lemma_opaque.gobra @@ -0,0 +1,90 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// A copy of a non-trivial lemma from the standard library. +// Lemmas are turned opaque. +package lemma_opaque + +// ##(-I ./..) + +import "util" + +// A set is empty if it has cardinality 0. +ghost +decreases +pure func IsEmpty(xs set[int]) bool { + return len(xs) == 0 +} + +// A set is a singleton if it has cardinality 1. +ghost +decreases +pure func IsSingleton(xs set[int]) bool { + return len(xs) == 1 +} + +// Returns a singleton containing x. +ghost +ensures IsSingleton(result) +ensures e in result +decreases +pure func SingletonSet(e int) (result set[int]) { + return set[int]{e} +} + +// Returns whether xs is a proper subset of ys. +ghost +decreases +pure func IsProperSubset(xs, ys set[int]) bool { + return xs subset ys && xs != ys +} + +// Returns an element from a non-empty set. +ghost +requires !IsEmpty(xs) +ensures e in xs +ensures IsSingleton(xs) ==> xs == SingletonSet(e) +decreases +pure func choose(xs set[int]) (e int) + +// Remove e from xs. Does not require e to be in xs. +ghost +ensures !(e in xs) ==> result == xs +ensures (e in xs) ==> (len(result) == len(xs) - 1) +ensures !(e in xs) ==> (len(result) == len(xs)) +decreases +pure func Remove(xs set[int], e int) (result set[int]) { + return xs setminus SingletonSet(e) +} + +// If xs is a subset of ys and both have the same cardinality, they are equal. +ghost +opaque +requires xs subset ys +requires len(xs) == len(ys) +ensures xs == ys +decreases +pure func SubsetEquality(xs, ys set[int]) util.Unit { + return util.Asserting(len(ys setminus xs) == len(ys) - len(xs)) +} + +// If xs is a subset of ys, then the cardinality of xs is less than or equal to the cardinality of ys. +// If xs is a strict subset of ys, then the cardinality of xs is less than the cardinality of ys. +ghost +opaque +decreases xs, ys +ensures xs subset ys ==> len(xs) <= len(ys) +ensures IsProperSubset(xs, ys) ==> len(xs) < len(ys) +pure func SubsetLen(xs, ys set[int]) util.Unit { + return (!(xs subset ys) || len(xs) == 0) ? util.Unit{} : + len(xs) == len(ys) ? + let _ := SubsetEquality(xs, ys) in + (let e := choose(xs) in + (SubsetLen(Remove(xs, e), Remove(ys, e)))) : + + let e:= choose(xs) in + (SubsetLen(Remove(xs, e), Remove(ys, e))) +} diff --git a/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..4f812f5 --- /dev/null +++ b/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!881,qi-$Set[Int]_prog.in_singleton_set,qi-quant-u-0,qi-quant-u-110,qi-quant-u-1,qi-$Set[Int]_prog.singleton_set_cardinality,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.in_singleton_set_equality,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-quant-u-2,qi-quant-u-115,qi-quant-u-113,qi-quant-u-3,qi-quant-u-112,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_union_in_one,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.in_intersection_in_both,qi-quant-u-4,qi-quant-u-117,qi-quant-u-5,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.in_empty_set,qi-$Set[Int]_prog.subset_definition,qi-k!643,qi-quant-u-6,qi-quant-u-119,qi-quant-u-7,qi-quant-u-127,qi-quant-u-8,qi-quant-u-126,qi-quant-u-20,qi-quant-u-9,qi-quant-u-121,qi-quant-u-21,qi-quant-u-147,qi-k!1885,qi-quant-u-10,qi-quant-u-130,qi-quant-u-128,qi-quant-u-129,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/sets_not_opaque/sets_not_opaque.gobra.vpr@428@12@430@56,qi-quant-u-137,qi-quant-u-12,qi-quant-u-136,qi-quant-u-13,qi-quant-u-132,qi-quant-u-18,qi-quant-u-145,qi-quant-u-19,qi-quant-u-144,qi-quant-u-16,qi-quant-u-143,qi-quant-u-17,qi-quant-u-141,qi-k!653,qi-quant-u-14,qi-quant-u-140,qi-quant-u-15,qi-quant-u-138,qi-quant-u-26,qi-quant-u-160,qi-quant-u-27,qi-quant-u-158,qi-quant-u-30,qi-quant-u-31,qi-quant-u-167,qi-quant-u-151,qi-quant-u-150,qi-quant-u-153,qi-quant-u-22,qi-quant-u-152,qi-quant-u-23,qi-$Set[Int]_prog.union_left_idempotency,qi-$Multiset[Int]_prog.count_card,qi-$Multiset[Int]_prog.card_non_negative,qi-$Multiset[Int]_prog.card_empty,qi-quant-u-72,qi-$Multiset[Int]_prog.singleton_unionone,qi-$Multiset[Int]_prog.card_union,qi-$Multiset[Int]_prog.count_union,qi-quant-u-73,qi-quant-u-251,qi-$Multiset[Int]_prog.card_unionone,qi-$Multiset[Int]_prog.count_empty,qi-$Multiset[Int]_prog.count_singleton,qi-$Multiset[Int]_prog.count_unionone,qi-k!4524,qi-k!4530,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/sets_not_opaque/sets_not_opaque.gobra.vpr@602@12@604@47,qi-$Set[Int]_prog.union_right_idempotency,qi-quant-u-24,qi-quant-u-156,qi-quant-u-155,qi-quant-u-25,qi-quant-u-154,qi-quant-u-88,qi-quant-u-89,qi-quant-u-284,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/sets_not_opaque/sets_not_opaque.gobra.vpr@486@6@489@41,qi-$Set[Int]_prog.intersection_right_idempotency,qi-$Set[Int]_prog.intersection_left_idempotency,qi-quant-u-162,qi-quant-u-161,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/sets_not_opaque/sets_not_opaque.gobra.vpr@219@13@221@54,qi-prog.getter_over_tuple2,qi-quant-u-44,qi-quant-u-45,qi-prog.set_ax_dec,qi-prog.set_ax_bound,qi-prog.integer_ax_dec,qi-prog.integer_ax_bound,execution_time +1,6,30,30,30,6,147,147,126,17,12,9,9,6,8,6,203,261,63,39,618,23,19,20,244,113,33,7,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.679861068725586 +1,6,30,30,30,6,150,149,122,19,13,10,10,6,8,6,196,258,61,40,603,23,19,20,240,115,35,7,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.73977780342102 +1,6,30,30,30,6,147,147,128,17,12,9,9,6,8,6,209,266,65,39,629,23,19,20,246,117,33,8,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.615512132644653 +1,6,30,30,30,6,147,147,124,17,12,9,9,6,8,6,205,261,66,39,624,23,19,20,240,117,33,8,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.836374521255493 +1,6,30,30,30,6,149,149,121,19,13,9,9,6,8,6,201,266,61,40,615,23,19,20,242,111,34,7,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.780789136886597 +1,6,30,30,30,6,149,149,117,19,13,9,9,6,8,6,192,255,55,40,590,23,19,20,235,105,34,6,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.957857131958008 +1,6,30,30,30,6,149,149,125,19,13,9,9,6,8,6,201,263,61,40,618,23,19,20,246,113,34,6,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.885841608047485 +1,6,30,30,30,6,150,149,119,19,13,10,10,6,8,6,191,257,56,40,597,23,19,20,235,113,35,6,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,13.012830018997192 +1,6,30,30,30,6,147,147,126,17,12,10,10,6,8,6,210,267,67,39,627,23,19,20,247,118,34,8,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,12,17,17,17,17,10,7,9,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,6,6,1,1,12.470592737197876 +1,6,30,30,30,6,147,147,125,17,12,9,9,6,8,6,201,262,64,39,618,23,19,20,242,112,33,7,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.679120063781738 +1,6,30,30,30,6,147,147,127,17,12,9,9,6,8,6,205,265,65,39,626,23,19,20,246,114,33,7,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,13.134193420410156 +1,6,30,30,30,6,147,147,124,17,12,9,9,6,8,6,202,258,60,39,618,23,19,20,237,116,33,7,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.548863172531128 +1,6,30,30,30,6,149,149,123,19,13,9,9,6,8,6,199,260,58,40,619,23,19,20,240,113,34,6,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,10,7,9,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,13.191413164138794 +1,6,30,30,30,6,147,147,123,17,12,9,9,6,8,6,199,255,63,39,614,23,19,20,237,116,33,7,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.992852926254272 +1,6,30,30,30,6,150,149,122,19,13,10,10,6,8,6,197,259,63,40,605,23,19,20,241,115,35,7,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.61899995803833 +1,6,30,30,30,6,147,147,127,17,12,9,9,6,8,6,209,264,64,39,631,23,19,20,246,118,33,7,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,13.570315599441528 +1,6,30,30,30,6,149,149,123,19,13,9,9,6,8,6,198,264,60,40,619,23,19,20,243,109,34,6,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.669540405273438 +1,6,30,30,30,6,149,149,121,19,13,9,9,6,8,6,196,252,60,40,599,23,19,20,235,116,34,6,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,10,7,9,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,13.075214624404907 +1,6,30,30,30,6,147,147,126,17,12,9,9,6,8,6,207,261,65,39,625,23,19,20,242,119,33,8,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,13.03046727180481 +1,6,30,30,30,6,147,147,126,17,12,9,9,6,8,6,202,262,63,39,618,23,19,20,244,113,33,7,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,10,7,9,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.889139890670776 +1,6,30,30,30,6,147,147,124,17,12,10,10,6,8,6,200,259,58,39,613,23,19,20,240,115,34,8,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.817270755767822 +1,6,30,30,30,6,147,147,125,17,12,9,9,6,8,6,209,264,66,39,628,23,19,20,247,115,33,7,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.79806113243103 +1,6,30,30,30,6,149,149,125,19,13,9,9,6,8,6,202,264,63,40,620,23,19,20,247,113,34,6,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.742648839950562 +1,6,30,30,30,6,149,149,122,19,13,9,9,6,8,6,197,259,63,40,605,23,19,20,241,112,34,6,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,10,7,9,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.64728856086731 +1,6,30,30,30,6,150,149,122,19,13,10,10,6,8,6,197,259,63,40,605,23,19,20,241,115,35,7,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.638679265975952 +1,6,30,30,30,6,147,147,126,17,12,9,9,6,8,6,205,263,67,39,622,23,19,20,246,113,33,7,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,13.057880640029907 +1,6,30,30,30,6,150,149,120,19,13,11,11,6,8,6,197,260,63,40,602,23,19,20,239,117,36,5,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.996777057647705 +1,6,30,30,30,6,149,149,123,19,13,9,9,6,8,6,203,260,61,40,619,23,19,20,240,114,34,6,7,1,4,4,4,12,13,13,4,12,12,6,6,2,3,3,2,2,3,13,18,18,18,18,8,5,7,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.647621154785156 +1,6,30,30,30,6,149,149,118,19,13,9,9,6,8,6,193,258,58,40,597,23,19,20,235,110,34,6,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,10,7,9,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,12.651878356933594 +1,6,30,30,30,6,149,149,121,19,13,9,9,6,8,6,196,260,60,40,606,23,19,20,238,110,34,6,7,1,4,4,4,12,13,13,4,13,13,6,6,2,3,3,2,2,3,13,18,18,18,18,9,6,8,1,1,1,1,1,3,7,7,7,7,5,5,5,5,1,1,1,2,2,2,2,2,2,2,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,1,1,1,1,4,4,4,1,2,1,1,1,2,3,2,1,7,6,1,1,13.0360267162323 diff --git a/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..1357c3c Binary files /dev/null and b/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..ccc7403 Binary files /dev/null and b/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque.gobra b/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque.gobra new file mode 100644 index 0000000..0673a83 --- /dev/null +++ b/evaluation/experiments/standard_library/sets_not_opaque/sets_not_opaque.gobra @@ -0,0 +1,494 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// A copy of the sets package from the standard library. +package sets_not_opaque + +// ##(-I ./..) + +import "util" + +// A set is empty if it has cardinality 0. +ghost +decreases +pure func IsEmpty(xs set[int]) bool { + return len(xs) == 0 +} + +// Returns the empty set. +ghost +ensures IsEmpty(result) +decreases +pure func Empty() (result set[int]) { + return set[int]{} +} + +// There is only one empty set. +ghost +requires IsEmpty(xs) +ensures xs == Empty() +decreases +pure func EmptyIsUnique(xs set[int]) util.Unit { + return util.Unit{} +} + +// An empty set doesn't have any elements. +ghost +requires IsEmpty(xs) +ensures !(e in xs) +decreases +pure func NotInEmpty(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// A set is a singleton if it has cardinality 1. +ghost +decreases +pure func IsSingleton(xs set[int]) bool { + return len(xs) == 1 +} + +// Returns a singleton containing x. +ghost +ensures IsSingleton(result) +ensures e in result +decreases +pure func SingletonSet(e int) (result set[int]) { + return set[int]{e} +} + +// If a is in a singleton set x, then x is of the form {a}. +ghost +requires IsSingleton(xs) +requires e in xs +ensures xs == SingletonSet(e) +decreases +pure func SingletonIsSingletonSet(xs set[int], e int) util.Unit { + return let _ := Choose(xs) in util.Unit{} +} + +// Elements in a singleton set are equal to each other. +ghost +requires IsSingleton(xs) +requires a in xs +requires b in xs +ensures a == b +decreases +pure func SingletonEquality(xs set[int], a int, b int) util.Unit { + return let _ := Choose(xs) in util.Unit{} +} + +// Constructs a set with all integers in the range [a, b). +ghost +requires a <= b +ensures forall i int :: { i in result } (a <= i && i < b) == i in result +ensures len(result) == b - a +decreases b - a +pure func Range(a, b int) (result set[int]) { + return a == b ? Empty() : Add(Range(a + 1, b), a) +} + +// Constructs a set with all integers in the range [0, n). +ghost +requires n >= 0 +ensures forall i int :: { i in result } (0 <= i && i < n) == i in result +ensures len(result) == n +decreases +pure func RangeFromZero(n int) (result set[int]) { + return Range(0, n) +} + +// Converts a set into a multiset where each element from the set has +// multiplicity 1 and any other element has multiplicity 0. +ghost +ensures forall i int :: {i # result} (i in s) ==> ((i # result) == 1) +ensures forall i int :: {i # result} (!(i in s)) ==> ((i # result) == 0) +decreases s +pure func ToMultiset(s set[int]) (result mset[int]) { + return IsEmpty(s) ? mset[int] {} : + let x := Choose(s) in + ((mset[int] {}) union (mset[int] {x})) union ToMultiset(Remove(s, x)) +} + +// Returns an element from a non-empty set. +ghost +requires !IsEmpty(xs) +ensures e in xs +ensures IsSingleton(xs) ==> xs == SingletonSet(e) +decreases +pure func Choose(xs set[int]) (e int) + +// Returns whether xs and ys are disjoint sets. +ghost +decreases +pure func AreDisjoint(xs, ys set[int]) bool { + return IsEmpty(xs intersection ys) +} + +// Definition of set equality. +ghost +ensures (xs == ys) == (forall e int :: {e in xs} {e in ys} ((e in xs) == (e in ys))) +decreases +pure func SetEquality(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// Definition of subset without quantifiers. +ghost +requires e in xs +requires xs subset ys +ensures e in ys +decreases +pure func InSubset(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// Subset relation is reflexive. +ghost +ensures xs subset xs +decreases +pure func SubsetIsReflexive(xs set[int]) util.Unit { + return util.Unit{} +} + +// Subset relation is transitive. +ghost +requires xs subset ys +requires ys subset zs +ensures xs subset zs +decreases +pure func SubsetIsTransitive(xs, ys, zs set[int]) util.Unit { + return util.Unit{} +} + +// If xs is a subset of ys and both have the same cardinality, they are equal. +ghost +requires xs subset ys +requires len(xs) == len(ys) +ensures xs == ys +decreases +pure func SubsetEquality(xs, ys set[int]) util.Unit { + return util.Asserting(len(ys setminus xs) == len(ys) - len(xs)) +} + +// Returns whether xs is a proper subset of ys. +ghost +decreases +pure func IsProperSubset(xs, ys set[int]) bool { + return xs subset ys && xs != ys +} + +// If e is in the union of xs and ys, then it must be in xs or ys. +ghost +ensures (e in (xs union ys)) == ((e in xs) || (e in ys)) +decreases +pure func InUnionInOne(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// Union is commutative. +ghost +ensures (xs union ys) == (ys union xs) +decreases +pure func UnionIsCommutative(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// Union is idempotent. +ghost +ensures (xs union ys) union ys == xs union ys +ensures xs union (xs union ys) == xs union ys +decreases +pure func UnionIsIdempotent(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// Add x to xs. +ghost +// Need this post-condition first to ensure the properties about the length. +ensures (e in xs) ==> result == xs +ensures (e in xs) ==> (len(result) == len(xs)) +ensures !(e in xs) ==> (len(result) == len(xs) + 1) +ensures e in result +decreases +pure func Add(xs set[int], e int) (result set[int]) { + return xs union SingletonSet(e) +} + +// If a is in xs union {b}, then a is equal to b, or a was already in xs. +ghost +ensures (a in Add(xs, b)) == ((a == b) || a in xs) +decreases +pure func InAdd(xs set[int], a, b int) util.Unit { + return util.Unit{} +} + +// If a is in xs, then a will remain in xs no matter what we add to it. +ghost +requires a in xs +ensures a in Add(xs, b) +decreases +pure func InvarianceInAdd(xs set[int], a, b int) util.Unit { + return util.Unit{} +} + +// Remove e from xs. Does not require e to be in xs. +ghost +ensures !(e in xs) ==> result == xs +ensures (e in xs) ==> (len(result) == len(xs) - 1) +ensures !(e in xs) ==> (len(result) == len(xs)) +decreases +pure func Remove(xs set[int], e int) (result set[int]) { + return xs setminus SingletonSet(e) +} + +// Intersection is commutative. +ghost +ensures (xs intersection ys) == (ys intersection xs) +decreases +pure func IntersectionIsCommutative(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// Intersection is idempotent. +ghost +ensures (xs intersection ys) intersection ys == (xs intersection ys) +ensures xs intersection (xs intersection ys) == (xs intersection ys) +decreases +pure func IntersectionIsIdempotent(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// If e is in the difference xs - ys, then e must be in xs but not in ys. +ghost +ensures (e in (xs setminus ys)) == ((e in xs) && !(e in ys)) +decreases +pure func InDifference(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// If ys contains e, then the difference xs - ys does not contain e. +ghost +requires e in ys +ensures !(e in (xs setminus ys)) +decreases +pure func NotInDifference(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// If e is in the intersection of xs and ys, then e must be both in xs and ys. +ghost +ensures e in (xs intersection ys) == ((e in xs) && (e in ys)) +decreases +pure func InIntersectionInBoth(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// If xs and ys are disjoint, adding and then removing one from the other +// yields the original set. +ghost +requires AreDisjoint(xs, ys) +ensures (xs union ys) setminus xs == ys +ensures (xs union ys) setminus ys == xs +decreases +pure func DisjointUnionDifference(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// If e is in xs, removing and adding it back yields the original set. +ghost +requires e in xs +ensures Add(Remove(xs, e), e) == xs +decreases +pure func AddRemove(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// If we remove e from the set xs, it doesn't matter whether we have added e +// to it before. +ghost +ensures Remove(Add(xs, e), e) == Remove(xs, e) +decreases +pure func RemoveAdd(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// xs - {e} is a subset of xs. +ghost +ensures Remove(xs, e) subset xs +decreases +pure func SubsetRemove(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// xs and ys are subsets of the union of xs and ys. +ghost +ensures xs subset (xs union ys) +ensures ys subset (xs union ys) +decreases +pure func SubsetUnion(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// The intersection of xs and ys are subsets of xs, and ys. +ghost +ensures (xs intersection ys) subset xs +ensures (xs intersection ys) subset ys +decreases +pure func SubsetIntersection(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// The difference xs - ys is a subset of xs. +ghost +ensures (xs setminus ys) subset xs +decreases +pure func SubsetDifference(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// The cardinality of a set is non-negative. +ghost +ensures len(xs) >= 0 +decreases +pure func NonNegativeLen(xs set[int]) util.Unit { + return util.Unit{} +} + +// If xs is a subset of ys, then the cardinality of xs is less than or equal to the cardinality of ys. +// If xs is a strict subset of ys, then the cardinality of xs is less than the cardinality of ys. +ghost +decreases xs, ys +ensures xs subset ys ==> len(xs) <= len(ys) +ensures IsProperSubset(xs, ys) ==> len(xs) < len(ys) +pure func SubsetLen(xs, ys set[int]) util.Unit { + return (!(xs subset ys) || len(xs) == 0) ? util.Unit{} : + len(xs) == len(ys) ? + let _ := SubsetEquality(xs, ys) in + (let e := Choose(xs) in + (SubsetLen(Remove(xs, e), Remove(ys, e)))) : + + let e:= Choose(xs) in + (SubsetLen(Remove(xs, e), Remove(ys, e))) +} + +// The cardinality of a union of two sets is greater than or equal to the cardinality of +// either individual set. +ghost +ensures len(xs union ys) >= len(xs) +ensures len(xs union ys) >= len(ys) +decreases ys +pure func UnionLenLower(xs, ys set[int]) util.Unit { + return IsEmpty(ys) ? util.Unit{} : + let y := Choose(ys) in + (let yr := Remove(ys, y) in + (y in xs ? + (let xr := Remove(xs, y) in + (let _ := util.Asserting(xr union yr == Remove(xs union ys, y)) in UnionLenLower(xr, yr))) : + (let _ := util.Asserting(xs union yr == Remove(xs union ys, y)) in UnionLenLower(xs, yr)))) +} + +// The cardinality of a union of two sets is less than or equal to the cardinality of +// both individual sets combined. +ghost +ensures len(xs union ys) <= len(xs) + len(ys) +decreases +pure func UnionLenUpper(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// The cardinality of the intersection of xs and ys is less than or equal to the +// cardinality of xs. +ghost +ensures len(xs intersection ys) <= len(xs) +decreases xs +pure func IntersectLenUpper(xs, ys set[int]) util.Unit { + return IsEmpty(xs) ? util.Unit{} : + let x := Choose(xs) in + (let _ := util.Asserting((Remove(xs, x)) intersection ys == Remove((xs intersection ys), x)) in + (IntersectLenUpper(Remove(xs, x), ys))) +} + +// The cardinality of the difference xs - ys is less than or equal to the cardinality of xs. +ghost +ensures len(xs setminus ys) <= len(xs) +decreases +pure func DifferenceLenUpper(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +ghost +ensures len(xs union ys) == len(xs) + len(ys) - len(xs intersection ys) +decreases +pure func UnionLenEq(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +ghost +ensures len(xs setminus ys) == len(xs) - len(xs intersection ys) +ensures len(xs setminus ys) + len(ys setminus xs) + len(xs intersection ys) == len(xs union ys) +decreases +pure func DifferenceLenEq(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// If e is in xs, adding it to xs doesn't change the cardinality. +// If e is not in xs, adding it to xs increases the cardinality by 1. +ghost +ensures (e in xs) ==> (len(Add(xs, e)) == len(xs)) +ensures !(e in xs) ==> (len(Add(xs, e)) == len(xs) + 1) +decreases +pure func AddLen(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// If e is in xs, removing it from xs reduces the cardinality by 1. +// If e is not in xs, removing it from xs doesn't change the cardinality. +ghost +ensures (e in xs) ==> (len(Remove(xs, e)) == len(xs) - 1) +ensures !(e in xs) ==> (len(Remove(xs, e)) == len(xs)) +decreases +pure func RemoveLen(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// Remove is right distributive over union. +ghost +ensures Remove(xs union ys, e) == Remove(xs, e) union Remove(ys, e) +decreases +pure func RemoveUnion(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// If e is in (xs union ys), removing it from both sets reduces the cardinality by 1. +ghost +requires e in (xs union ys) +ensures len(Remove(xs, e) union Remove(ys, e)) == len(xs union ys) - 1 +decreases +pure func RemoveUnionLen(xs, ys set[int], e int) util.Unit { + return RemoveUnion(xs, ys, e) +} + +// If xs and ys are disjoint, the cardinality of their union is equal to the +// sum of the cardinality of xs, and the cardinality of ys. +ghost +requires AreDisjoint(xs, ys) +ensures len(xs union ys) == len(xs) + len(ys) +decreases +pure func DisjointUnionLen(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// If xs solely contains integers in the range [a, b), then its size is +// bounded by b - a. +ghost +requires forall i int :: { i in xs } i in xs ==> (a <= i && i < b) +requires a <= b +ensures len(xs) <= b - a +decreases +pure func BoundedSetLen(xs set[int], a, b int) util.Unit { + return SubsetLen(xs, Range(a, b)) +} diff --git a/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..7c9ffd9 --- /dev/null +++ b/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!881,qi-$Set[Int]_prog.in_singleton_set,qi-quant-u-0,qi-quant-u-110,qi-quant-u-1,qi-$Set[Int]_prog.singleton_set_cardinality,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.in_singleton_set_equality,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-quant-u-2,qi-quant-u-115,qi-quant-u-113,qi-quant-u-3,qi-quant-u-112,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_union_in_one,qi-$Set[Int]_prog.in_empty_set,qi-quant-u-4,qi-quant-u-117,qi-quant-u-5,qi-$Set[Int]_prog.subset_definition,qi-k!643,qi-quant-u-6,qi-quant-u-119,qi-quant-u-7,qi-quant-u-126,qi-quant-u-8,qi-quant-u-125,qi-quant-u-9,qi-quant-u-121,qi-quant-u-133,qi-quant-u-10,qi-quant-u-132,qi-quant-u-24,qi-quant-u-11,qi-quant-u-127,qi-quant-u-25,qi-quant-u-153,qi-k!2008,qi-quant-u-12,qi-quant-u-136,qi-quant-u-134,qi-quant-u-135,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/sets_opaque/sets_opaque.gobra.vpr@448@12@450@56,qi-quant-u-18,qi-quant-u-145,qi-quant-u-19,qi-quant-u-17,qi-quant-u-141,qi-quant-u-14,qi-quant-u-140,qi-quant-u-15,qi-quant-u-138,qi-quant-u-160,qi-quant-u-27,qi-quant-u-158,qi-quant-u-148,qi-quant-u-20,qi-quant-u-147,qi-quant-u-21,qi-$Set[Int]_prog.union_left_idempotency,qi-quant-u-150,qi-quant-u-23,qi-quant-u-149,qi-$Set[Int]_prog.intersection_right_idempotency,qi-$Set[Int]_prog.intersection_left_idempotency,qi-quant-u-157,qi-quant-u-156,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/sets_opaque/sets_opaque.gobra.vpr@222@13@224@54,qi-k!653,qi-quant-u-69,qi-quant-u-241,qi-quant-u-79,qi-quant-u-262,qi-$Multiset[Int]_prog.count_card,qi-$Multiset[Int]_prog.card_non_negative,qi-$Multiset[Int]_prog.card_empty,qi-quant-u-92,qi-$Multiset[Int]_prog.singleton_unionone,qi-$Multiset[Int]_prog.card_union,qi-$Multiset[Int]_prog.count_union,qi-quant-u-93,qi-quant-u-293,qi-$Multiset[Int]_prog.card_unionone,qi-$Multiset[Int]_prog.count_empty,qi-$Multiset[Int]_prog.count_singleton,qi-$Multiset[Int]_prog.count_unionone,qi-k!5521,qi-k!5527,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/sets_opaque/sets_opaque.gobra.vpr@637@12@639@47,qi-$Set[Int]_prog.union_right_idempotency,qi-prog./home/daniel/practical-work/gobra-libs-eval/report/code/standard_library/sets_opaque/sets_opaque.gobra.vpr@511@6@514@41,qi-prog.getter_over_tuple2,qi-quant-u-40,qi-quant-u-41,qi-prog.set_ax_dec,qi-prog.set_ax_bound,qi-prog.integer_ax_dec,qi-prog.integer_ax_bound,execution_time +1,6,30,30,30,6,146,145,120,19,13,11,11,6,8,6,225,111,34,39,583,248,184,55,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,13.186906099319458 +1,6,30,30,30,6,146,145,123,19,13,11,11,6,8,6,228,118,34,39,606,255,194,57,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,10,7,9,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.737457036972046 +1,6,30,30,30,6,144,143,126,17,12,11,11,6,8,6,232,115,33,38,611,257,198,61,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,12,9,11,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.656487464904785 +1,6,30,30,30,6,144,143,127,17,12,11,11,6,8,6,233,120,33,38,615,255,200,64,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,13,10,12,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.932893991470337 +1,6,30,30,30,6,144,143,125,17,12,11,11,6,8,6,234,119,33,38,613,256,203,66,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,9,6,8,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.736277341842651 +1,6,30,30,30,6,146,145,123,19,13,11,11,6,8,6,230,116,34,39,607,254,195,59,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,10,7,9,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.941861152648926 +1,6,30,30,30,6,144,143,126,17,12,11,11,6,8,6,233,115,33,38,613,258,199,63,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,9,6,8,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.93529748916626 +1,6,30,30,30,6,144,143,126,17,12,11,11,6,8,6,233,115,33,38,613,260,197,63,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,13,10,12,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.75205945968628 +1,6,30,30,30,6,146,145,123,19,13,11,11,6,8,6,232,116,34,39,600,256,192,64,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,8,5,7,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,13.304299354553223 +1,6,30,30,30,6,146,145,118,19,13,11,11,6,8,6,226,110,34,39,581,248,184,57,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,10,7,9,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.706437349319458 +1,6,30,30,30,6,144,143,125,17,12,11,11,6,8,6,227,120,33,38,608,251,196,60,8,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,13.408269882202148 +1,6,30,30,30,6,146,145,119,19,13,11,11,6,8,6,219,119,34,39,586,242,186,56,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,12,9,11,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.654552936553955 +1,6,30,30,30,6,144,143,124,17,12,11,11,6,8,6,231,116,33,38,605,256,195,59,8,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.997634172439575 +1,6,30,30,30,6,144,143,123,17,12,11,11,6,8,6,231,116,33,38,599,250,196,61,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.693997383117676 +1,6,30,30,30,6,146,145,122,19,13,11,11,6,8,6,233,113,34,39,604,258,195,59,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,13,10,12,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.928714990615845 +1,6,30,30,30,6,144,143,125,17,12,11,11,6,8,6,227,119,33,38,608,251,196,60,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,10,7,9,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.775447607040405 +1,6,30,30,30,6,146,145,121,19,13,11,11,6,8,6,227,112,34,39,591,252,186,57,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,8,5,7,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.863157033920288 +1,6,30,30,30,6,144,143,127,17,12,11,11,6,8,6,233,120,33,38,615,255,200,64,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.789527893066406 +1,6,30,30,30,6,146,145,124,19,13,11,11,6,8,6,233,113,34,39,608,258,195,59,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,12,9,11,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.729537725448608 +1,6,30,30,30,6,144,143,126,17,12,11,11,6,8,6,232,115,33,38,611,257,198,61,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,10,7,9,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.979766130447388 +1,6,30,30,30,6,144,143,128,17,12,11,11,6,8,6,237,117,33,38,616,259,200,64,8,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,10,7,9,1,1,7,7,7,7,4,4,4,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.985901594161987 +1,6,30,30,30,6,146,145,121,19,13,11,11,6,8,6,229,112,34,39,595,254,188,61,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,13,10,12,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.859172821044922 +1,6,30,30,30,6,144,143,127,17,12,11,11,6,8,6,235,120,33,38,619,257,202,68,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,13.019877433776855 +1,6,30,30,30,6,146,145,121,19,13,11,11,6,8,6,226,114,34,39,591,252,186,57,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.705912590026855 +1,6,30,30,30,6,144,143,127,17,12,11,11,6,8,6,233,120,33,38,615,255,200,64,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.775066137313843 +1,6,30,30,30,6,146,145,120,19,13,11,11,6,8,6,232,111,34,39,590,254,191,63,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,13.206603288650513 +1,6,30,30,30,6,146,145,120,19,13,11,11,6,8,6,227,111,34,39,587,250,186,59,6,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,13,10,12,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.766444683074951 +1,6,30,30,30,6,144,143,123,17,12,11,11,6,8,6,232,113,33,38,602,253,196,62,8,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,12,9,11,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.723960638046265 +1,6,30,30,30,6,144,143,127,17,12,11,11,6,8,6,235,116,33,38,608,255,198,62,8,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,14,11,12,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.88353419303894 +1,6,30,30,30,6,146,145,125,19,13,11,11,6,8,6,238,115,34,39,612,261,198,66,7,23,19,20,7,1,3,3,3,11,16,16,16,16,12,13,13,4,13,13,6,6,4,3,3,2,2,3,11,8,10,1,1,7,7,7,7,6,6,6,2,2,2,2,2,1,1,1,2,1,2,2,3,1,1,1,4,4,4,2,2,2,2,4,10,2,2,2,7,8,8,3,3,1,3,1,3,2,1,6,6,1,1,12.844374179840088 diff --git a/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..b980e72 Binary files /dev/null and b/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..643e294 Binary files /dev/null and b/evaluation/experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/standard_library/sets_opaque/sets_opaque.gobra b/evaluation/experiments/standard_library/sets_opaque/sets_opaque.gobra new file mode 100644 index 0000000..29c7690 --- /dev/null +++ b/evaluation/experiments/standard_library/sets_opaque/sets_opaque.gobra @@ -0,0 +1,535 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// Copy of the sets package from the standard library where all lemmas are +// turned opaque. +package sets_opaque + +// ##(-I ./..) + +import "util" + +// A set is empty if it has cardinality 0. +ghost +decreases +pure func IsEmpty(xs set[int]) bool { + return len(xs) == 0 +} + +// Returns the empty set. +ghost +ensures IsEmpty(result) +decreases +pure func Empty() (result set[int]) { + return set[int]{} +} + +// There is only one empty set. +ghost +opaque +requires IsEmpty(xs) +ensures xs == Empty() +decreases +pure func EmptyIsUnique(xs set[int]) util.Unit { + return util.Unit{} +} + +// An empty set doesn't have any elements. +ghost +opaque +requires IsEmpty(xs) +ensures !(e in xs) +decreases +pure func NotInEmpty(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// A set is a singleton if it has cardinality 1. +ghost +decreases +pure func IsSingleton(xs set[int]) bool { + return len(xs) == 1 +} + +// Returns a singleton containing x. +ghost +ensures IsSingleton(result) +ensures e in result +decreases +pure func SingletonSet(e int) (result set[int]) { + return set[int]{e} +} + +// If a is in a singleton set x, then x is of the form {a}. +ghost +opaque +requires IsSingleton(xs) +requires e in xs +ensures xs == SingletonSet(e) +decreases +pure func SingletonIsSingletonSet(xs set[int], e int) util.Unit { + return let _ := Choose(xs) in util.Unit{} +} + +// Elements in a singleton set are equal to each other. +ghost +opaque +requires IsSingleton(xs) +requires a in xs +requires b in xs +ensures a == b +decreases +pure func SingletonEquality(xs set[int], a int, b int) util.Unit { + return let _ := Choose(xs) in util.Unit{} +} + +// Constructs a set with all integers in the range [a, b). +ghost +requires a <= b +ensures forall i int :: { i in result } (a <= i && i < b) == i in result +ensures len(result) == b - a +decreases b - a +pure func Range(a, b int) (result set[int]) { + return a == b ? Empty() : Add(Range(a + 1, b), a) +} + +// Constructs a set with all integers in the range [0, n). +ghost +requires n >= 0 +ensures forall i int :: { i in result } (0 <= i && i < n) == i in result +ensures len(result) == n +decreases +pure func RangeFromZero(n int) (result set[int]) { + return Range(0, n) +} + +// Converts a set into a multiset where each element from the set has +// multiplicity 1 and any other element has multiplicity 0. +ghost +ensures forall i int :: {i # result} (i in s) ==> ((i # result) == 1) +ensures forall i int :: {i # result} (!(i in s)) ==> ((i # result) == 0) +decreases s +pure func ToMultiset(s set[int]) (result mset[int]) { + return IsEmpty(s) ? mset[int] {} : + let x := Choose(s) in + ((mset[int] {}) union (mset[int] {x})) union ToMultiset(Remove(s, x)) +} + +// Returns an element from a non-empty set. +ghost +requires !IsEmpty(xs) +ensures e in xs +ensures IsSingleton(xs) ==> xs == SingletonSet(e) +decreases +pure func Choose(xs set[int]) (e int) + +// Returns whether xs and ys are disjoint sets. +ghost +decreases +pure func AreDisjoint(xs, ys set[int]) bool { + return IsEmpty(xs intersection ys) +} + +// Definition of set equality. +ghost +opaque +ensures (xs == ys) == (forall e int :: {e in xs} {e in ys} ((e in xs) == (e in ys))) +decreases +pure func SetEquality(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// Definition of subset without quantifiers. +ghost +opaque +requires e in xs +requires xs subset ys +ensures e in ys +decreases +pure func InSubset(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// Subset relation is reflexive. +ghost +opaque +ensures xs subset xs +decreases +pure func SubsetIsReflexive(xs set[int]) util.Unit { + return util.Unit{} +} + +// Subset relation is transitive. +ghost +opaque +requires xs subset ys +requires ys subset zs +ensures xs subset zs +decreases +pure func SubsetIsTransitive(xs, ys, zs set[int]) util.Unit { + return util.Unit{} +} + +// If xs is a subset of ys and both have the same cardinality, they are equal. +ghost +opaque +requires xs subset ys +requires len(xs) == len(ys) +ensures xs == ys +decreases +pure func SubsetEquality(xs, ys set[int]) util.Unit { + return util.Asserting(len(ys setminus xs) == len(ys) - len(xs)) +} + +// Returns whether xs is a proper subset of ys. +ghost +decreases +pure func IsProperSubset(xs, ys set[int]) bool { + return xs subset ys && xs != ys +} + +// If e is in the union of xs and ys, then it must be in xs or ys. +ghost +opaque +ensures (e in (xs union ys)) == ((e in xs) || (e in ys)) +decreases +pure func InUnionInOne(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// Union is commutative. +ghost +opaque +ensures (xs union ys) == (ys union xs) +decreases +pure func UnionIsCommutative(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// Union is idempotent. +ghost +opaque +ensures (xs union ys) union ys == xs union ys +ensures xs union (xs union ys) == xs union ys +decreases +pure func UnionIsIdempotent(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// Add x to xs. +ghost +// Need this post-condition first to ensure the properties about the length. +ensures (e in xs) ==> result == xs +ensures (e in xs) ==> (len(result) == len(xs)) +ensures !(e in xs) ==> (len(result) == len(xs) + 1) +ensures e in result +decreases +pure func Add(xs set[int], e int) (result set[int]) { + return xs union SingletonSet(e) +} + +// If a is in xs union {b}, then a is equal to b, or a was already in xs. +ghost +opaque +ensures (a in Add(xs, b)) == ((a == b) || a in xs) +decreases +pure func InAdd(xs set[int], a, b int) util.Unit { + return util.Unit{} +} + +// If a is in xs, then a will remain in xs no matter what we add to it. +ghost +opaque +requires a in xs +ensures a in Add(xs, b) +decreases +pure func InvarianceInAdd(xs set[int], a, b int) util.Unit { + return util.Unit{} +} + +// Remove e from xs. Does not require e to be in xs. +ghost +ensures !(e in xs) ==> result == xs +ensures (e in xs) ==> (len(result) == len(xs) - 1) +ensures !(e in xs) ==> (len(result) == len(xs)) +decreases +pure func Remove(xs set[int], e int) (result set[int]) { + return xs setminus SingletonSet(e) +} + +// Intersection is commutative. +ghost +opaque +ensures (xs intersection ys) == (ys intersection xs) +decreases +pure func IntersectionIsCommutative(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// Intersection is idempotent. +ghost +opaque +ensures (xs intersection ys) intersection ys == (xs intersection ys) +ensures xs intersection (xs intersection ys) == (xs intersection ys) +decreases +pure func IntersectionIsIdempotent(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// If e is in the difference xs - ys, then e must be in xs but not in ys. +ghost +opaque +ensures (e in (xs setminus ys)) == ((e in xs) && !(e in ys)) +decreases +pure func InDifference(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// If ys contains e, then the difference xs - ys does not contain e. +ghost +opaque +requires e in ys +ensures !(e in (xs setminus ys)) +decreases +pure func NotInDifference(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// If e is in the intersection of xs and ys, then e must be both in xs and ys. +ghost +opaque +ensures e in (xs intersection ys) == ((e in xs) && (e in ys)) +decreases +pure func InIntersectionInBoth(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// If xs and ys are disjoint, adding and then removing one from the other +// yields the original set. +ghost +opaque +requires AreDisjoint(xs, ys) +ensures (xs union ys) setminus xs == ys +ensures (xs union ys) setminus ys == xs +decreases +pure func DisjointUnionDifference(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// If e is in xs, removing and adding it back yields the original set. +ghost +opaque +requires e in xs +ensures Add(Remove(xs, e), e) == xs +decreases +pure func AddRemove(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// If we remove e from the set xs, it doesn't matter whether we have added e +// to it before. +ghost +opaque +ensures Remove(Add(xs, e), e) == Remove(xs, e) +decreases +pure func RemoveAdd(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// xs - {e} is a subset of xs. +ghost +opaque +ensures Remove(xs, e) subset xs +decreases +pure func SubsetRemove(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// xs and ys are subsets of the union of xs and ys. +ghost +opaque +ensures xs subset (xs union ys) +ensures ys subset (xs union ys) +decreases +pure func SubsetUnion(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// The intersection of xs and ys are subsets of xs, and ys. +ghost +opaque +ensures (xs intersection ys) subset xs +ensures (xs intersection ys) subset ys +decreases +pure func SubsetIntersection(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// The difference xs - ys is a subset of xs. +ghost +opaque +ensures (xs setminus ys) subset xs +decreases +pure func SubsetDifference(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// The cardinality of a set is non-negative. +ghost +opaque +ensures len(xs) >= 0 +decreases +pure func NonNegativeLen(xs set[int]) util.Unit { + return util.Unit{} +} + +// If xs is a subset of ys, then the cardinality of xs is less than or equal to the cardinality of ys. +// If xs is a strict subset of ys, then the cardinality of xs is less than the cardinality of ys. +ghost +opaque +decreases xs, ys +ensures xs subset ys ==> len(xs) <= len(ys) +ensures IsProperSubset(xs, ys) ==> len(xs) < len(ys) +pure func SubsetLen(xs, ys set[int]) util.Unit { + return (!(xs subset ys) || len(xs) == 0) ? util.Unit{} : + len(xs) == len(ys) ? + let _ := SubsetEquality(xs, ys) in + (let e := Choose(xs) in + (SubsetLen(Remove(xs, e), Remove(ys, e)))) : + + let e:= Choose(xs) in + (SubsetLen(Remove(xs, e), Remove(ys, e))) +} + +// The cardinality of a union of two sets is greater than or equal to the cardinality of +// either individual set. +ghost +opaque +ensures len(xs union ys) >= len(xs) +ensures len(xs union ys) >= len(ys) +decreases ys +pure func UnionLenLower(xs, ys set[int]) util.Unit { + return IsEmpty(ys) ? util.Unit{} : + let y := Choose(ys) in + (let yr := Remove(ys, y) in + (y in xs ? + (let xr := Remove(xs, y) in + (let _ := util.Asserting(xr union yr == Remove(xs union ys, y)) in UnionLenLower(xr, yr))) : + (let _ := util.Asserting(xs union yr == Remove(xs union ys, y)) in UnionLenLower(xs, yr)))) +} + +// The cardinality of a union of two sets is less than or equal to the cardinality of +// both individual sets combined. +ghost +opaque +ensures len(xs union ys) <= len(xs) + len(ys) +decreases +pure func UnionLenUpper(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// The cardinality of the intersection of xs and ys is less than or equal to the +// cardinality of xs. +ghost +opaque +ensures len(xs intersection ys) <= len(xs) +decreases xs +pure func IntersectLenUpper(xs, ys set[int]) util.Unit { + return IsEmpty(xs) ? util.Unit{} : + let x := Choose(xs) in + (let _ := util.Asserting((Remove(xs, x)) intersection ys == Remove((xs intersection ys), x)) in + (IntersectLenUpper(Remove(xs, x), ys))) +} + +// The cardinality of the difference xs - ys is less than or equal to the cardinality of xs. +ghost +opaque +ensures len(xs setminus ys) <= len(xs) +decreases +pure func DifferenceLenUpper(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +ghost +opaque +ensures len(xs union ys) == len(xs) + len(ys) - len(xs intersection ys) +decreases +pure func UnionLenEq(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +ghost +opaque +ensures len(xs setminus ys) == len(xs) - len(xs intersection ys) +ensures len(xs setminus ys) + len(ys setminus xs) + len(xs intersection ys) == len(xs union ys) +decreases +pure func DifferenceLenEq(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// If e is in xs, adding it to xs doesn't change the cardinality. +// If e is not in xs, adding it to xs increases the cardinality by 1. +ghost +opaque +ensures (e in xs) ==> (len(Add(xs, e)) == len(xs)) +ensures !(e in xs) ==> (len(Add(xs, e)) == len(xs) + 1) +decreases +pure func AddLen(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// If e is in xs, removing it from xs reduces the cardinality by 1. +// If e is not in xs, removing it from xs doesn't change the cardinality. +ghost +opaque +ensures (e in xs) ==> (len(Remove(xs, e)) == len(xs) - 1) +ensures !(e in xs) ==> (len(Remove(xs, e)) == len(xs)) +decreases +pure func RemoveLen(xs set[int], e int) util.Unit { + return util.Unit{} +} + +// Remove is right distributive over union. +ghost +opaque +ensures Remove(xs union ys, e) == Remove(xs, e) union Remove(ys, e) +decreases +pure func RemoveUnion(xs, ys set[int], e int) util.Unit { + return util.Unit{} +} + +// If e is in (xs union ys), removing it from both sets reduces the cardinality by 1. +ghost +opaque +requires e in (xs union ys) +ensures len(Remove(xs, e) union Remove(ys, e)) == len(xs union ys) - 1 +decreases +pure func RemoveUnionLen(xs, ys set[int], e int) util.Unit { + return RemoveUnion(xs, ys, e) +} + +// If xs and ys are disjoint, the cardinality of their union is equal to the +// sum of the cardinality of xs, and the cardinality of ys. +ghost +opaque +requires AreDisjoint(xs, ys) +ensures len(xs union ys) == len(xs) + len(ys) +decreases +pure func DisjointUnionLen(xs, ys set[int]) util.Unit { + return util.Unit{} +} + +// If xs solely contains integers in the range [a, b), then its size is +// bounded by b - a. +ghost +opaque +requires forall i int :: { i in xs } i in xs ==> (a <= i && i < b) +requires a <= b +ensures len(xs) <= b - a +decreases +pure func BoundedSetLen(xs set[int], a, b int) util.Unit { + return SubsetLen(xs, Range(a, b)) +} diff --git a/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..1ccf268 --- /dev/null +++ b/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!282,qi-prog.getter_over_tuple2,qi-quant-u-2,qi-quant-u-3,execution_time +1,3,2,1,8.335532665252686 +1,3,2,1,8.26456594467163 +1,3,2,1,8.425381183624268 +1,3,2,1,8.322588920593262 +1,3,2,1,8.393415927886963 +1,3,2,1,8.249688148498535 +1,3,2,1,8.442358016967773 +1,3,2,1,8.977903366088867 +1,3,2,1,8.78366994857788 +1,3,2,1,8.594573259353638 +1,3,2,1,8.3912034034729 +1,3,2,1,8.748830318450928 +1,3,2,1,8.379246234893799 +1,3,2,1,8.458815336227417 +1,3,2,1,8.532689332962036 +1,3,2,1,8.356615543365479 +1,3,2,1,8.422563552856445 +1,3,2,1,8.405136823654175 +1,3,2,1,8.30770230293274 +1,3,2,1,8.362667798995972 +1,3,2,1,8.493568420410156 +1,3,2,1,8.470894813537598 +1,3,2,1,8.551515340805054 +1,3,2,1,8.3363516330719 +1,3,2,1,8.233821392059326 +1,3,2,1,8.448015928268433 +1,3,2,1,8.614681243896484 +1,3,2,1,8.452851057052612 +1,3,2,1,8.353624105453491 +1,3,2,1,8.54117488861084 diff --git a/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..63d1eaa Binary files /dev/null and b/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..fd38912 Binary files /dev/null and b/evaluation/experiments/standard_library/util/util-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/standard_library/util/util.gobra b/evaluation/experiments/standard_library/util/util.gobra new file mode 100644 index 0000000..2357870 --- /dev/null +++ b/evaluation/experiments/standard_library/util/util.gobra @@ -0,0 +1,44 @@ +// Copyright 2023 ETH Zurich +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package util + +type Unit struct{} + +// Useful for marking functions as never called, by putting it in the precondition +const Uncallable bool = false + +ghost +requires false +decreases +pure func Unreachable() Unit { + return Unit{} +} + +ghost +ensures false +decreases +func IgnoreBranch() + +ghost +ensures false +decreases +func TODO() + +ghost +requires b +decreases +pure func Asserting(ghost b bool) Unit { + return Unit{} +} diff --git a/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..1d61bf6 --- /dev/null +++ b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!456,qi-prog.getter_over_tuple2,qi-quant-u-4,qi-quant-u-5,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-quant-u-6,qi-quant-u-7,qi-quant-u-12,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_union_in_one,qi-quant-u-2,qi-quant-u-3,qi-quant-u-10,qi-quant-u-0,qi-quant-u-1,qi-quant-u-8,qi-k!369,qi-$Set[Int]_prog.in_empty_set,execution_time +1,3,2,1,18,18,2,2,2,32,30,30,18,546,1957,650,775,817,469,7,7,7,7,7,7,52,52,9.538095712661743 +1,3,2,1,18,18,2,2,2,32,30,30,18,583,2116,706,847,890,502,7,7,7,7,7,7,53,59,9.574820280075073 +1,3,2,1,18,18,2,2,2,32,30,30,18,543,1949,652,765,827,470,7,7,7,7,7,7,52,52,9.544771194458008 +1,3,2,1,18,18,2,2,2,32,30,30,18,524,1918,632,778,799,459,7,7,7,7,7,7,56,55,9.67936110496521 +1,3,2,1,18,18,2,2,2,32,30,30,18,800,2925,916,1062,1205,635,7,7,7,7,7,7,71,95,9.677863836288452 +1,3,2,1,18,18,2,2,2,32,30,30,18,500,1943,625,740,811,433,7,7,7,7,7,7,51,53,9.421724081039429 +1,3,2,1,18,18,2,2,2,32,30,30,18,472,1795,590,701,762,425,7,7,7,7,7,7,52,52,9.402156591415405 +1,3,2,1,18,18,2,2,2,32,30,30,18,561,2021,672,806,854,492,7,7,7,7,7,7,53,55,9.749915599822998 +1,3,2,1,18,18,2,2,2,32,30,30,18,577,2201,683,795,896,467,7,7,7,7,7,7,59,62,9.590282201766968 +1,3,2,1,18,18,2,2,2,32,30,30,18,560,2040,677,823,843,483,7,7,7,7,7,7,53,54,9.786418914794922 +1,3,2,1,18,18,2,2,2,32,30,30,18,666,2423,779,909,1013,537,7,7,7,7,7,7,66,76,9.420378684997559 +1,3,2,1,18,18,2,2,2,32,30,30,18,817,2989,952,1143,1193,649,7,7,7,7,7,7,73,90,10.082459688186646 +1,3,2,1,18,18,2,2,2,32,30,30,18,482,1861,607,735,768,421,7,7,7,7,7,7,49,49,9.781207084655762 +1,3,2,1,18,18,2,2,2,32,30,30,18,537,1955,649,775,833,477,7,7,7,7,7,7,52,52,9.55460238456726 +1,3,2,1,18,18,2,2,2,32,30,30,18,655,2360,784,946,983,558,7,7,7,7,7,7,60,72,9.846718549728394 +1,3,2,1,18,18,2,2,2,32,30,30,18,504,1957,634,746,837,445,7,7,7,7,7,7,53,54,10.194448709487915 +1,3,2,1,18,18,2,2,2,32,30,30,18,569,2081,688,849,868,502,7,7,7,7,7,7,55,63,9.69931674003601 +1,3,2,1,18,18,2,2,2,32,30,30,18,511,1943,634,735,831,428,7,7,7,7,7,7,51,53,9.344871759414673 +1,3,2,1,18,18,2,2,2,32,30,30,18,702,2590,829,947,1097,576,7,7,7,7,7,7,71,79,9.390014171600342 +1,3,2,1,18,18,2,2,2,32,30,30,18,699,2558,823,978,1058,581,7,7,7,7,7,7,66,74,9.626146793365479 +1,3,2,1,18,18,2,2,2,32,30,30,18,460,1751,574,690,751,425,7,7,7,7,7,7,52,52,9.431775569915771 +1,3,2,1,18,18,2,2,2,32,30,30,18,520,1940,645,766,818,466,7,7,7,7,7,7,52,52,9.625763654708862 +1,3,2,1,18,18,2,2,2,32,30,30,18,541,1963,651,777,823,479,7,7,7,7,7,7,52,52,9.568470478057861 +1,3,2,1,18,18,2,2,2,32,30,30,18,773,2841,890,1045,1165,619,7,7,7,7,7,7,71,88,9.576039552688599 +1,3,2,1,18,18,2,2,2,32,30,30,18,592,2160,712,826,903,514,7,7,7,7,7,7,56,59,9.418208360671997 +1,3,2,1,18,18,2,2,2,32,30,30,18,494,1963,643,777,810,452,7,7,7,7,7,7,47,49,9.665214776992798 +1,3,2,1,18,18,2,2,2,32,30,30,18,695,2561,845,1008,1069,599,7,7,7,7,7,7,58,77,9.849989652633667 +1,3,2,1,18,18,2,2,2,32,30,30,18,514,1870,618,771,773,452,7,7,7,7,7,7,53,53,9.291632175445557 +1,3,2,1,18,18,2,2,2,32,30,30,18,581,2108,702,821,891,498,7,7,7,7,7,7,55,56,9.579374313354492 +1,3,2,1,18,18,2,2,2,32,30,30,18,699,2622,858,1012,1077,601,7,7,7,7,7,7,62,69,9.533780574798584 diff --git a/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..596ff85 Binary files /dev/null and b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..22a522b Binary files /dev/null and b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..fdd9949 --- /dev/null +++ b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!310,qi-prog.getter_over_tuple2,qi-quant-u-4,qi-quant-u-5,qi-quant-u-6,qi-quant-u-7,qi-quant-u-12,qi-quant-u-2,qi-quant-u-3,qi-quant-u-10,qi-quant-u-0,qi-quant-u-1,qi-quant-u-8,execution_time +1,3,2,1,2,2,2,7,7,7,7,7,7,8.578405380249023 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.974594593048096 +1,3,2,1,2,2,2,7,7,7,7,7,7,9.011709213256836 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.703482866287231 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.811612367630005 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.747565746307373 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.868013143539429 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.772895812988281 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.765146732330322 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.806260108947754 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.878298997879028 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.752681970596313 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.808711290359497 +1,3,2,1,2,2,2,7,7,7,7,7,7,9.000406503677368 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.93709111213684 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.710549116134644 +1,3,2,1,2,2,2,7,7,7,7,7,7,9.347690105438232 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.647255897521973 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.930657386779785 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.703575849533081 +1,3,2,1,2,2,2,7,7,7,7,7,7,9.163296699523926 +1,3,2,1,2,2,2,7,7,7,7,7,7,9.102988958358765 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.71028995513916 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.945674180984497 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.843187808990479 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.91843318939209 +1,3,2,1,2,2,2,7,7,7,7,7,7,9.378389120101929 +1,3,2,1,2,2,2,7,7,7,7,7,7,9.12360143661499 +1,3,2,1,2,2,2,7,7,7,7,7,7,8.929125308990479 +1,3,2,1,2,2,2,7,7,7,7,7,7,9.05980658531189 diff --git a/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..1f97adb Binary files /dev/null and b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..fd38912 Binary files /dev/null and b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted.gobra b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted.gobra new file mode 100644 index 0000000..c00ae2b --- /dev/null +++ b/evaluation/experiments/synthetic_set/fully_assisted/fully_assisted.gobra @@ -0,0 +1,90 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// Contains a function to synthetically push the quantifier instantiations +// to the hundreds. We prove the assertions "manually" using the standard +// library. +package fully_assisted + +// We copy an abstract version of the required lemmas in the standard library +// to here. Otherwise, Gobra would try to verify these lemmas again, which +// would not work whenever we disable set axiomatization. + +type Unit struct{} + +ghost +ensures len(xs setminus ys) == len(xs) - len(xs intersection ys) +ensures len(xs setminus ys) + len(ys setminus xs) + len(xs intersection ys) == len(xs union ys) +decreases +pure func DifferenceLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) == len(xs) + len(ys) - len(xs intersection ys) +decreases +pure func UnionLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) <= len(xs) + len(ys) +decreases +pure func UnionLenUpper(xs, ys set[int]) Unit + +ghost +decreases +func synthetic(a, b, c, d, e, f set[int]) { + // Sprinkle in unions and intersections; maybe will lead to more + // instantiations of quantifiers like in_intersection_in_both or + // in_left_in_union + u0 := a union b + i0 := a intersection b + u1 := c union d + i1 := c intersection d + u2 := e union f + i2 := e intersection f + + // Assert what we would learn from DifferenceLenEq, UnionLenUpper + // and UnionLenEq. + // It appears that both of the following sections are needed to generate + // a lot of quantifier instantiations with some variance. + + // DifferenceLenEq + i3 := u0 intersection i0 + m0 := u0 setminus i0 + + i4 := m0 intersection i1 + m1 := m0 setminus i1 + DifferenceLenEq(m0, i1) + assert len(m1) == len(m0) - len(i4) + + i5 := m1 intersection u2 + m2 := m1 setminus u2 + DifferenceLenEq(m1, u2) + assert len(m2) == len(m1) - len(i5) + + // UnionLenEq, UnionLenUpper + UnionLenEq(a, b) + UnionLenUpper(a, b) + if len(i0) == 0 { + assert len(u0) == len(a) + len(b) + } else { + assert len(u0) < len(a) + len(b) + } + + UnionLenEq(c, d) + UnionLenUpper(c, d) + if len(i1) == 0 { + assert len(u1) == len(c) + len(d) + } else { + assert len(u1) < len(c) + len(d) + } + + UnionLenEq(e, f) + UnionLenUpper(e, f) + if len(i2) == 0 { + assert len(u2) == len(e) + len(f) + } else { + assert len(u2) < len(e) + len(f) + } +} diff --git a/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..c585517 --- /dev/null +++ b/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!456,qi-prog.getter_over_tuple2,qi-quant-u-0,qi-quant-u-1,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.in_union_in_one,qi-quant-u-4,qi-quant-u-5,qi-quant-u-10,qi-quant-u-2,qi-quant-u-3,qi-quant-u-8,qi-k!369,qi-$Set[Int]_prog.in_empty_set,execution_time +1,3,2,1,18,18,27,60,60,38,2055,646,760,835,533,394,7,7,7,7,7,7,120,123,9.745379209518433 +1,3,2,1,18,18,27,60,60,38,1924,604,757,811,506,411,7,7,7,7,7,7,104,109,9.577247142791748 +1,3,2,1,18,18,27,60,60,38,1938,620,763,816,507,440,7,7,7,7,7,7,104,104,9.393931865692139 +1,3,2,1,18,18,27,60,60,38,1628,520,616,682,396,327,7,7,7,7,7,7,96,98,9.393034934997559 +1,3,2,1,18,18,27,60,60,38,1900,607,740,786,487,393,7,7,7,7,7,7,101,102,9.546810626983643 +1,3,2,1,18,18,27,60,60,38,2361,752,857,979,618,492,7,7,7,7,7,7,126,131,9.7517569065094 +1,3,2,1,18,18,27,60,60,38,2058,668,818,847,537,437,7,7,7,7,7,7,111,112,9.555549621582031 +1,3,2,1,18,18,27,60,60,38,1805,592,688,770,445,380,7,7,7,7,7,7,106,107,9.577258110046387 +1,3,2,1,18,18,27,60,60,38,1719,534,643,722,441,352,7,7,7,7,7,7,101,101,10.029932737350464 +1,3,2,1,18,18,27,60,60,38,1519,478,583,615,366,314,7,7,7,7,7,7,91,90,9.579591035842896 +1,3,2,1,18,18,27,60,60,38,1932,606,734,788,488,394,7,7,7,7,7,7,104,106,9.680372476577759 +1,3,2,1,18,18,27,60,60,38,1646,514,622,674,406,338,7,7,7,7,7,7,100,101,9.42332148551941 +1,3,2,1,18,18,27,60,60,38,2138,672,769,882,552,431,7,7,7,7,7,7,113,117,9.66365909576416 +1,3,2,1,18,18,27,60,60,38,1542,483,584,634,368,314,7,7,7,7,7,7,92,91,9.450870752334595 +1,3,2,1,18,18,27,60,60,38,2094,677,820,862,554,454,7,7,7,7,7,7,117,119,9.71985673904419 +1,3,2,1,18,18,27,60,60,38,1891,595,744,764,494,393,7,7,7,7,7,7,103,104,9.637270450592041 +1,3,2,1,18,18,27,60,60,38,2047,647,749,852,527,414,7,7,7,7,7,7,107,110,9.450352668762207 +1,3,2,1,18,18,27,60,60,40,1501,460,565,611,344,312,7,7,7,7,7,7,83,83,9.543634414672852 +1,3,2,1,18,18,27,60,60,38,2519,826,985,1038,670,542,7,7,7,7,7,7,124,133,9.473111867904663 +1,3,2,1,18,18,27,60,60,38,1901,604,717,798,480,393,7,7,7,7,7,7,103,106,9.527998447418213 +1,3,2,1,18,18,27,60,60,38,1722,538,662,710,435,353,7,7,7,7,7,7,103,102,9.843971490859985 +1,3,2,1,18,18,27,60,60,38,1860,576,719,762,480,386,7,7,7,7,7,7,102,102,9.417147874832153 +1,3,2,1,18,18,27,60,60,38,2272,736,877,921,596,469,7,7,7,7,7,7,124,128,9.46385931968689 +1,3,2,1,18,18,27,60,60,38,2011,629,731,838,514,402,7,7,7,7,7,7,114,118,9.70307445526123 +1,3,2,1,18,18,27,60,60,38,1710,540,655,706,434,354,7,7,7,7,7,7,101,101,9.5003342628479 +1,3,2,1,18,18,27,60,60,38,2181,671,792,904,567,434,7,7,7,7,7,7,119,126,9.46611213684082 +1,3,2,1,18,18,27,60,60,38,1878,587,695,788,474,380,7,7,7,7,7,7,100,102,9.690132141113281 +1,3,2,1,18,18,27,60,60,38,1747,542,667,727,441,355,7,7,7,7,7,7,94,96,9.868340253829956 +1,3,2,1,18,18,27,60,60,38,1741,549,673,725,453,358,7,7,7,7,7,7,99,100,9.480613946914673 +1,3,2,1,18,18,27,60,60,40,2057,654,779,847,530,434,7,7,7,7,7,7,117,122,9.589053630828857 diff --git a/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..e1b6045 Binary files /dev/null and b/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..13d878f Binary files /dev/null and b/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq.gobra b/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq.gobra new file mode 100644 index 0000000..cb76fbf --- /dev/null +++ b/evaluation/experiments/synthetic_set/no_diffleneq/no_diffleneq.gobra @@ -0,0 +1,90 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// Contains a function to synthetically push the quantifier instantiations +// to the hundreds. We prove the assertions "manually" using the standard +// library. We do not use DifferenceLenEq. +package no_diffleneq + +// We copy an abstract version of the required lemmas in the standard library +// to here. Otherwise, Gobra would try to verify these lemmas again, which +// would not work whenever we disable set axiomatization. + +type Unit struct{} + +ghost +ensures len(xs setminus ys) == len(xs) - len(xs intersection ys) +//ensures len(xs setminus ys) + len(ys setminus xs) + len(xs intersection ys) == len(xs union ys) +decreases +pure func DifferenceLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) == len(xs) + len(ys) - len(xs intersection ys) +decreases +pure func UnionLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) <= len(xs) + len(ys) +decreases +pure func UnionLenUpper(xs, ys set[int]) Unit + +ghost +decreases +func synthetic(a, b, c, d, e, f set[int]) { + // Sprinkle in unions and intersections; maybe will lead to more + // instantiations of quantifiers like in_intersection_in_both or + // in_left_in_union + u0 := a union b + i0 := a intersection b + u1 := c union d + i1 := c intersection d + u2 := e union f + i2 := e intersection f + + // Assert what we would learn from DifferenceLenEq, UnionLenUpper + // and UnionLenEq. + // It appears that both of the following sections are needed to generate + // a lot of quantifier instantiations with some variance. + + // DifferenceLenEq + i3 := u0 intersection i0 + m0 := u0 setminus i0 + + i4 := m0 intersection i1 + m1 := m0 setminus i1 + //DifferenceLenEq(m0, i1) + assert len(m1) == len(m0) - len(i4) + + i5 := m1 intersection u2 + m2 := m1 setminus u2 + //DifferenceLenEq(m1, u2) + assert len(m2) == len(m1) - len(i5) + + // UnionLenEq, UnionLenUpper + UnionLenEq(a, b) + UnionLenUpper(a, b) + if len(i0) == 0 { + assert len(u0) == len(a) + len(b) + } else { + assert len(u0) < len(a) + len(b) + } + + UnionLenEq(c, d) + UnionLenUpper(c, d) + if len(i1) == 0 { + assert len(u1) == len(c) + len(d) + } else { + assert len(u1) < len(c) + len(d) + } + + UnionLenEq(e, f) + UnionLenUpper(e, f) + if len(i2) == 0 { + assert len(u2) == len(e) + len(f) + } else { + assert len(u2) < len(e) + len(f) + } +} diff --git a/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..91a9573 --- /dev/null +++ b/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!456,qi-prog.getter_over_tuple2,qi-quant-u-2,qi-quant-u-3,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.in_union_in_one,qi-quant-u-4,qi-quant-u-5,qi-quant-u-10,qi-k!369,qi-$Set[Int]_prog.in_empty_set,execution_time +1,3,2,1,18,18,27,60,60,38,1894,596,740,791,498,405,7,7,7,104,108,9.44849443435669 +1,3,2,1,18,18,27,60,60,38,1659,513,630,685,421,343,7,7,7,96,96,9.497825860977173 +1,3,2,1,18,18,27,60,60,38,1725,539,654,706,429,362,7,7,7,93,93,9.6283700466156 +1,3,2,1,18,18,27,60,60,38,1778,562,693,737,466,379,7,7,7,103,104,9.390675067901611 +1,3,2,1,18,18,27,60,60,38,1751,553,687,708,444,392,7,7,7,107,107,9.543347358703613 +1,3,2,1,18,18,27,60,60,38,2455,795,942,1019,659,529,7,7,7,129,138,10.182425737380981 +1,3,2,1,18,18,27,60,60,38,2292,747,886,960,602,504,7,7,7,117,123,9.363703489303589 +1,3,2,1,18,18,27,60,60,38,1806,581,686,764,445,378,7,7,7,101,104,9.515135288238525 +1,3,2,1,18,18,27,60,60,38,2136,670,788,867,550,432,7,7,7,112,119,9.475162982940674 +1,3,2,1,18,18,27,60,60,38,1923,611,737,799,489,397,7,7,7,107,110,9.707395792007446 +1,3,2,1,18,18,27,60,60,38,1989,614,727,804,513,400,7,7,7,112,115,9.766398429870605 +1,3,2,1,18,18,27,60,60,38,2288,743,899,942,604,488,7,7,7,123,126,9.583626747131348 +1,3,2,1,18,18,27,60,60,38,1754,539,646,721,446,357,7,7,7,103,103,9.67577600479126 +1,3,2,1,18,18,27,60,60,38,2290,734,867,968,603,491,7,7,7,119,128,9.901206970214844 +1,3,2,1,18,18,27,60,60,38,1974,629,763,823,510,419,7,7,7,102,107,9.846839189529419 +1,3,2,1,18,18,27,60,60,38,2117,683,842,884,569,463,7,7,7,112,117,9.56575894355774 +1,3,2,1,18,18,27,60,60,38,1995,626,746,816,523,410,7,7,7,109,112,9.768954753875732 +1,3,2,1,18,18,27,60,60,38,2023,620,738,818,517,402,7,7,7,111,114,9.578772068023682 +1,3,2,1,18,18,27,60,60,38,2003,640,784,838,529,438,7,7,7,108,108,9.767804145812988 +1,3,2,1,18,18,27,60,60,38,1666,538,668,676,413,355,7,7,7,97,96,9.344282627105713 +1,3,2,1,18,18,27,60,60,38,1738,560,684,730,433,371,7,7,7,102,103,9.774399280548096 +1,3,2,1,18,18,27,60,60,38,2117,677,802,887,560,449,7,7,7,116,119,9.324067115783691 +1,3,2,1,18,18,27,60,60,38,1961,606,741,780,500,406,7,7,7,106,108,9.868307828903198 +1,3,2,1,18,18,27,60,60,38,2155,675,795,862,557,430,7,7,7,119,123,9.446878433227539 +1,3,2,1,18,18,27,60,60,38,1615,511,624,678,397,337,7,7,7,92,92,9.682556867599487 +1,3,2,1,18,18,27,60,60,38,2181,692,824,902,581,463,7,7,7,120,124,9.522209644317627 +1,3,2,1,18,18,27,60,60,38,2301,737,857,941,611,482,7,7,7,125,128,10.159232378005981 +1,3,2,1,18,18,27,60,60,38,1732,536,676,694,437,360,7,7,7,96,96,9.960811614990234 +1,3,2,1,18,18,27,60,60,38,2167,702,852,911,580,479,7,7,7,116,119,9.571574926376343 +1,3,2,1,18,18,27,60,60,38,2345,760,893,997,618,514,7,7,7,121,125,9.665184497833252 diff --git a/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..36490b4 Binary files /dev/null and b/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..c0a67bf Binary files /dev/null and b/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper.gobra b/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper.gobra new file mode 100644 index 0000000..31daaea --- /dev/null +++ b/evaluation/experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper.gobra @@ -0,0 +1,90 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// Contains a function to synthetically push the quantifier instantiations +// to the hundreds. We prove the assertions "manually" using the standard +// library. We do not use DifferenceLenEq nor UnionLenUpper. +package no_diffleneq_unionlenupper + +// We copy an abstract version of the required lemmas in the standard library +// to here. Otherwise, Gobra would try to verify these lemmas again, which +// would not work whenever we disable set axiomatization. + +type Unit struct{} + +ghost +ensures len(xs setminus ys) == len(xs) - len(xs intersection ys) +//ensures len(xs setminus ys) + len(ys setminus xs) + len(xs intersection ys) == len(xs union ys) +decreases +pure func DifferenceLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) == len(xs) + len(ys) - len(xs intersection ys) +decreases +pure func UnionLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) <= len(xs) + len(ys) +decreases +pure func UnionLenUpper(xs, ys set[int]) Unit + +ghost +decreases +func synthetic(a, b, c, d, e, f set[int]) { + // Sprinkle in unions and intersections; maybe will lead to more + // instantiations of quantifiers like in_intersection_in_both or + // in_left_in_union + u0 := a union b + i0 := a intersection b + u1 := c union d + i1 := c intersection d + u2 := e union f + i2 := e intersection f + + // Assert what we would learn from DifferenceLenEq, UnionLenUpper + // and UnionLenEq. + // It appears that both of the following sections are needed to generate + // a lot of quantifier instantiations with some variance. + + // DifferenceLenEq + i3 := u0 intersection i0 + m0 := u0 setminus i0 + + i4 := m0 intersection i1 + m1 := m0 setminus i1 + //DifferenceLenEq(m0, i1) + assert len(m1) == len(m0) - len(i4) + + i5 := m1 intersection u2 + m2 := m1 setminus u2 + //DifferenceLenEq(m1, u2) + assert len(m2) == len(m1) - len(i5) + + // UnionLenEq, UnionLenUpper + UnionLenEq(a, b) + //UnionLenUpper(a, b) + if len(i0) == 0 { + assert len(u0) == len(a) + len(b) + } else { + assert len(u0) < len(a) + len(b) + } + + UnionLenEq(c, d) + //UnionLenUpper(c, d) + if len(i1) == 0 { + assert len(u1) == len(c) + len(d) + } else { + assert len(u1) < len(c) + len(d) + } + + UnionLenEq(e, f) + //UnionLenUpper(e, f) + if len(i2) == 0 { + assert len(u2) == len(e) + len(f) + } else { + assert len(u2) < len(e) + len(f) + } +} diff --git a/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..98b10ba --- /dev/null +++ b/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!456,qi-prog.getter_over_tuple2,qi-quant-u-4,qi-quant-u-5,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.in_union_in_one,qi-k!369,qi-$Set[Int]_prog.in_empty_set,execution_time +1,3,2,1,18,18,27,60,60,38,1701,519,636,694,431,343,99,100,9.68796157836914 +1,3,2,1,18,18,27,60,60,38,1750,551,682,720,449,372,96,97,9.373748779296875 +1,3,2,1,18,18,27,60,60,40,1699,528,641,679,400,359,89,88,9.41971492767334 +1,3,2,1,18,18,27,60,60,38,1699,525,645,698,434,355,96,96,9.459847688674927 +1,3,2,1,18,18,27,60,60,38,2192,707,856,919,579,474,113,120,9.4464693069458 +1,3,2,1,18,18,27,60,60,38,1843,572,698,742,468,364,98,99,9.584553718566895 +1,3,2,1,18,18,27,60,60,38,2199,709,869,926,590,490,115,122,9.615959644317627 +1,3,2,1,18,18,27,60,60,38,2470,795,931,1049,653,526,124,135,9.580095767974854 +1,3,2,1,18,18,27,60,60,38,1533,478,575,628,368,306,96,96,9.429539442062378 +1,3,2,1,18,18,27,60,60,38,1769,551,658,747,449,368,96,96,9.368203163146973 +1,3,2,1,18,18,27,60,60,38,1760,547,660,739,448,371,96,95,9.405307292938232 +1,3,2,1,18,18,27,60,60,38,2065,658,785,845,536,432,113,114,9.399497509002686 +1,3,2,1,18,18,27,60,60,38,1878,577,702,771,476,377,105,105,9.306172609329224 +1,3,2,1,18,18,27,60,60,38,1712,536,640,691,405,331,88,88,9.469423055648804 +1,3,2,1,18,18,27,60,60,38,1723,530,663,707,439,365,96,96,9.487361907958984 +1,3,2,1,18,18,27,60,60,38,1932,609,729,789,500,392,112,114,9.291025400161743 +1,3,2,1,18,18,27,60,60,38,1636,503,625,668,411,331,100,99,9.506308317184448 +1,3,2,1,18,18,27,60,60,38,1685,515,643,689,429,353,95,95,9.400606870651245 +1,3,2,1,18,18,27,60,60,38,1830,571,689,752,475,379,99,99,9.519744157791138 +1,3,2,1,18,18,27,60,60,38,2252,730,861,937,590,464,121,125,9.407500743865967 +1,3,2,1,18,18,27,60,60,38,2370,759,876,995,622,493,115,124,9.452322959899902 +1,3,2,1,18,18,27,60,60,38,1840,574,703,757,475,388,102,103,9.544098854064941 +1,3,2,1,18,18,27,60,60,38,1662,513,649,677,427,344,95,95,9.442953109741211 +1,3,2,1,18,18,27,60,60,40,2084,652,786,853,536,444,115,120,9.203182697296143 +1,3,2,1,18,18,27,60,60,38,1533,486,588,627,358,304,86,85,9.302816390991211 +1,3,2,1,18,18,27,60,60,40,1731,533,651,706,439,364,101,101,9.773189306259155 +1,3,2,1,18,18,27,60,60,38,1728,533,660,697,443,354,96,96,9.380340099334717 +1,3,2,1,18,18,27,60,60,40,1747,541,656,711,425,359,95,96,9.258466005325317 +1,3,2,1,18,18,27,60,60,38,1728,531,651,713,436,358,96,96,9.294753074645996 +1,3,2,1,18,18,27,60,60,38,1998,635,751,809,519,413,112,114,9.466650485992432 diff --git a/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..7a1b86d Binary files /dev/null and b/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..f261688 Binary files /dev/null and b/evaluation/experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/synthetic_set/not_assisted/not_assisted.gobra b/evaluation/experiments/synthetic_set/not_assisted/not_assisted.gobra new file mode 100644 index 0000000..25f8543 --- /dev/null +++ b/evaluation/experiments/synthetic_set/not_assisted/not_assisted.gobra @@ -0,0 +1,89 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// Contains a function to synthetically push the quantifier instantiations +// to the hundreds. We do not assist the verifier. +package not_assisted + +// We copy an abstract version of the required lemmas in the standard library +// to here. Otherwise, Gobra would try to verify these lemmas again, which +// would not work whenever we disable set axiomatization. + +type Unit struct{} + +ghost +ensures len(xs setminus ys) == len(xs) - len(xs intersection ys) +//ensures len(xs setminus ys) + len(ys setminus xs) + len(xs intersection ys) == len(xs union ys) +decreases +pure func DifferenceLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) == len(xs) + len(ys) - len(xs intersection ys) +decreases +pure func UnionLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) <= len(xs) + len(ys) +decreases +pure func UnionLenUpper(xs, ys set[int]) Unit + +ghost +decreases +func synthetic(a, b, c, d, e, f set[int]) { + // Sprinkle in unions and intersections; maybe will lead to more + // instantiations of quantifiers like in_intersection_in_both or + // in_left_in_union + u0 := a union b + i0 := a intersection b + u1 := c union d + i1 := c intersection d + u2 := e union f + i2 := e intersection f + + // Assert what we would learn from DifferenceLenEq, UnionLenUpper + // and UnionLenEq. + // It appears that both of the following sections are needed to generate + // a lot of quantifier instantiations with some variance. + + // DifferenceLenEq + i3 := u0 intersection i0 + m0 := u0 setminus i0 + + i4 := m0 intersection i1 + m1 := m0 setminus i1 + //DifferenceLenEq(m0, i1) + assert len(m1) == len(m0) - len(i4) + + i5 := m1 intersection u2 + m2 := m1 setminus u2 + //DifferenceLenEq(m1, u2) + assert len(m2) == len(m1) - len(i5) + + // UnionLenEq, UnionLenUpper + //UnionLenEq(a, b) + //UnionLenUpper(a, b) + if len(i0) == 0 { + assert len(u0) == len(a) + len(b) + } else { + assert len(u0) < len(a) + len(b) + } + + //UnionLenEq(c, d) + //UnionLenUpper(c, d) + if len(i1) == 0 { + assert len(u1) == len(c) + len(d) + } else { + assert len(u1) < len(c) + len(d) + } + + //UnionLenEq(e, f) + //UnionLenUpper(e, f) + if len(i2) == 0 { + assert len(u2) == len(e) + len(f) + } else { + assert len(u2) < len(e) + len(f) + } +} diff --git a/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv b/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv new file mode 100644 index 0000000..f89a719 --- /dev/null +++ b/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv @@ -0,0 +1,31 @@ +qi-k!456,qi-prog.getter_over_tuple2,qi-quant-u-2,qi-quant-u-3,qi-$Set[Int]_prog.equality_definition,qi-$Set[Int]_prog.native_equality,qi-quant-u-4,qi-quant-u-5,qi-quant-u-10,qi-$Set[Int]_prog.cardinality_difference,qi-$Set[Int]_prog.card_non_negative,qi-$Set[Int]_prog.empty_set_cardinality,qi-$Set[Int]_prog.cardinality_sums,qi-$Set[Int]_prog.not_in_difference,qi-$Set[Int]_prog.in_intersection_in_both,qi-$Set[Int]_prog.in_right_in_union,qi-$Set[Int]_prog.in_left_in_union,qi-$Set[Int]_prog.in_difference,qi-$Set[Int]_prog.in_union_in_one,qi-quant-u-6,qi-quant-u-7,qi-quant-u-12,qi-quant-u-0,qi-quant-u-1,qi-quant-u-8,qi-k!369,qi-$Set[Int]_prog.in_empty_set,execution_time +1,3,2,1,18,18,2,2,2,32,30,30,18,584,1869,689,774,447,389,7,7,7,7,7,7,86,88,9.549062967300415 +1,3,2,1,18,18,2,2,2,32,30,30,18,596,1821,688,742,429,388,7,7,7,7,7,7,86,86,9.43303656578064 +1,3,2,1,18,18,2,2,2,32,30,30,18,695,2077,799,870,512,454,7,7,7,7,7,7,99,101,9.796035766601562 +1,3,2,1,18,18,2,2,2,32,30,30,18,710,2090,833,884,519,508,7,7,7,7,7,7,92,95,9.568249225616455 +1,3,2,1,18,18,2,2,2,32,30,30,18,567,1729,657,696,383,405,7,7,7,7,7,7,86,85,9.461755275726318 +1,3,2,1,18,18,2,2,2,32,30,30,18,671,2063,765,865,491,437,7,7,7,7,7,7,95,97,9.650207757949829 +1,3,2,1,18,18,2,2,2,32,30,30,18,586,1790,670,715,402,357,7,7,7,7,7,7,92,92,9.550570249557495 +1,3,2,1,18,18,2,2,2,32,30,30,18,604,1911,690,770,446,389,7,7,7,7,7,7,91,94,9.594697952270508 +1,3,2,1,18,18,2,2,2,32,30,30,18,566,1738,666,706,398,352,7,7,7,7,7,7,83,83,9.456947803497314 +1,3,2,1,18,18,2,2,2,32,30,30,18,593,1816,693,740,430,377,7,7,7,7,7,7,87,86,9.671915769577026 +1,3,2,1,18,18,2,2,2,32,30,30,18,689,2118,839,877,525,498,7,7,7,7,7,7,91,95,9.92549443244934 +1,3,2,1,18,18,2,2,2,32,30,30,18,794,2414,919,996,615,523,7,7,7,7,7,7,113,119,9.447190999984741 +1,3,2,1,18,18,2,2,2,32,30,30,18,774,2368,891,988,583,508,7,7,7,7,7,7,99,102,9.568503379821777 +1,3,2,1,18,18,2,2,2,32,30,30,18,575,1830,658,744,422,363,7,7,7,7,7,7,85,86,9.47563648223877 +1,3,2,1,18,18,2,2,2,32,30,30,18,729,2236,865,938,569,524,7,7,7,7,7,7,105,109,9.376817464828491 +1,3,2,1,18,18,2,2,2,32,30,30,18,688,2065,794,863,501,450,7,7,7,7,7,7,90,91,9.535146713256836 +1,3,2,1,18,18,2,2,2,32,30,30,18,629,1967,755,796,485,421,7,7,7,7,7,7,90,90,9.421860933303833 +1,3,2,1,18,18,2,2,2,32,30,30,18,582,1778,679,715,406,376,7,7,7,7,7,7,86,86,9.442054510116577 +1,3,2,1,18,18,2,2,2,32,30,30,18,502,1609,593,644,346,318,7,7,7,7,7,7,82,82,9.420251607894897 +1,3,2,1,18,18,2,2,2,32,30,30,18,497,1598,584,640,338,318,7,7,7,7,7,7,75,75,9.440504312515259 +1,3,2,1,18,18,2,2,2,32,30,30,18,564,1775,674,713,420,421,7,7,7,7,7,7,87,87,9.321737051010132 +1,3,2,1,18,18,2,2,2,32,30,30,18,577,1840,693,735,441,381,7,7,7,7,7,7,88,88,9.716878652572632 +1,3,2,1,18,18,2,2,2,32,30,30,18,756,2389,871,973,592,488,7,7,7,7,7,7,111,116,9.938218832015991 +1,3,2,1,18,18,2,2,2,32,30,30,18,599,1815,676,745,416,382,7,7,7,7,7,7,84,84,9.494309425354004 +1,3,2,1,18,18,2,2,2,32,30,30,18,770,2309,889,989,582,518,7,7,7,7,7,7,96,99,9.45365309715271 +1,3,2,1,18,18,2,2,2,32,30,30,18,631,1900,736,790,458,412,7,7,7,7,7,7,89,90,9.74614930152893 +1,3,2,1,18,18,2,2,2,32,30,30,18,750,2310,894,934,588,486,7,7,7,7,7,7,111,115,9.48568344116211 +1,3,2,1,18,18,2,2,2,32,30,30,18,690,2081,822,869,508,474,7,7,7,7,7,7,94,97,9.701004028320312 +1,3,2,1,18,18,2,2,2,32,30,30,18,826,2526,930,1065,641,539,7,7,7,7,7,7,112,119,9.550092220306396 +1,3,2,1,18,18,2,2,2,32,30,30,18,713,2134,840,873,520,458,7,7,7,7,7,7,96,100,9.671375274658203 diff --git a/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png b/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png new file mode 100644 index 0000000..ad0315d Binary files /dev/null and b/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.execution_time.png differ diff --git a/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png b/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png new file mode 100644 index 0000000..a72b7a0 Binary files /dev/null and b/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.qi.png differ diff --git a/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq.gobra b/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq.gobra new file mode 100644 index 0000000..9b85023 --- /dev/null +++ b/evaluation/experiments/synthetic_set/weak_diffleneq/weak_diffleneq.gobra @@ -0,0 +1,91 @@ +/* + This file is part of gobra-libs which is released under the MIT license. + See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE + for full license details. +*/ + +// Contains a function to synthetically push the quantifier instantiations +// to the hundreds. We prove the assertions "manually" using the standard +// library. Additionally, we weaken DifferenceLenEq to only include +// the relevant postcondition. +package weak_diffleneq + +// We copy an abstract version of the required lemmas in the standard library +// to here. Otherwise, Gobra would try to verify these lemmas again, which +// would not work whenever we disable set axiomatization. + +type Unit struct{} + +ghost +ensures len(xs setminus ys) == len(xs) - len(xs intersection ys) +//ensures len(xs setminus ys) + len(ys setminus xs) + len(xs intersection ys) == len(xs union ys) +decreases +pure func DifferenceLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) == len(xs) + len(ys) - len(xs intersection ys) +decreases +pure func UnionLenEq(xs, ys set[int]) Unit + +ghost +ensures len(xs union ys) <= len(xs) + len(ys) +decreases +pure func UnionLenUpper(xs, ys set[int]) Unit + +ghost +decreases +func synthetic(a, b, c, d, e, f set[int]) { + // Sprinkle in unions and intersections; maybe will lead to more + // instantiations of quantifiers like in_intersection_in_both or + // in_left_in_union + u0 := a union b + i0 := a intersection b + u1 := c union d + i1 := c intersection d + u2 := e union f + i2 := e intersection f + + // Assert what we would learn from DifferenceLenEq, UnionLenUpper + // and UnionLenEq. + // It appears that both of the following sections are needed to generate + // a lot of quantifier instantiations with some variance. + + // DifferenceLenEq + i3 := u0 intersection i0 + m0 := u0 setminus i0 + + i4 := m0 intersection i1 + m1 := m0 setminus i1 + DifferenceLenEq(m0, i1) + assert len(m1) == len(m0) - len(i4) + + i5 := m1 intersection u2 + m2 := m1 setminus u2 + DifferenceLenEq(m1, u2) + assert len(m2) == len(m1) - len(i5) + + // UnionLenEq, UnionLenUpper + UnionLenEq(a, b) + UnionLenUpper(a, b) + if len(i0) == 0 { + assert len(u0) == len(a) + len(b) + } else { + assert len(u0) < len(a) + len(b) + } + + UnionLenEq(c, d) + UnionLenUpper(c, d) + if len(i1) == 0 { + assert len(u1) == len(c) + len(d) + } else { + assert len(u1) < len(c) + len(d) + } + + UnionLenEq(e, f) + UnionLenUpper(e, f) + if len(i2) == 0 { + assert len(u2) == len(e) + len(f) + } else { + assert len(u2) < len(e) + len(f) + } +} diff --git a/evaluation/scripts/noaxioms_sets.vpr b/evaluation/scripts/noaxioms_sets.vpr new file mode 100644 index 0000000..f29cf71 --- /dev/null +++ b/evaluation/scripts/noaxioms_sets.vpr @@ -0,0 +1,19 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at http://mozilla.org/MPL/2.0/. +// +// Copyright (c) 2011-2023 ETH Zurich. + +domain $Set[E] { + function Set_in(e: E, s: $Set[E]): Bool + function Set_card(s: $Set[E]): Int + function Set_empty(): $Set[E] + function Set_singleton(e: E): $Set[E] + function Set_unionone(s: $Set[E], e: E): $Set[E] + function Set_union(s1: $Set[E], s2: $Set[E]): $Set[E] + function Set_disjoint(s1: $Set[E], s2: $Set[E]): Bool + function Set_difference(s1: $Set[E], s2: $Set[E]): $Set[E] + function Set_intersection(s1: $Set[E], s2: $Set[E]): $Set[E] + function Set_subset(s1: $Set[E], s2: $Set[E]): Bool + function Set_equal(s1: $Set[E], s2: $Set[E]): Bool +} diff --git a/evaluation/scripts/plot.py b/evaluation/scripts/plot.py new file mode 100644 index 0000000..b9f5dbf --- /dev/null +++ b/evaluation/scripts/plot.py @@ -0,0 +1,251 @@ +""" +Module for plotting the results of profile.py. + +This file is part of gobra-libs which is released under the MIT license. +See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE +for full license details. +""" + +import argparse +import pandas as pd +import numpy as np +import seaborn as sns +import matplotlib.pyplot as plt + +from util import file_path + + +def parse_args(): + """Parse command line arguments.""" + parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter) + parser.add_argument("csv_path", type=program_path, nargs="+", help="CSV files to be analyzed") + parser.add_argument("--filter_anonymous", action="store_true", required=False, + help="filter out anonymous quantifier instantiations like quant-u-17 and k!512.") + parser.add_argument("--name", type=str, required=False, + help="Prefix for the output files.") + parser.add_argument("--top", type=int, required=False, + help="Plot only the top n quantifiers") + parser.add_argument("--variants", type=str, nargs="+", required=False, + help="Names of the variants in the plot (only works with multiple CSV files).") + parser.add_argument("--qi_size", type=int, nargs=2, required=False, default=[6, 4], + help="Size of the quantifier instantiation plot (width height)") + parser.add_argument("--execution_time_size", type=int, nargs=2, required=False, default=[6, 4], + help="Size of the execution time plot (width height)") + parser.add_argument("--start_at_zero_qi", action="store_true", required=False, + help="Start the axis for the number of quantifier instantiations at zero.") + parser.add_argument("--start_at_zero_execution_time", action="store_true", required=False, + help="Start the axis for execution time at zero.") + + return parser.parse_args() + + +def main(): + """Main function of the plotter.""" + # Parse arguments + args = parse_args() + + if (args.variants is not None) and (len(args.csv_path) != len(args.variants)): + raise argparse.ArgumentTypeError(f"Number of variants must match the number of CSV files") + + if len(args.csv_path) == 1: + plot(args, args.csv_path[0]) + else: + plot_multiple(args, args.csv_path) + + +def plot(args, csv_path): + """Plot a single CSV file.""" + if args.name is None: + args.name = csv_path.with_suffix("") + + # Read CSV file using pandas. + df = pd.read_csv(csv_path) + + # Remove "qi-" from column names that start with it. + df.columns = df.columns.str.replace('qi-', '') + + # Remove columns that start with "quant-u" or "k!". + if args.filter_anonymous: + df = df.filter(regex='^(?!quant-u|k!).*$', axis=1) + + # Split the DataFrame into two DataFrames: one with only the 'execution_time' column and one with all other columns. + execution_time_df = df[['execution_time']] + qi_df = df.drop(columns=['execution_time']) + + # Determine Top Quantifiers if needed + if args.top is not None: + top_quantifiers = qi_df.median(numeric_only=True).nlargest(args.top).index + qi_df = qi_df.loc[:, list(top_quantifiers)] + + qi_df = qi_df.sort_values(by=qi_df.index[0], ascending=False, axis=1) + + sns.set_theme(rc={'figure.figsize': args.qi_size}) + + # Generate plot for quantifier instantiations. + plt.figure() # Create a new figure + + if len(df) == 1: + sns.barplot(qi_df, orient='h') + else: + sns.boxplot(qi_df, orient='h') + + if args.start_at_zero_qi: + plt.xlim(0, None) + + plt.tight_layout() + plt.savefig((str(args.name) + ".qi.pdf"), dpi=600) + plt.close() + + # Generate plot for execution time if we have more than one measurement + sns.set_theme(rc={'figure.figsize': args.execution_time_size}) + if len(df) > 1: + plt.figure() + + # Calculate median, quartiles, and IQR + median = execution_time_df['execution_time'].median() + quartile1 = execution_time_df['execution_time'].quantile(0.25) + quartile3 = execution_time_df['execution_time'].quantile(0.75) + iqr = quartile3 - quartile1 + + # Identify outliers + outliers = execution_time_df[(execution_time_df < (quartile1 - 1.5 * iqr)) | + (execution_time_df > (quartile3 + 1.5 * iqr))] + + # Plot histogram + sns.histplot(execution_time_df) + + # Overlay the median and quartiles using Matplotlib + plt.axvline(median, color='red', linestyle='--', label='Median') + plt.axvline(quartile1, color='green', linestyle='--', label='1st Quartile (Q1)') + plt.axvline(quartile3, color='green', linestyle='--', label='3rd Quartile (Q3)') + + # Plot the outliers as individual points + plt.scatter(outliers['execution_time'], np.zeros(len(outliers)), color='orange', s=30, label='Outliers') + + plt.title('Histogram of Execution Times with Quartiles and Median') + + if args.start_at_zero_execution_time: + plt.xlim(0, None) + + plt.xlabel('Execution Time (seconds)', labelpad=15) + plt.ylabel('Frequency', labelpad=15) + plt.legend() + plt.savefig((str(args.name) + ".execution_time.pdf"), dpi=600) + plt.close() + + +def plot_multiple(args, csv_paths): + """Plot multiple CSV files on the same graph.""" + # Default value for variant name + if args.variants is None: + args.variants = [csv_path.stem.split("-")[0] for csv_path in csv_paths] + + # Load csv files and set the variant name + # Note that zip won't ignore extra elements in the longer list since they have the same length (checked in main) + dfs = [] + for csv_path, variant in zip(csv_paths, args.variants): + df = pd.read_csv(csv_path) + df['Variant'] = variant + dfs.append(df) + + # Remove "qi-" from column names that start with it. + for df in dfs: + df.columns = df.columns.str.replace('qi-', '') + + if args.filter_anonymous: + # Remove columns that start with "quant-u" or "k!". + dfs = [df.filter(regex='^(?!quant-u|k!).*$', axis=1) for df in dfs] + + # Split the DataFrames into two DataFrames: one with only the 'execution_time' column and one with all other + # columns. + execution_time_dfs = [df[['execution_time']] for df in dfs] + qi_dfs = [df.drop(columns=['execution_time']) for df in dfs] + + if args.top is not None: + # Determine Top Quantifiers for Each DataFrame + top_quantifiers_sets = [] + for df in qi_dfs: + top_quantifiers = df.median(numeric_only=True).nlargest(args.top).index + top_quantifiers_sets.append(set(top_quantifiers)) + + # Take the Union of Top Quantifiers + union_top_quantifiers = set.union(*top_quantifiers_sets) + + # Filter Each DataFrame on the Union of Top Quantifiers + qi_dfs_filtered = [] + for df in qi_dfs: + # Keep 'Variant' column and any quantifier in the union of top quantifiers + columns_to_keep = ['Variant'] + list(union_top_quantifiers.intersection(df.columns)) + qi_dfs_filtered.append(df.loc[:, columns_to_keep]) + + # Now qi_dfs_filtered contains the DataFrames filtered to include only the union of top quantifiers + qi_dfs = qi_dfs_filtered + + # Sort each DataFrame in qi_dfs by the median number of instantiations + sorted_qi_dfs = [] + for df in qi_dfs: + # Calculate the median of each column + medians = df.median(numeric_only=True).sort_values(ascending=False) + # Sort the DataFrame based on the calculated medians + sorted_df = df.loc[:, df.columns.intersection(['Variant']).append(medians.index)] + sorted_qi_dfs.append(sorted_df) + + # Now, sorted_qi_dfs contains the sorted DataFrames + qi_dfs = sorted_qi_dfs + + # Combine the DataFrames into a single DataFrame for plotting + qi_cdf = pd.concat(qi_dfs) + qi_mdf = qi_cdf.melt(id_vars="Variant", var_name="quantifier", value_name="instantiations") + + sns.set_theme(rc={'figure.figsize': args.qi_size}) + plt.figure() + + if len(qi_dfs[0]) == 1: + sns.barplot(x='instantiations', y='quantifier', hue='Variant', data=qi_mdf, orient="h") + else: + sns.boxplot(x='instantiations', y='quantifier', hue='Variant', data=qi_mdf, orient="h") + + if args.start_at_zero_qi: + plt.xlim(0, None) + + plt.xlabel('Number of Instantiations', labelpad=15) + plt.ylabel('Quantifier', labelpad=15) + plt.tight_layout() + + if args.name is None: + args.name = "multi" + + plt.savefig(f"{args.name}.qi.pdf", dpi=600) + plt.close() + + sns.set_theme(rc={'figure.figsize': args.execution_time_size}) + + # Combine DataFrames and rename the columns to the variant names + execution_time_cdf = pd.concat(execution_time_dfs, axis=1) + execution_time_cdf.columns = args.variants + + plt.figure() + + sns.boxplot(execution_time_cdf) + + if args.start_at_zero_execution_time: + plt.ylim(0, None) + plt.xlabel('Variant', labelpad=15) + plt.ylabel('Execution time (s)', labelpad=15) + + plt.tight_layout() + plt.savefig(f"{args.name}.execution_time.pdf", dpi=600) + plt.close() + + +def program_path(string): + """Checks that the string is a valid path to a CSV file.""" + path = file_path(string) + if path.suffix != ".csv": + raise argparse.ArgumentTypeError(f"Wrong suffix: {path} is not a valid path to a CSV file") + + return path + + +if __name__ == '__main__': + main() diff --git a/evaluation/scripts/profile-all.sh b/evaluation/scripts/profile-all.sh new file mode 100755 index 0000000..497a238 --- /dev/null +++ b/evaluation/scripts/profile-all.sh @@ -0,0 +1,58 @@ +#!/bin/bash + +# This file is part of gobra-libs which is released under the MIT license. +# See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE +# for full license details. + +# DEPENDENCIES (plot.py): +# - pandas +# - numpy +# - seaborn +# - matplotlib +# In case you forgot to install the dependencies before running the script, you +# do not need to rerun this script (and in particular profiling); instead, simply use +# the command for plotting found below. + +# USAGE: +# First, make sure the following variables are set: +# - SILICON_PATH (should contain path to silicon.sh) +# - Z3_PATH (should contain path to the Z3 binary) +# - GOBRA_PATH (should contain path to Gobra jar) +# Note that we only tested Z3 4.8.7; newer versions may not work as +# they produce errors and different output. +# +# Finally, switch into the same directory as profile-all.sh, and run the script. +# +# In case you are using a laptop, make sure that power settings are set +# properly: +# - don't run from battery +# - make sure the correct profile is set, i.e. not the battery profile +# (e.g., in TLP) +# - make sure the laptop will not suspend +# - check packages that may interfere +# - use systemd-inhibit + +ITERATIONS=30 + +# Check if the environment variables are unset or empty +if [ -z "$SILICON_PATH" ]; then + echo "Please set the environment variable SILICON_PATH to the path to silicon.sh." + exit 1 +fi +if [ -z "$Z3_PATH" ]; then + echo "Please set the environment variable Z3_PATH to the path to the Z3 binary." + exit 1 +fi +if [ -z "$GOBRA_PATH" ]; then + echo "Please set the environment variable GOBRA_PATH to the path to the Gobra jar." + exit 1 +fi + +# profile with disabled set axioms +python3 profile.py ../experiments/synthetic_set/fully_assisted/fully_assisted.gobra --disableSetAxiomatization --z3RandomizeSeeds --iterations $ITERATIONS --silicon_path $SILICON_PATH --z3_path $Z3_PATH --gobra $GOBRA_PATH + +# profile all files "normally" +find ../experiments -type f -name "*.gobra" -exec python3 profile.py {} --z3RandomizeSeeds --iterations $ITERATIONS --silicon_path $SILICON_PATH --z3_path $Z3_PATH --gobra $GOBRA_PATH \; + +# generate plots for every csv +find ../experiments -type f -name "*.csv" -exec python3 plot.py --qi_size 9 9 {} \; diff --git a/evaluation/scripts/profile.py b/evaluation/scripts/profile.py new file mode 100644 index 0000000..e4d73e5 --- /dev/null +++ b/evaluation/scripts/profile.py @@ -0,0 +1,269 @@ +""" +Module for profiling Viper and Gobra programs using Silicon and Z3's quantifier instantiation profiling. + +This file is part of gobra-libs which is released under the MIT license. +See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE +for full license details. +""" + +import argparse +import csv +import logging +import os +import time +import subprocess + +from util import file_path + + +def main(): + """Main function of the profiler.""" + # Set up logging. + logging.basicConfig(level=logging.DEBUG) + + args = parse_args() + + metadata = generate_metadata(args) + + # Set Z3 environment for Gobra and Silicon. + os.environ["Z3_EXE"] = str(args.z3_path) + + # Generate Gobra file if needed. + if args.program_path.suffix == '.gobra': + if args.gobra_path is None: + logging.error("Path to Gobra jar is required for Gobra files.") + raise ValueError("Path to Gobra jar is required for Gobra files.") + + # TODO Refactor this so we can pass it using shell=False + command = f"java -jar -Xss128m {args.gobra_path} --printVpr --noVerify -i {args.program_path}" + + logging.info("Generating Viper file.") + time_checked_command(command, shell=True) + logging.info("Viper file generated.") + + vpr_file_path = args.program_path.with_suffix(".gobra.vpr") + # Due to program_path(), this is a Viper file. + else: + vpr_file_path = args.program_path + + data = [] + for i in range(0, args.iterations): + # --useOldAxiomatization: At the time of writing, the new axiomatization has anonymous axioms, which is why + # we use the old one with names. + command = [args.silicon_path, "--useOldAxiomatization", "--numberOfParallelVerifiers", "1", "--z3Args", + f'smt.qi.profile=true smt.qi.profile_freq={args.granularity}', vpr_file_path] + if args.z3RandomizeSeeds: + command.append("--z3RandomizeSeeds") + if args.disableSetAxiomatization: + # Get current directory of this file. + script_dir = os.path.dirname(os.path.abspath(__file__)) + command.append("--setAxiomatizationFile") + command.append(os.path.join(script_dir, 'noaxioms_sets.vpr')) + + logging.info(f"Running Silicon with profiling. Iteration: {i + 1} of {args.iterations}.") + command, execution_time = time_checked_command(command) + logging.info(f"Silicon finished in {execution_time} seconds.") + + logging.info("Processing Silicon's profiling output") + data_point = process_output(command.stdout) + logging.info("Silicon's profiling output processed.") + data_point["execution_time"] = execution_time + + data.append(data_point) + + # Write CSV files. + csv_path = (metadata["program_path"].parent / format_metadata(metadata)).with_suffix(".csv") + write_to_csv(data, csv_path) + + +def process_output(output): + """Process the data from Silicon's output. + + When we profile Silicon, based on the granularity, a quantifier instantiation may trigger the printing of how often + that quantifier has been instantiated up until now. This function processes that output and returns a dictionary + which maps quantifier names to the latest number of instantiations. + """ + # Drop Silicon's preamble and epilogue. + output = output.splitlines() + output = output[1:-1] + + # Keeps track of current number of quantifier instantiations. + result = {} + + # Process the output. + # Example of rows we are processing: + # [quantifier_instances] $Multiset[Int]_prog.card_non_negative : 2500 : 10 : 11 + for line in output: + columns = [column.strip() for column in line.split(":")] + + # We don't want to include "[quantifier_instances] " in the name. + name = columns[0].removeprefix("[quantifier_instances]").strip() + instantiations = int(columns[1]) + + # We prefix the name with "qi-" to indicate that it is a quantifier instantiation to avoid potential name + # clashes with execution_time. + result[f"qi-{name}"] = instantiations + + return result + + +def time_checked_command(command, shell=False): + """Runs a command and returns its runtime and its results. + + This function is a wrapper around subprocess.run. It measures the runtime and checks whether the command was + successful. If it wasn't, it prints all the information and raises RuntimeError. + """ + logging.debug(f"Running {command}") + try: + start_time = time.time() + command = subprocess.run(command, capture_output=True, text=True, check=True, shell=shell) + end_time = time.time() + execution_time = end_time - start_time + logging.debug(f"Command finished in {execution_time} seconds.") + return command, execution_time + except subprocess.CalledProcessError as e: + logging.error(f"Command {e.cmd} failed with return code {e.returncode}.") + logging.error(f"stdout: {e.stdout}") + logging.error(f"stderr: {e.stderr}") + raise RuntimeError(f"Command {e.cmd} failed with return code {e.returncode}.") + + +def parse_args(): + """Parse command line arguments.""" + parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter) + parser.add_argument("program_path", type=program_path, + help="Gobra or Viper program to be profiled") + parser.add_argument("--silicon_path", type=file_path, required=True, + help="path to silicon.sh") + parser.add_argument("--z3_path", type=file_path, required=True, + help="path to Z3 binary") + parser.add_argument("--gobra_path", type=file_path, required=False, + help="path to Gobra jar") + parser.add_argument("--iterations", type=positive, required=False, default=1, + help="number of times profiling is repeated") + parser.add_argument("--granularity", type=positive, required=False, default=1, + help="granularity of quantifier reporting") + parser.add_argument("--z3RandomizeSeeds", action="store_true", required=False, + help=("set various Z3 random seeds to random values. Note that " + "profiling may be non-deterministic even if this setting " + "is set to False.")) + parser.add_argument("--disableSetAxiomatization", action="store_true", required=False, + help="disable the axiomatization of set operations.") + return parser.parse_args() + + +def generate_metadata(args): + """Generate metadata for the CSV file.""" + logging.info("Generating metadata.") + metadata = {"program_path": args.program_path} + logging.info(f"Program path: {metadata['program_path']}") + + logging.info("Getting version of Silicon.") + command = [args.silicon_path] + logging.debug(f"Running {command}") + # This shouldn't be checked, as Silicon will return a non-zero exit code (no option for version). + command_stdout = subprocess.run(command, capture_output=True, text=True).stdout + # Convert + # Silicon 1.1-SNAPSHOT (7fea2aa7+) + # Command-line interface: Required option 'file' not found. + # Run with just --help for usage and options + # to + # 7fea2aa7 + metadata["silicon_version"] = command_stdout.splitlines()[0].split()[-1][1:-2] + logging.info(f"Silicon version: {metadata['silicon_version']}") + + logging.info("Getting version of Z3.") + command = [args.z3_path, "-version"] + command, _ = time_checked_command(command) + # Convert "Z3 version 4.8.7 - 64 bit" to "4_8_7". + metadata["z3_version"] = command.stdout.split()[2].replace('.', '_') + logging.info(f"Z3 version: {metadata['z3_version']}") + + # Get version of Gobra if applicable. + if args.gobra_path is not None: + logging.info("Getting version of Gobra.") + # TODO Refactor this so we can pass it using shell=False + command = f"java -jar {args.gobra_path} --version" + command, _ = time_checked_command(command, shell=True) + # Convert + # + # Gobra (c) Copyright ETH Zurich 2012 - 2022 + # version 1.1-SNAPSHOT (529d2a49@(detached)) + # + # to + # 529d2a49 + metadata["gobra_version"] = command.stdout.splitlines()[2].split()[-1].partition("@")[0][1:] + logging.info(f"Gobra version: {metadata['gobra_version']}") + else: + logging.info("Getting Gobra version not required. Continuing.") + + metadata["iterations"] = args.iterations + logging.info(f"Iterations: {metadata['iterations']}") + + metadata["granularity"] = args.granularity + logging.info(f"Granularity: {metadata['granularity']}") + metadata["z3RandomizeSeeds"] = args.z3RandomizeSeeds + logging.info(f"Z3 randomize seeds: {metadata['z3RandomizeSeeds']}") + metadata["disableSetAxiomatization"] = args.disableSetAxiomatization + logging.info(f"Disable set axiomatization: {metadata['disableSetAxiomatization']}") + + logging.info("Metadata generated.") + return metadata + + +def format_metadata(metadata): + """Format the metadata into a single string. + + This string is used as the filename of the CSV file. + """ + result = f'{metadata["program_path"].stem.replace(".", "_")}' + if metadata["z3RandomizeSeeds"]: + result += "-rand" + if metadata["disableSetAxiomatization"]: + result += "-no_set_axiom" + result += f'-iter_{metadata["iterations"]}' + result += f'-gran_{metadata["granularity"]}' + result += f'-sil_ver_{metadata["silicon_version"]}' + result += f'-z3_ver_{metadata["z3_version"]}' + if "gobra_version" in metadata: + result += f'-gobra_ver_{metadata["gobra_version"]}' + + return result + + +def write_to_csv(data, csv_path): + """Write the data to a CSV file.""" + logging.info(f"Writing {csv_path}.") + with open(csv_path, 'w', newline='') as csvfile: + writer = csv.DictWriter(csvfile, fieldnames=data[0].keys()) + writer.writeheader() + + for data_point in data: + writer.writerow(data_point) + + logging.info(f"Data written to {csv_path}.") + + +def program_path(string): + """Checks that the string is a valid path to either a Viper or Gobra program.""" + path = file_path(string) + if path.suffix not in [".gobra", ".vpr"]: + raise argparse.ArgumentTypeError(f"Wrong suffix: {path} is not a valid path to a program") + + # TODO Check whether this workaround is a bug in Gobra that we need to report + if not path.is_absolute(): + path = path.resolve() + + return path + + +def positive(string): + """Checks that the string is a positive integer.""" + value = int(string) + if value <= 0: + raise argparse.ArgumentTypeError(f"{value} is not larger or equal to 1") + return value + + +if __name__ == "__main__": + main() diff --git a/evaluation/scripts/util.py b/evaluation/scripts/util.py new file mode 100644 index 0000000..a560994 --- /dev/null +++ b/evaluation/scripts/util.py @@ -0,0 +1,17 @@ +""" +This file is part of gobra-libs which is released under the MIT license. +See LICENSE or go to https://github.com/viperproject/gobra-libs/blob/main/LICENSE +for full license details. +""" + +import argparse +import os +from pathlib import Path + + +def file_path(string): + """Checks that the string is a valid path to a file.""" + if os.path.isfile(string): + return Path(string) + else: + raise argparse.ArgumentTypeError(f"{string} is not a valid path to a file") diff --git a/evaluation/selected_plots/programproofs_assisted_compare_all.execution_time.pdf b/evaluation/selected_plots/programproofs_assisted_compare_all.execution_time.pdf new file mode 100644 index 0000000..1458b54 Binary files /dev/null and b/evaluation/selected_plots/programproofs_assisted_compare_all.execution_time.pdf differ diff --git a/evaluation/selected_plots/programproofs_assisted_compare_all.qi.pdf b/evaluation/selected_plots/programproofs_assisted_compare_all.qi.pdf new file mode 100644 index 0000000..f1434a5 Binary files /dev/null and b/evaluation/selected_plots/programproofs_assisted_compare_all.qi.pdf differ diff --git a/evaluation/selected_plots/programproofs_assisted_compare_no_first_half.execution_time.pdf b/evaluation/selected_plots/programproofs_assisted_compare_no_first_half.execution_time.pdf new file mode 100644 index 0000000..5a4a5f7 Binary files /dev/null and b/evaluation/selected_plots/programproofs_assisted_compare_no_first_half.execution_time.pdf differ diff --git a/evaluation/selected_plots/programproofs_assisted_compare_no_first_half.qi.pdf b/evaluation/selected_plots/programproofs_assisted_compare_no_first_half.qi.pdf new file mode 100644 index 0000000..ade2109 Binary files /dev/null and b/evaluation/selected_plots/programproofs_assisted_compare_no_first_half.qi.pdf differ diff --git a/evaluation/selected_plots/programproofs_first_half.execution_time.pdf b/evaluation/selected_plots/programproofs_first_half.execution_time.pdf new file mode 100644 index 0000000..aca814c Binary files /dev/null and b/evaluation/selected_plots/programproofs_first_half.execution_time.pdf differ diff --git a/evaluation/selected_plots/programproofs_first_half.qi.pdf b/evaluation/selected_plots/programproofs_first_half.qi.pdf new file mode 100644 index 0000000..09741c8 Binary files /dev/null and b/evaluation/selected_plots/programproofs_first_half.qi.pdf differ diff --git a/evaluation/selected_plots/standard_library_dicts_opaque.execution_time.pdf b/evaluation/selected_plots/standard_library_dicts_opaque.execution_time.pdf new file mode 100644 index 0000000..7cdf0a8 Binary files /dev/null and b/evaluation/selected_plots/standard_library_dicts_opaque.execution_time.pdf differ diff --git a/evaluation/selected_plots/standard_library_dicts_opaque.qi.pdf b/evaluation/selected_plots/standard_library_dicts_opaque.qi.pdf new file mode 100644 index 0000000..528a365 Binary files /dev/null and b/evaluation/selected_plots/standard_library_dicts_opaque.qi.pdf differ diff --git a/evaluation/selected_plots/standard_library_lemma_opaque.execution_time.pdf b/evaluation/selected_plots/standard_library_lemma_opaque.execution_time.pdf new file mode 100644 index 0000000..a2fa63e Binary files /dev/null and b/evaluation/selected_plots/standard_library_lemma_opaque.execution_time.pdf differ diff --git a/evaluation/selected_plots/standard_library_lemma_opaque.qi.pdf b/evaluation/selected_plots/standard_library_lemma_opaque.qi.pdf new file mode 100644 index 0000000..61b6086 Binary files /dev/null and b/evaluation/selected_plots/standard_library_lemma_opaque.qi.pdf differ diff --git a/evaluation/selected_plots/standard_library_sets_opaque.execution_time.pdf b/evaluation/selected_plots/standard_library_sets_opaque.execution_time.pdf new file mode 100644 index 0000000..f6e4973 Binary files /dev/null and b/evaluation/selected_plots/standard_library_sets_opaque.execution_time.pdf differ diff --git a/evaluation/selected_plots/standard_library_sets_opaque.qi.pdf b/evaluation/selected_plots/standard_library_sets_opaque.qi.pdf new file mode 100644 index 0000000..e5391b2 Binary files /dev/null and b/evaluation/selected_plots/standard_library_sets_opaque.qi.pdf differ diff --git a/evaluation/selected_plots/synthetic_assisted_compare.execution_time.pdf b/evaluation/selected_plots/synthetic_assisted_compare.execution_time.pdf new file mode 100644 index 0000000..54f5e7b Binary files /dev/null and b/evaluation/selected_plots/synthetic_assisted_compare.execution_time.pdf differ diff --git a/evaluation/selected_plots/synthetic_assisted_compare.qi.pdf b/evaluation/selected_plots/synthetic_assisted_compare.qi.pdf new file mode 100644 index 0000000..0767830 Binary files /dev/null and b/evaluation/selected_plots/synthetic_assisted_compare.qi.pdf differ diff --git a/evaluation/selected_plots/synthetic_compare_noaxioms.execution_time.pdf b/evaluation/selected_plots/synthetic_compare_noaxioms.execution_time.pdf new file mode 100644 index 0000000..ca40ddc Binary files /dev/null and b/evaluation/selected_plots/synthetic_compare_noaxioms.execution_time.pdf differ diff --git a/evaluation/selected_plots/synthetic_not_assisted_single.qi.pdf b/evaluation/selected_plots/synthetic_not_assisted_single.qi.pdf new file mode 100644 index 0000000..1255d14 Binary files /dev/null and b/evaluation/selected_plots/synthetic_not_assisted_single.qi.pdf differ diff --git a/evaluation/selected_plots/used_commands.md b/evaluation/selected_plots/used_commands.md new file mode 100644 index 0000000..169d636 --- /dev/null +++ b/evaluation/selected_plots/used_commands.md @@ -0,0 +1,23 @@ +# Description +This file lists the commands used to generate the combined plots used +in the report. Note that single plots are stored in the same directory +as the csv by default. Combined plots, or plots where `--name` was passed +are stored in the current working directory. + +# Commands +The following commands were executed from the `combined_plots` directory. + +## Using opaque +- standard_library_lemma_opaque: `python ../scripts/plot.py ../experiments/standard_library/lemma_not_opaque/lemma_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/standard_library/lemma_opaque/lemma_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --top 2 --name standard_library_lemma_opaque --qi_size 8 4 --variants "Default" "opaque"` +- standard_library_sets_opaque: `python ../scripts/plot.py ../experiments/standard_library/sets_not_opaque/sets_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/standard_library/sets_opaque/sets_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --top 3 --qi_size 8 4 --name standard_library_sets_opaque --start_at_zero_qi --variants "Default" "opaque"` +- standard_library_dicts_opaque: `python ../scripts/plot.py ../experiments/standard_library/dicts_not_opaque/dicts_not_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/standard_library/dicts_opaque/dicts_opaque-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --name standard_library_dicts_opaque --top 3 --qi_size 8 4 --variants "Default" opaque` + +## Manual Proofs +- synthetic_not_assisted_single: `python ../scripts/plot.py ../experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --name synthetic_not_assisted_single --top 6` +- synthetic_compare_noaxioms: `python ../scripts/plot.py ../experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/synthetic_set/fully_assisted/fully_assisted-rand-no_set_axiom-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --name synthetic_compare_noaxioms --variants Automatic Manual` + +## Assisting the Verifier +- synthetic_assisted_compare: `python ../scripts/plot.py --name synthetic_assisted_compare ../experiments/synthetic_set/fully_assisted/fully_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/synthetic_set/weak_diffleneq/weak_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/synthetic_set/no_diffleneq/no_diffleneq-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/synthetic_set/no_diffleneq_unionlenupper/no_diffleneq_unionlenupper-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/synthetic_set/not_assisted/not_assisted-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --qi_size 8 4 --execution_time_size 7 4 --top 3 --variants "Fully Assisted" "Weak Eq" "No Eq" "No Eq/Upper" "Unassisted"` +- programproofs_first_half: `python ../scripts/plot.py ../experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --name programproofs_first_half --top 5 --qi_size 8 4` +- programproofs_assisted_compare_all: `python ../scripts/plot.py ../experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/program_proofs_example_10_2/first_half/first_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --name programproofs_assisted_compare_all --variants "Full" "First Half" "Last Half" "Minimal" --qi_size 12 20` +- programproofs_assisted_compare_no_first_half: `python ../scripts/plot.py ../experiments/program_proofs_example_10_2/full/full-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/program_proofs_example_10_2/last_half/last_half-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv ../experiments/program_proofs_example_10_2/minimal/minimal-rand-iter_30-gran_1-sil_ver_0608ac9-z3_ver_4_8_7-gobra_ver_0608ac92.csv --name programproofs_assisted_compare_no_first_half --variants "Full" "Last Half" "Minimal" --top 3 --filter --qi_size 8 4`