-
Notifications
You must be signed in to change notification settings - Fork 7
Middleware'19 Artifact Evaluation
This page contains a link to the artifacts (in the form of a docker container) as well as detailed instruction explaining how to reproduce the security and performance evaluation from the paper.
This wiki is written for artifact evaluation and for those who want to reproduce our numbers. We provide a docker image that contains all the sources and scripts we used to get our results.
Please refer Section 6 of SlimGuard paper for our computer specifications, we recommend to use similar CPU as well as RAM.
In order to get the docker up and running, use following commands:
docker pull bcliu430/slimguard
docker run -it bcliu430/slimguard /bin/bashYou will then get a bash shell.
To view the graphs in PDF format, we recommend to copy the PDFs to your host
machine. To do so, please use docker cp command. For more information.
please visit docker
doc
Estimated time: 2 hours
In the /home directory, there is a script called run_parsec.sh.
simply type ./run_parsec.sh.
This will generate 4 files, including: vanilla, openbsd, guarder, and
slimguard. Each file contains three columns, the first column is name of
the benchmark, the second column is execution time in seconds, the third column
is Maximum memory foorprints.
Estimated time: 30s
In /home/mibench, there is a run.sh, in order to run the mibench
results, you will need to just use ./run.sh. It will generate 2 files for
each allcoator, one file is memory consumption in MB, the other one is
execution time in us. Refer run.sh for more details.
Figure 8 is obtained based on this benchmark.
To get the plot:
-
Generate Parsec results
run single thread script(run_parsec.sh) in
home, this will generate result files in/home/result/<allocator>.parsec, where allocator is the name of different allocators. -
Generate Mibench raw data
run mibench script(run.sh) in
/home/mibench, this generates the raw data in/home/mibench/result -
Pre-processing mibench raw data
run
./merge.sh. This will merge the mibench data, and put it in/home/result/<allocator>.mibench, where allocator should be the corresponding allocator name. -
Merge parsec and mibench result togegher
In
/home/result, run./merge.sh, it will generate./<allocator>.csvfor each allocator -
Plot Figure 8
In
/home/result, simply runpython3 plot_macro.pyand it will plot two graphs named time_overhead.pdf and mem_overhead.pdf
Estimated time: 1 hour
In the /home directory, there is a script called run_parsec_mt.sh.
simply type ./run_parsec_mt.sh. It will run a subset of parsec, and save
the results to files. Each file contains three columns, the first column is
name of the benchmark, the second column is execution time in seconds, the
third column is Maximum memory foorprints.
Figure 9 is obtained based on this benchmark.
To reproduce this result:
-
Generate raw data
In
/home, run./run_parsec_mt.sh. It will generate _< allocator >mt in/home/resultfor Glibc, SlimGuard and Guarder -
Plot Figure 9
In
/home/result, simply runpython3 plot_mt.py, this will generate a graph named multithreading.pdf
Estimated time: 10min
We believe, ideally, number of entropy bits has negligible impact on
performance. We setup different number of entropy bits(2, 4, 6, 8, 10) for both
Guarder and SlimGuard, we use a simple malloc/free microbenchmark to evaluate
the impacts. To modify the entropy bits for each allocator, slimguard has a
macro defined in /home/SlimGuard/include/slimguard.h:20; guarder defines
its entropy bits at /home/Guarder/xdefines.hh:118.
In order to run the benchmarks, following:
cd /home/microbench/entropy_study
./run.shThe results are in slimguard*.csv and guarder*.csv, each should contain two columns first column is malloc time, and the second column is free time, both in CPU ticks, please refer Figure 5 for more details. There is also result files named slimguard_< iteration >_< entropy bit >.txt, it contains the over-provisioning ratio on different situations, see table 2 for more details.
Figure 4, Figure 5 and Table 2 are obtained based on this benchmark.
To reproduce Figure 4/5, run python3 plot.py
To reproduce Table 2, simply do python3 overprovisioning.py, this
will give you a simple table in the shell showing the table 2
Estimated time: 10min
We measure the impact of numbers of guard pages on both SlimGuard and Guarder.
We setup different guard page ratios for both allocators. In SlimGuard, we set
up the guard page ratio at /home/SlimGuard/include/slimguard.h:19; for
Guarder, we can modify its guard page ratio at
/home/Guarder/xdefines.hh:137.
To run this set of benchmark, run:
cd /home/microbench/guard_page
./run.shThe results are in slimguard.csv and guarder.csv, each should contain two columns first column is malloc time, and the second column is free time, both in CPU ticks.
Figure 6 is obtained based on this benchmark.
To plot this, run python3 plot.py in /home/microbench/guard_page.
It will generate a pdf named gp_ratio.pdf
Estimated time: 10min
In order to measure the initialization time between SlimGuard and guarder, we
use a very simple program with only a malloc() and free() in it. We
also use different guard page ratios obtained from previous step to see the
impact for initialization time. Use the following commands to obtain results.
cd /home/microbench/init_time
./run.shThere are two files generated, guarder.csv and slimguard.csv, it contains 50 execution times of each allocator.
Figure 7 is obtained based on this benchmark.
To plot this, run python3 plot.py in /home/microbench/init_time.
It will generate a pdf named init.pdf
Since some of the vulnerabilities are patched, or it needs to convert from stack to heap, we have provided the source code in order to easily reproduce the bugs.
See Section 5.3 of SlimGuard paper for more details.
cd /home/bugbench/gzip-1.2.4/script
./run_bugcd /home/bugbench/ncompress/script
./run_bugcd /home/gnu-ed-1.14.1
./run_bug.shcd /home/ImageMagick-7.0.7-25/utilities
./run_bug.sh