Benchmarking Android Data Leak Detection Tools

In 2017, Android hit a global mobile market share of 88% which makes it the most popular mobile platform. Application stores, such as the Google Play Store, are offering millions of mobile applications to the consumers, which are installed and updated on a daily basis. However, the security of those appli- cations is a major concern. A thorough security analysis before the publication of each application is time and resource consum- ing. Hence, platform providers cannot and do not manually vet every application handed in for publication. Consequently, many malicious and vulnerable applications find their way to the app stores and through there to the end users devices. Those appli- cations exhibit serious security issues, such as leaking of sensitive information. During the previous years, researchers have been proposing a myriad of techniques and tools to detect such issues. There also exist large scale taxonomies classifying such tools into different categories. However, it is unclear how these tools perform com- pared to each other. Such a comparison is almost infeasible, since most tools are no longer available or cannot be set up any more. In this work, we review static analysis tools for detecting data leaks in Android applications. Out of 87 tools in the vulnerability detection domain, we are able to obtain 22 tools. We then identify 5 tools in the data leak detection domain and run them. We run them on a given data set with known data leak vulnerabilities and compare their performance. Furthermore, we run the tools on a larger set of real-world applications to assess the prevalence of data leak issues in open-source Android applications. We propose our own approach to compare security analysis tools by normalising their interfaces. This simplifies result repro- duction and extension to other security vulnerability domains. In addition, the user experience and usability is highly improved.