upload

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Giovanni Rosa
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,121 @@
+# Dockleaner
+
+Rule-based refactoring tool for fixing Dockerfile smells detected by [hadolint](https://github.com/hadolint/hadolint).
+
+A detailed description of the tool and its evaluation can be found in the [preprint of the original article](https://giovannirosa.com/assets/pdf/rosa2024fixingsmells.pdf).
+
+## Setup
+
+The first step is to install [hadolint](https://github.com/hadolint/hadolint/releases/tag/v2.12.0).
+Next, clone the repository and install the requirements:
+```
+conda create -n dockleaner python=3.7
+conda activate dockleaner
+
+pip install -r requirements.txt
+```
+
+Dockleaner was tested on `macOS` and `Arch Linux` using `Python 3.7`.
+
+## Usage
+
+Run `python3 dockleaner.py --help` to see the following help message:
+
+```
+optional arguments:
+  -h, --help            show this help message and exit
+  --cache, -c           If selected, clears the cache of pulled image
+  --overwrite, -o       Set TRUE to overwrite the target Dockerfile after the
+                        fix
+  --ignore rules_to_ignore [rules_to_ignore ...], -i rules_to_ignore [rules_to_ignore ...]
+                        The rules that the solver must ignore
+  --rule rules_to_fix [rules_to_fix ...]
+                        Specify one or more specific rules to fix
+
+required arguments:
+  --path filepath, -p filepath
+                        The path of the Dockerfile
+  --last-edit dockerfile_date, -d dockerfile_date
+                        Last edit date of the given Dockerfile. Format "YYYY-
+                        MM-DD".
+```
+
+Thus, the command:
+```
+python3 -u dockleaner.py -p Dockerfile -d "2023-04-12" --rule "DL3008" --overwrite
+```
+will fix the Dockerfile `Dockerfile` by pinning versions for apt packages and overwriting the file.
+
+## Supported Smells
+
+- :calendar:         [DL3000](https://github.com/hadolint/hadolint/wiki/DL3000)
+- :calendar:         [DL3002](https://github.com/hadolint/hadolint/wiki/DL3002)
+- :white_check_mark: [DL3003](https://github.com/hadolint/hadolint/wiki/DL3003)
+- :calendar:         [DL3004](https://github.com/hadolint/hadolint/wiki/DL3004)
+- :calendar:         [DL3005](https://github.com/hadolint/hadolint/wiki/DL3005)
+- :white_check_mark: [DL3006](https://github.com/hadolint/hadolint/wiki/DL3006)
+- :white_check_mark: [DL3007](https://github.com/hadolint/hadolint/wiki/DL3007)
+- :white_check_mark: [DL3008](https://github.com/hadolint/hadolint/wiki/DL3008)
+- :white_check_mark: [DL3009](https://github.com/hadolint/hadolint/wiki/DL3009)
+- :calendar:         [DL3013](https://github.com/hadolint/hadolint/wiki/DL3013)
+- :ok:               [DL3014](https://github.com/hadolint/hadolint/wiki/DL3014)
+- :white_check_mark: [DL3015](https://github.com/hadolint/hadolint/wiki/DL3015)
+- :calendar:         [DL3016](https://github.com/hadolint/hadolint/wiki/DL3016)
+- :white_check_mark: [DL3020](https://github.com/hadolint/hadolint/wiki/DL3020)
+- :calendar:         [DL3022](https://github.com/hadolint/hadolint/wiki/DL3022)
+- :calendar:         [DL3024](https://github.com/hadolint/hadolint/wiki/DL3024)
+- :white_check_mark: [DL3025](https://github.com/hadolint/hadolint/wiki/DL3025)
+- :calendar:         [DL3027](https://github.com/hadolint/hadolint/wiki/DL3027)
+- :calendar:         [DL3028](https://github.com/hadolint/hadolint/wiki/DL3028)
+- :calendar:         [DL3029](https://github.com/hadolint/hadolint/wiki/DL3029)
+- :ok:               [DL3042](https://github.com/hadolint/hadolint/wiki/DL3042)
+- :calendar:         [DL3045](https://github.com/hadolint/hadolint/wiki/DL3045)
+- :ok:               [DL3047](https://github.com/hadolint/hadolint/wiki/DL3047)
+- :white_check_mark: [DL3048](https://github.com/hadolint/hadolint/wiki/DL3048)
+- :white_check_mark: [DL3059](https://github.com/hadolint/hadolint/wiki/DL3059)
+- :white_check_mark: [DL4000](https://github.com/hadolint/hadolint/wiki/DL4000)
+- :calendar:         [DL4001](https://github.com/hadolint/hadolint/wiki/DL4001)
+- :calendar:         [DL4003](https://github.com/hadolint/hadolint/wiki/DL4003)
+- :white_check_mark: [DL4006](https://github.com/hadolint/hadolint/wiki/DL4006)
+
+**Legend:**
+
+- :white_check_mark: Supported
+- :ok: Supported, but not validated via pull requests
+- :pushpin: To be implemented
+- :calendar: For future implementation
+
+## License
+
+The project is licensed under the MIT License. See [LICENSE](LICENSE) for details.
+
+## Acknowledgements
+
+The first prototype of Dockleaner was developed by [Alessandro Giagnorio](https://github.com/Devy99). Withouth his work, this project would not have been possible.
+The prototype has been extended and improved by [Simone Scalabrino](https://github.com/intersimone999) and [Giovanni Rosa](https://github.com/grosa1) which is currently the maintainer of the project.
+
+## How to Cite
+
+The proposal of the study, which includes the tool and its evaulation, was first presented at ICSME'22 - Registered Reports Track.
+```
+@article{rosa2022fixing,
+  title={Fixing dockerfile smells: An empirical study},
+  author={Rosa, Giovanni and Scalabrino, Simone and Oliveto, Rocco},
+  journal={arXiv preprint arXiv:2208.09097},
+  year={2022}
+}
+```
+
+Next, the complete study has been accepted at Empirical Software Engineering (EMSE).
+```
+@article{rosa2024fixingsmells,
+  author    = {Giovanni Rosa and
+              Federico Zappone and
+              Simone Scalabrino and
+              Rocco Oliveto},
+  title     = {Fixing Dockerfile Smells: An Empirical Study},
+  journal   = {Empirical Software Engineering},
+  year      = {2024},
+  note      = {To appear}
+}
+```

dockleaner.py

@@ -0,0 +1,191 @@
+from __future__ import annotations
+
+import argparse
+import datetime
+import difflib
+import logging
+from pyclbr import Class
+from typing import List
+
+import docker
+
+from logic.dockerfile_obj import Dockerfile
+from smell_solvers.smell_solver import SmellSolver
+from utils.cache_handler import clear_cache
+
+logging.basicConfig(level=logging.INFO, format='%(asctime)s :: %(levelname)s :: %(message)s')
+logger = logging.getLogger(__name__.split('.')[0])
+logger.setLevel(logging.DEBUG)
+
+
+def get_argparser() -> argparse.ArgumentParser:
+    """
+    Get the configured argument parser
+    """
+
+    parser = argparse.ArgumentParser(description='Fix Dockerfile smells')
+     # todo: check cache
+    parser.add_argument('--cache', '-c',
+                        action='store_true',
+                        dest='cache',
+                        required=False,
+                        help='If selected, clears the cache of pulled image')
+    parser.add_argument('--overwrite', '-o',
+                        action='store_true',
+                        dest='overwrite',
+                        required=False,
+                        help='Set TRUE to overwrite the target Dockerfile after the fix')
+    parser.add_argument('--ignore', '-i',
+                        metavar='rules_to_ignore',
+                        dest='ignored',
+                        nargs='+',
+                        required=False,
+                        help='The rules that the solver must ignore')
+    parser.add_argument('--rule',
+                        metavar='rules_to_fix',
+                        dest='to_fix',
+                        nargs='+',
+                        required=False,
+                        help='Specify one or more specific rules to fix')
+
+    required = parser.add_argument_group('required arguments')
+    required.add_argument('--path', '-p',
+                          metavar='filepath',
+                          dest='path',
+                          required=True,
+                          help='The path of the Dockerfile')
+# todo: if not date provided, use current as default
+    required.add_argument('--last-edit', '-d',
+                          metavar='dockerfile_date',
+                          dest='date',
+                          required=True,
+                          type=date_string,
+                          help='Last edit date of the given Dockerfile. Format "YYYY-MM-DD".')
+
+    return parser
+
+
+def date_string(s: str) -> datetime:
+    """
+    Get date from given date string
+    :param s: date string in the format 'YYYY-MM-DD'
+    :return: datetime.datetime(YYYY, MM, DD, 0, 0, 0)
+    """
+
+    try:
+        return datetime.datetime.strptime(s, "%Y-%m-%d")
+    except ValueError:
+        msg = f"Not a valid date (Expected format, 'YYYY-MM-DD'): '{s}'."
+        raise argparse.ArgumentTypeError(msg)
+
+
+def get_class(kls: str) -> Class:
+    """
+    Retrieve the instance of a specific class
+    :param kls: class filepath
+    :return: a Class object
+    """
+    parts = kls.split('.')
+    module = ".".join(parts[:-1])
+    m = __import__(module)
+    for comp in parts[1:]:
+        m = getattr(m, comp)
+    return m
+
+
+def fix_dockerfile(dockerfile: Dockerfile, ignored_rules: List) -> None:
+    """
+    Fix the given Dockerfile
+    :param dockerfile: Dockerfile to fix
+    :param ignored_rules: list of rules to not fix
+    """
+    RULES_PATH = 'smell_solvers.rules.'
+    solver = SmellSolver()
+
+    keys = list(dockerfile.smells_dict.keys())
+    fixed_lines = []
+    dict_changed = True
+    while dict_changed:
+        dict_changed = False
+
+        for pos in keys:
+            smells = dockerfile.smells_dict[pos]
+            for smell in smells:
+                if solver.strategy_exists(smell.code) and smell.code not in ignored_rules:
+                    logger.info(f'Fixing smell {smell.code} on line {smell.line}')
+                    # Dynamically choose the strategy
+                    solver.strategy = get_class(RULES_PATH + smell.code + '.' + smell.code)()
+                    solver.fix_smell(dockerfile, pos)
+                    fixed_lines.append(smell.line)
+
+            # Check if dictionary line position changed after fix
+            if dockerfile.lines_changed:
+                keys = list(set(dockerfile.smells_dict.keys()) - set(fixed_lines))
+                dict_changed = True
+                dockerfile.lines_changed = False
+                break
+
+            if dict_changed:
+                break
+
+
+def produce_dockerfile(dockerfile: Dockerfile, overwrite: bool) -> None:
+    """
+    Generate the fixed Dockerfile
+    :param dockerfile: Dockerfile to produce
+    :param overwrite: boolean that represent the need to overwrite or not the original file
+    """
+    filepath = dockerfile.filepath + '-fixed' if not overwrite else dockerfile.filepath
+
+    with open(filepath, 'w') as file:
+        file.writelines(dockerfile.lines)
+
+        if not overwrite:
+            logger.info(f'You can find the resulting Dockerfile at: {filepath}')
+
+
+def produce_log(filepath) -> None:
+    """
+    Generate the log of Dockerfile fixes
+    :param filepath: path of the original Dockerfile
+    """
+    original_dockerfile = open(filepath).readlines()
+    fixed_dockerfile = open(filepath + '-fixed').readlines()
+
+    diff = difflib.HtmlDiff().make_file(original_dockerfile, fixed_dockerfile, filepath, filepath + '-fixed', charset='utf-8')
+    with open(filepath + '-log.html', 'w', encoding='utf8') as file:
+        file.write(diff)
+
+
+if __name__ == '__main__':
+    try:
+        docker.from_env().info()
+    except:
+        logger.error("Docker daemon is not available. Please install and start Docker before running the tool.")
+
+    parser = get_argparser()
+    args = parser.parse_args()
+
+    if args.cache:
+        clear_cache()
+
+    fix_and_overwrite = True if args.overwrite else False
+
+    ignored_rules = list()
+    if args.to_fix:
+        logger.info("Fixing only: %s", args.to_fix)
+        ignored_rules = [r for r in SmellSolver()._available_strategies if r not in args.to_fix]
+    elif args.ignored:
+        logger.info("Ignoring rules: %s", args.ignored)
+        ignored_rules = args.ignored
+
+    dockerfile = Dockerfile(args.path, args.date)
+
+    if dockerfile.smells_dict:
+        fix_dockerfile(dockerfile, ignored_rules)
+        produce_dockerfile(dockerfile, fix_and_overwrite)
+
+        if not fix_and_overwrite:
+            produce_log(args.path)
+    else:
+        logger.info(f'Your Dockerfile has no smells.')