feats: adding pytorch code to train and test

LICENCE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 Upciti
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.MD

@@ -1,5 +1,125 @@
 # On the Design of Privacy-Aware Cameras: A Study on Deep Neural Networks
-Accepted ECCVW IWDSC
 
+This repository contains Pytorch training and evaluation code for semantic segmentation using DeepLabv3. It was partially inspired from [aerial_mtl][mtl-github].
 
+For details, see [On the Design of Privacy-Aware Cameras: a Study on Deep Neural Networks][arxiv-paper].
+
+This paper was accepted on [ECCV 2022][eccv2022] [IWDSC][iwdsc] (International Workshop on Distributed Cameras).
+
+If you use this code for a paper, please cite: 
+
+```shell
+@article{carvalho2022privacy,
+  title={On the Design of Privacy-Aware Cameras: a Study on Deep Neural Networks},
+  author={Marcela Carvalho and Oussama Ennaffi and Sylvain Chateau and Samy Ait Bachir},
+  journal={available-soon},
+  year={2022}
+}
+```
 Code will be available soon.
+
+## Usage
+Clone this repository locally:
+```shell
+git clone https://github.com/upciti/privacy-by-design-semseg
+```
+
+We use poetry to... so, to install all the dependencies, we suggest using python ... and pyenv here is the explanation.
+
+Compared to conda, which is the togo solution in the field, poetry has the advantage of... However there are some problems when we use some library that is not on their repo.
+
+```shell
+poetry install  # reads the pyptoject.toml file, resolves dependencies and installs them
+poetry shell  # to activate the virtual environment in the current shell
+poe poe-torch-cuda11  # to install Pytorch with CUDA 11.6
+```
+
+## Data preparation
+To generate out-of-focus dataset, you must download ... image from Cityscape and disparity maps.
+
+To convert disparity maps to depth maps, we use:
+
+```shell
+this to convert to depth maps
+```
+
+Then, these depth maps are used with this matlab/octave code to generate defocused images using the method... The original code was written by ... and ... in ...
+
+run xyz to generate the dataset. Change the parameters according to the experiment in the paper, or personal usage.
+
+Then, define the following environment variable to the 
+
+## Model zoo
+
+| name               | url |
+|--------------------| --- |
+| infocus_color      | [weights will be soon available][model-link] |
+| infocus_gray       | [weights will be soon available][model-link] |
+| defocus_color      | [weights will be soon available][model-link] |
+| defocus_gray       | [weights will be soon available][model-link] |
+
+We suggest to save weights inside a folder called ```checkpoints/model_name```.
+
+For the command lines we mention in this README, we consider you saved ```infocus_color``` weights in ```./checkpoints/infocus_color/0300.pth.tar```.
+
+
+## Visualisation
+We adopt visdom to visualise training artifacts and metrics results. To run visdom, in another shell inside the project folder, run:
+
+```shell
+poetry shell
+visdom -p $DISPLAY_PORT
+```
+
+
+## Training
+To train a pretrained DeepLabV3 with one of our weights with a single GPU, run: 
+
+```shell
+python main.py --train --name name-of-the-new-project-with-pretrained-weights --dataroot PATH_TO_CITYSCAPES --batchSize $batch_size --nEpochs $end --display_id $display_id --port $port --use_resize --data_augmentation f f --resume
+```
+
+To train from scratch, run:
+
+```shell
+python main.py --train --name name-of-the-new-project --dataroot PATH_TO_CITYSCAPES --batchSize $batch_size --nEpochs $end --display_id $display_id --port $port --use_resize --data_augmentation f f 
+```
+
+
+## Evaluation
+To train a pretrained DeepLabV3 with one of our weights with a single GPU, run: 
+
+Generate images only:
+```shell
+python main.py --test --test_only --name infocus_color --save_samples --use_resize --display_id 0 --dataroot PATH_TO_DATA
+```
+
+
+Generate images and evaluation:
+```shell
+python main.py --test --test_metrics --name infocus_color --use_resize --display_id 0 --dataroot ./datasets/public_datasets/Cityscapes
+```
+
+As a result from this last run, you should get the outputs under the results file and the following metric results:
+```
+OA:  0.9387813619099232
+AA:  0.7375342230955388
+mIOU 0.6486886632431875
+```
+
+Generate only evaluation (resulting segmentation must be in the corresponding folder):
+```shell
+python main.py --test --evaluate_only --name infocus_color --use_resize --display_id 0 --dataroot ./datasets/public_datasets/Cityscapes
+```
+
+## License
+Code (scripts) are released under the [MIT license][license].
+
+[//]: # (References)
+
+[mtl-github]: https://github.com/marcelampc/aerial_mtl
+[iwdsc]: https://iwdsc.github.io/
+[eccv2022]: https://eccv2022.ecva.net/
+[arxiv-paper]: https://arxiv.org/list/cs.CV/recent
+[model-link]: broken
+[license]: LICENSE

config/pre_install.sh

@@ -0,0 +1,21 @@
+#conda install pytorch torchvision cuda80 -c soumith
+
+pip install ipdb
+pip install visdom
+conda install -c conda-forge tqdm
+conda install -c menpo opencv
+pip install -U matplotlib2tikz
+pip install seaborn
+
+# preinstall for DFC - # python 3.6.5
+conda install -c conda-forge rasterio
+conda install shapely
+conda install -c conda-forge geopandas
+pip install PyCRS
+
+# preinstall for DFC - # python 3.7
+conda install -c conda-forge rasterio
+conda install -c conda-forge libiconv
+conda install -c conda-forge geopandas
+# conda install -c anaconda shapely # comes with geopandas
+pip install PyCRS

dataloader/data_loader.py

@@ -0,0 +1,47 @@
+import torch
+import torchvision.transforms as transforms
+
+from dataloader.dataset import DatasetFromFolder
+
+
+def create_data_loader(opt, dataset=DatasetFromFolder):
+    data_transform = transforms.Compose(
+        [
+            transforms.ToTensor(),  # divides float version by 255
+            # normalize
+        ]
+    )
+
+    if opt.test or opt.visualize:
+        opt.batchSize = 1
+        shuffle = False
+        split = opt.test_split
+        phase = "test"
+    else:
+        shuffle = True
+        split = opt.train_split
+        phase = "train"
+
+    resize = opt.use_resize
+
+    set_dataloader = dataset(
+        opt=opt,
+        root=opt.dataroot,
+        phase=phase,
+        data_split=split,
+        data_augmentation=opt.data_augmentation,
+        resize=resize,
+        data_transform=data_transform,
+        image_size=opt.image_size,
+        output_size=opt.output_size,
+        dataset_name=opt.dataset_name,
+    )
+    data_loader = torch.utils.data.DataLoader(
+        set_dataloader,
+        batch_size=opt.batchSize,
+        shuffle=shuffle,
+        num_workers=opt.nThreads,
+        drop_last=True,
+    )
+
+    return data_loader

dataloader/dataset.py

@@ -0,0 +1,67 @@
+import torch.utils.data as data
+from PIL import ImageFile
+
+from .dataset_utils import check_files, get_paths_list, load_img
+
+ImageFile.LOAD_TRUNCATED_IMAGES = True
+from .online_data_augmentation import DataAugmentation
+
+
+class DatasetFromFolder(data.Dataset):
+    def __init__(
+        self,
+        opt=None,
+        root=None,
+        phase=None,
+        data_split=None,
+        data_augmentation=None,
+        resize=True,
+        data_transform=None,
+        image_size=[256],
+        output_size=0,
+        dataset_name="",
+    ):
+        super(DatasetFromFolder, self).__init__()
+        self.input_list, self.target_list = get_paths_list(root, phase, data_split)
+
+        check_files(self.input_list, root)
+
+        self.data_transform = data_transform
+        self.image_size = image_size if len(image_size) == 2 else image_size * 2
+        self.dataset_name = dataset_name
+
+        if output_size == 0:
+            self.output_size = self.image_size
+        else:
+            self.output_size = output_size if len(output_size) == 2 else output_size * 2
+
+        self.data_augmentation = data_augmentation
+        self.resize = resize
+        self.state = 0
+
+        self.phase = phase
+        self.data_augm_obj = DataAugmentation(
+            data_augmentation,
+            resize,
+            self.image_size,
+            data_transform=self.data_transform,
+        )
+        self.data_augm_obj.set_probabilities()
+
+    def __getitem__(self, index):
+        # not used on cityscapes
+        input_img = load_img(self.input_list[index])[0]
+        target_imgs = [load_img(target[index])[0] for target in self.target_list[index]]
+
+        self.data_augm_obj.set_probabilities()
+
+        input_img_tensor = self.data_augm_obj.apply_image_transform(input_img)[0]
+
+        targets_tensor = [
+            self.data_augm_obj.apply_image_transform(target)[0]
+            for i, target in enumerate(target_imgs)
+        ]
+        return input_img_tensor, targets_tensor
+
+    def __len__(self):
+        return len(self.input_list)

dataloader/dataset_bank.py

@@ -0,0 +1,88 @@
+from os import listdir
+from os.path import join
+from ipdb import set_trace as st
+import glob
+import sys
+
+IMG_EXTENSIONS = [
+    ".jpg",
+    ".JPG",
+    ".jpeg",
+    ".JPEG",
+    ".png",
+    ".PNG",
+    ".ppm",
+    ".PPM",
+    ".bmp",
+    ".BMP",
+]
+
+
+def dataset_cityscapes_get_rgb(root, data_split):
+    path = join(root, "leftImg8bit", data_split)
+    city = "*"
+    return sorted(glob.glob(join(path, city, "*.png")))
+
+
+def dataset_cityscapes_get_defocused_rgb(root, data_split, focus_prefix=""):
+    path = join(
+        root,
+        "defocus-blur-experiments" + focus_prefix,
+        "defocused_leftImg8bit",
+        data_split,
+    )
+    city = "*"
+    print(path)
+    return sorted(glob.glob(join(path, city, "*.png")))
+
+
+def dataset_cityscapes_get_defocused_rgb_focal_plane(root, data_split, focus_prefix=""):
+    path = join(root, "defocus-blur-experiments" + focus_prefix, "lp_val")
+    print(path)
+    return sorted(glob.glob(join(path, "*.png")))
+
+
+def dataset_cityscapes_get_disparity(root, data_split):
+    path = join(root, "disparity", data_split)
+    city = "*"
+    return sorted(glob.glob(join(path, city, "*.png")))
+
+
+def dataset_cityscapes_get_instance(root, data_split):
+    path = join(root, "gtFine_trainvaltest/gtFine", data_split)
+    city = "*"
+    return sorted(glob.glob(join(path, city, "*instanceIds.png")))
+
+
+def dataset_cityscapes_get_semantics(root, data_split):
+    path = join(root, "gtFine_trainvaltest/gtFine", data_split)
+    city = "*"
+    return sorted(glob.glob(join(path, city, "*labelIds.png")))
+
+
+def dataset_cityscapes_get_semantics_lp(root):
+    path = join(root, "*labelIds.png")
+    return sorted(glob.glob(path))
+
+
+def dataset_cityscapes(root, data_split, phase):
+    if "defocus_lp" in root:
+        focus_prefix = root.split("/")[-1].replace("defocus_lp", "")
+        root = root.split("/")[:-1]
+        root = join(*root)
+        rgb_images = dataset_cityscapes_get_defocused_rgb_focal_plane(
+            root, data_split, focus_prefix
+        )
+    elif "defocus" in root:
+        focus_prefix = root.split("/")[-1].replace("defocus", "")
+        root = root.split("/")[:-1]
+        root = join(*root)
+        rgb_images = dataset_cityscapes_get_defocused_rgb(
+            root, data_split, focus_prefix
+        )
+    else:
+        rgb_images = dataset_cityscapes_get_rgb(root, data_split)
+    if phase == "test":
+        return rgb_images
+
+    return rgb_images, dataset_cityscapes_get_semantics(root, data_split)