Logging (deepfakes#541)

* Convert prints to logger. Further logging improvements. Tidy up * Fix system verbosity. Allow SystemExit * Fix reload extract bug * Child Traceback handling * Safer shutdown procedure * Add shutdown event to queue manager * landmarks_as_xy > property. GUI notes + linting. Aligner bugfix * fix FaceFilter. Enable nFilter when no Filter is supplied * Fix blurry face filter * Continue on IO error. Better error handling * Explicitly print stack trace tocrash log * Windows Multiprocessing bugfix * Add git info and conda version to crash log * Windows/Anaconda mp bugfix * Logging fixes for training
o2team · Dec 4, 2018 · 7f53911 · 7f53911
1 parent 3df1371
commit 7f53911
Show file tree

Hide file tree

Showing 45 changed files with 2,178 additions and 1,155 deletions.
diff --git a/INSTALL.md b/INSTALL.md
@@ -13,10 +13,10 @@
     - [Notes](#notes)
 - [Windows Install Guide](#windows-install-guide)
     - [Prerequisites](#prerequisites-1)
+        - [Microsoft Visual Studio 2015](#microsoft-visual-studio-2015)
         - [Cuda](#cuda)
         - [cuDNN](#cudnn)
         - [CMake](#cmake)
-        - [Microsoft Visual Studio 2015](#microsoft-visual-studio-2015)
         - [Anaconda](#anaconda)
         - [Git](#git)
     - [Setup](#setup-1)
@@ -196,6 +196,20 @@ If you are experiencing issues, please raise them in the [faceswap-playground](h
 Setting up Faceswap can seem a little intimidating to new users, but it isn't that complicated, although a little time consuming. It is recommended to use Linux where possible as Windows will hog about 20% of your GPU Memory, making Faceswap run a little slower, however using Windows is perfectly fine and 100% supported.
 
 ## Prerequisites
+### Microsoft Visual Studio 2015
+**Important** Make sure to downoad the 2015 version of Microsoft Visual Studio
+
+Download and install Microsoft Visual Studio 2015 from: https://go.microsoft.com/fwlink/?LinkId=532606&clcid=0x409
+
+On the install screen:
+- Select "Custom" then click "Next"\
+![MSVS Custom](https://i.imgur.com/Bx8fjzT.png)
+- Uncheck all previously checked options
+- Expand "Programming Languages" and select "Visual C++"\
+![MSVS C++](https://i.imgur.com/c8k1IYD.png)
+- Select "Next" and "Install"
+
+
 ### Cuda
 **GPU Only** If you do not have an Nvidia GPU you can skip this step.
 
@@ -212,7 +226,7 @@ As with Cuda you will need to install the correct version of cuDNN that the late
 
 Download cuDNN from https://developer.nvidia.com/cudnn. You will need to create an account with Nvidia. 
 
-At the bottom of the list of latest cuDNN release will be a link to "Archived cuDNN Releases". Select this and choose the latest version of cuDNN that supports the version of Cuda you installed and is less than or equal to the latest version that Tensorflow supports. (Eg Tensorflow 1.12 supports Cuda 9.0 and cuDNN 7.2. There is not an archived version of cuDNN 7.2 for Cuda 9.0, so select cuDNN version 7.1)
+At the bottom of the list of latest cuDNN release will be a link to "Archived cuDNN Releases". Select this and choose the latest version of cuDNN that supports the version of Cuda you installed and has a minor version greater than or equal to the latest version that Tensorflow supports. (Eg Tensorflow 1.12 supports Cuda 9.0 and cuDNN 7.2. There is not an archived version of cuDNN 7.2 for Cuda 9.0, so select cuDNN version 7.3)
 - Open the zip file
 - Extract all of the files and folders into your Cuda folder (It is likely to be located in `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA`):\
 ![cudnn to cuda](https://i.imgur.com/X098w0N.png)
@@ -224,19 +238,6 @@ When installing CMake make sure to enable the option to CMake to the system path
 ![cmake path](https://i.imgur.com/XTtacdY.png)
 
 
-### Microsoft Visual Studio 2015
-**Important** Make sure to downoad the 2015 version of Microsoft Visual Studio
-
-Download and install Microsoft Visual Studio 2015 from: https://go.microsoft.com/fwlink/?LinkId=532606&clcid=0x409
-
-On the install screen:
-- Select "Custom" then click "Next"\
-![MSVS Custom](https://i.imgur.com/Bx8fjzT.png)
-- Uncheck all previously checked options
-- Expand "Programming Languages" and select "Visual C++"\
-![MSVS C++](https://i.imgur.com/c8k1IYD.png)
-- Select "Next" and "Install"
-
 ### Anaconda
 Download and install the latest Python 3 Anacconda from: https://www.anaconda.com/download/. Unless you know what you are doing, you can leave all the options at default.
 

diff --git a/lib/FaceFilter.py b/lib/FaceFilter.py
diff --git a/lib/Serializer.py b/lib/Serializer.py
@@ -2,7 +2,7 @@
 """
 Library providing convenient classes and methods for writing data to files.
 """
-import sys
+import logging
 import json
 import pickle
 
@@ -11,22 +11,28 @@
 except ImportError:
     yaml = None
 
+logger = logging.getLogger(__name__)  # pylint: disable=invalid-name
 
-class Serializer(object):
+
+class Serializer():
+    """ Parent Serializer class """
     ext = ""
     woptions = ""
     roptions = ""
 
     @classmethod
     def marshal(cls, input_data):
+        """ Override for marshalling """
         raise NotImplementedError()
 
     @classmethod
     def unmarshal(cls, input_string):
+        """ Override for unmarshalling """
         raise NotImplementedError()
 
 
 class YAMLSerializer(Serializer):
+    """ YAML Serializer """
     ext = "yml"
     woptions = "w"
     roptions = "r"
@@ -41,6 +47,7 @@ def unmarshal(cls, input_string):
 
 
 class JSONSerializer(Serializer):
+    """ JSON Serializer """
     ext = "json"
     woptions = "w"
     roptions = "r"
@@ -55,6 +62,7 @@ def unmarshal(cls, input_string):
 
 
 class PickleSerializer(Serializer):
+    """ Picke Serializer """
     ext = "p"
     woptions = "wb"
     roptions = "rb"
@@ -64,31 +72,33 @@ def marshal(cls, input_data):
         return pickle.dumps(input_data)
 
     @classmethod
-    def unmarshal(cls, input_bytes):
+    def unmarshal(cls, input_bytes):  # pylint: disable=arguments-differ
         return pickle.loads(input_bytes)
 
 
 def get_serializer(serializer):
+    """ Return requested serializer """
     if serializer == "json":
         return JSONSerializer
-    elif serializer == "pickle":
+    if serializer == "pickle":
         return PickleSerializer
-    elif serializer == "yaml" and yaml is not None:
+    if serializer == "yaml" and yaml is not None:
         return YAMLSerializer
-    elif serializer == "yaml" and yaml is None:
-        print("You must have PyYAML installed to use YAML as the serializer.\n"
-              "Switching to JSON as the serializer.", file=sys.stderr)
+    if serializer == "yaml" and yaml is None:
+        logger.warning("You must have PyYAML installed to use YAML as the serializer."
+                       "Switching to JSON as the serializer.")
     return JSONSerializer
 
 
 def get_serializer_from_ext(ext):
+    """ Get the sertializer from filename extension """
     if ext == ".json":
         return JSONSerializer
-    elif ext == ".p":
+    if ext == ".p":
         return PickleSerializer
-    elif ext in (".yaml", ".yml") and yaml is not None:
+    if ext in (".yaml", ".yml") and yaml is not None:
         return YAMLSerializer
-    elif ext in (".yaml", ".yml") and yaml is None:
-        print("You must have PyYAML installed to use YAML as the serializer.\n"
-              "Switching to JSON as the serializer.", file=sys.stderr)
+    if ext in (".yaml", ".yml") and yaml is None:
+        logger.warning("You must have PyYAML installed to use YAML as the serializer.\n"
+                       "Switching to JSON as the serializer.")
     return JSONSerializer
diff --git a/lib/aligner.py b/lib/aligner.py
@@ -1,12 +1,17 @@
 #!/usr/bin/env python3
 """ Aligner for faceswap.py """
 
+import logging
+
 import cv2
 import numpy as np
 
 from lib.umeyama import umeyama
 from lib.align_eyes import align_eyes as func_align_eyes, FACIAL_LANDMARKS_IDXS
 
+logger = logging.getLogger(__name__)  # pylint: disable=invalid-name
+
+
 MEAN_FACE_X = np.array([
     0.000213256, 0.0752622, 0.18113, 0.29077, 0.393397, 0.586856, 0.689483,
     0.799124, 0.904991, 0.98004, 0.490127, 0.490127, 0.490127, 0.490127,
@@ -36,49 +41,61 @@ class Extract():
 
     def extract(self, image, face, size, align_eyes):
         """ Extract a face from an image """
+        logger.trace("size: %s. align_eyes: %s", size, align_eyes)
         alignment = get_align_mat(face, size, align_eyes)
         extracted = self.transform(image, alignment, size, 48)
+        logger.trace("Returning face and alignment matrix: (alignment_matrix: %s)", alignment)
         return extracted, alignment
 
     @staticmethod
     def transform_matrix(mat, size, padding):
         """ Transform the matrix for current size and padding """
+        logger.trace("size: %s. padding: %s", size, padding)
         matrix = mat * (size - 2 * padding)
         matrix[:, 2] += padding
+        logger.trace("Returning: %s", matrix)
         return matrix
 
     def transform(self, image, mat, size, padding=0):
         """ Transform Image """
+        logger.trace("matrix: %s, size: %s. padding: %s", mat, size, padding)
         matrix = self.transform_matrix(mat, size, padding)
         return cv2.warpAffine(  # pylint: disable=no-member
             image, matrix, (size, size))
 
     def transform_points(self, points, mat, size, padding=0):
         """ Transform points along matrix """
+        logger.trace("points: %s, matrix: %s, size: %s. padding: %s", points, mat, size, padding)
         matrix = self.transform_matrix(mat, size, padding)
         points = np.expand_dims(points, axis=1)
         points = cv2.transform(  # pylint: disable=no-member
             points, matrix, points.shape)
-        return np.squeeze(points)
+        retval = np.squeeze(points)
+        logger.trace("Returning: %s", retval)
+        return retval
 
     def get_original_roi(self, mat, size, padding=0):
         """ Return the square aligned box location on the original
             image """
+        logger.trace("matrix: %s, size: %s. padding: %s", mat, size, padding)
         matrix = self.transform_matrix(mat, size, padding)
         points = np.array([[0, 0],
                            [0, size - 1],
                            [size - 1, size - 1],
                            [size - 1, 0]], np.int32)
         points = points.reshape((-1, 1, 2))
         matrix = cv2.invertAffineTransform(matrix)  # pylint: disable=no-member
+        logger.trace("Returning: (points: %s, matrix: %s", points, matrix)
         return cv2.transform(points, matrix)  # pylint: disable=no-member
 
     @staticmethod
     def get_feature_mask(aligned_landmarks_68, size,
                          padding=0, dilation=30):
         """ Return the face feature mask """
         # pylint: disable=no-member
-        scale = size - 2*padding
+        logger.trace("aligned_landmarks_68: %s, size: %s, padding: %s, dilation: %s",
+                     aligned_landmarks_68, size, padding, dilation)
+        scale = size - 2 * padding
         translation = padding
         pad_mat = np.matrix([[scale, 0.0, translation],
                              [0.0, scale, translation]])
@@ -123,12 +140,14 @@ def get_feature_mask(aligned_landmarks_68, size,
             kernel = np.ones((dilation, dilation), np.uint8)
             mask = cv2.dilate(mask, kernel, iterations=1)
 
+        logger.trace("Returning: %s", mask)
         return mask
 
 
 def get_align_mat(face, size, should_align_eyes):
     """ Return the alignment Matrix """
-    mat_umeyama = umeyama(np.array(face.landmarks_as_xy()[17:]),
+    logger.trace("size: %s, should_align_eyes: %s", size, should_align_eyes)
+    mat_umeyama = umeyama(np.array(face.landmarks_as_xy[17:]),
                           LANDMARKS_2D,
                           True)[0:2]
 
@@ -138,7 +157,7 @@ def get_align_mat(face, size, should_align_eyes):
     mat_umeyama = mat_umeyama * size
 
     # Convert to matrix
-    landmarks = np.matrix(face.landmarks_as_xy())
+    landmarks = np.matrix(face.landmarks_as_xy)
 
     # cv2 expects points to be in the form
     # np.array([ [[x1, y1]], [[x2, y2]], ... ]), we'll expand the dim
@@ -167,4 +186,5 @@ def get_align_mat(face, size, should_align_eyes):
     # Remove the extra row added, shape needs to be 2x3
     transform_mat = np.delete(transform_mat, 2, 0)
     transform_mat = transform_mat / size
+    logger.trace("Returning: %s", transform_mat)
     return transform_mat