Faceswap 2.0 (deepfakes#1045)

* Core Updates
    - Remove lib.utils.keras_backend_quiet and replace with get_backend() where relevant
    - Document lib.gpu_stats and lib.sys_info
    - Remove call to GPUStats.is_plaidml from convert and replace with get_backend()
    - lib.gui.menu - typofix

* Update Dependencies
Bump Tensorflow Version Check

* Port extraction to tf2

* Add custom import finder for loading Keras or tf.keras depending on backend

* Add `tensorflow` to KerasFinder search path

* Basic TF2 training running

* model.initializers - docstring fix

* Fix and pass tests for tf2

* Replace Keras backend tests with faceswap backend tests

* Initial optimizers update

* Monkey patch tf.keras optimizer

* Remove custom Adam Optimizers and Memory Saving Gradients

* Remove multi-gpu option. Add Distribution to cli

* plugins.train.model._base: Add Mirror, Central and Default distribution strategies

* Update tensorboard kwargs for tf2

* Penalized Loss - Fix for TF2 and AMD

* Fix syntax for tf2.1

* requirements typo fix

* Explicit None for clipnorm if using a distribution strategy

* Fix penalized loss for distribution strategies

* Update Dlight

* typo fix

* Pin to TF2.2

* setup.py - Install tensorflow from pip if not available in Conda

* Add reduction options and set default for mirrored distribution strategy

* Explicitly use default strategy rather than nullcontext

* lib.model.backup_restore documentation

* Remove mirrored strategy reduction method and default based on OS

* Initial restructure - training

* Remove PingPong
Start model.base refactor

* Model saving and resuming enabled

* More tidying up of model.base

* Enable backup and snapshotting

* Re-enable state file
Remove loss names from state file
Fix print loss function
Set snapshot iterations correctly

* Revert original model to Keras Model structure rather than custom layer
Output full model and sub model summary
Change NNBlocks to callables rather than custom keras layers

* Apply custom Conv2D layer

* Finalize NNBlock restructure
Update Dfaker blocks

* Fix reloading model under a different distribution strategy

* Pass command line arguments through to trainer

* Remove training_opts from model and reference params directly

* Tidy up model __init__

* Re-enable tensorboard logging
Suppress "Model Not Compiled" warning

* Fix timelapse

* lib.model.nnblocks - Bugfix residual block
Port dfaker
bugfix original

* dfl-h128 ported

* DFL SAE ported

* IAE Ported

* dlight ported

* port lightweight

* realface ported

* unbalanced ported

* villain ported

* lib.cli.args - Update Batchsize + move allow_growth to config

* Remove output shape definition
Get image sizes per side rather than globally

* Strip mask input from encoder

* Fix learn mask and output learned mask to preview

* Trigger Allow Growth prior to setting strategy

* Fix GUI Graphing

* GUI - Display batchsize correctly + fix training graphs

* Fix penalized loss

* Enable mixed precision training

* Update analysis displayed batch to match input

* Penalized Loss - Multi-GPU Fix

* Fix all losses for TF2

* Fix Reflect Padding

* Allow different input size for each side of the model

* Fix conv-aware initialization on reload

* Switch allow_growth order

* Move mixed_precision to cli

* Remove distrubution strategies

* Compile penalized loss sub-function into LossContainer

* Bump default save interval to 250
Generate preview on first iteration but don't save
Fix iterations to start at 1 instead of 0
Remove training deprecation warnings
Bump some scripts.train loglevels

* Add ability to refresh preview on demand on pop-up window

* Enable refresh of training preview from GUI

* Fix Convert
Debug logging in Initializers

* Fix Preview Tool

* Update Legacy TF1 weights to TF2
Catch stats error on loading stats with missing logs

* lib.gui.popup_configure - Make more responsive + document

* Multiple Outputs supported in trainer
Original Model - Mask output bugfix

* Make universal inference model for convert
Remove scaling from penalized mask loss (now handled at input to y_true)

* Fix inference model to work properly with all models

* Fix multi-scale output for convert

* Fix clipnorm issue with distribution strategies
Edit error message on OOM

* Update plaidml losses

* Add missing file

* Disable gmsd loss for plaidnl

* PlaidML - Basic training working

* clipnorm rewriting for mixed-precision

* Inference model creation bugfixes

* Remove debug code

* Bugfix: Default clipnorm to 1.0

* Remove all mask inputs from training code

* Remove mask inputs from convert

* GUI - Analysis Tab - Docstrings

* Fix rate in totals row

* lib.gui - Only update display pages if they have focus

* Save the model on first iteration

* plaidml - Fix SSIM loss with penalized loss

* tools.alignments - Remove manual and fix jobs

* GUI - Remove case formatting on help text

* gui MultiSelect custom widget - Set default values on init

* vgg_face2 - Move to plugins.extract.recognition and use plugins._base base class
cli - Add global GPU Exclude Option
tools.sort - Use global GPU Exlude option for backend
lib.model.session - Exclude all GPUs when running in CPU mode
lib.cli.launcher - Set backend to CPU mode when all GPUs excluded

* Cascade excluded devices to GPU Stats

* Explicit GPU selection for Train and Convert

* Reduce Tensorflow Min GPU Multiprocessor Count to 4

* remove compat.v1 code from extract

* Force TF to skip mixed precision compatibility check if GPUs have been filtered

* Add notes to config for non-working AMD losses

* Rasie error if forcing extract to CPU mode

* Fix loading of legace dfl-sae weights + dfl-sae typo fix

* Remove unused requirements
Update sphinx requirements
Fix broken rst file locations

* docs: lib.gui.display

* clipnorm amd condition check

* documentation - gui.display_analysis

* Documentation - gui.popup_configure

* Documentation - lib.logger

* Documentation - lib.model.initializers

* Documentation - lib.model.layers

* Documentation - lib.model.losses

* Documentation - lib.model.nn_blocks

* Documetation - lib.model.normalization

* Documentation - lib.model.session

* Documentation - lib.plaidml_stats

* Documentation: lib.training_data

* Documentation: lib.utils

* Documentation: plugins.train.model._base

* GUI Stats: prevent stats from using GPU

* Documentation - Original Model

* Documentation: plugins.model.trainer._base

* linting

* unit tests: initializers + losses

* unit tests: nn_blocks

* bugfix - Exclude gpu devices in train, not include

* Enable Exclude-Gpus in Extract

* Enable exclude gpus in tools

* Disallow multiple plugin types in a single model folder

* Automatically add exclude_gpus argument in for cpu backends

* Cpu backend fixes

* Relax optimizer test threshold

* Default Train settings - Set mask to Extended

* Update Extractor cli help text
Update to Python 3.8

* Fix FAN to run on CPU

* lib.plaidml_tools - typofix

* Linux installer - check for curl

* linux installer - typo fix

.install/linux/faceswap_setup_x64.sh

@@ -70,6 +70,15 @@ check_for_sudo() {
     fi
 }
 
+check_for_curl() {
+    # Ensure that curl is available on the system
+    if ! command -V curl &> /dev/null ; then
+        error "'curl' is required for running the Faceswap installer, but could not be found. \
+        Please install 'curl' using the package manager for your distribution before proceeding."
+        exit 1
+    fi
+}
+
 create_tmp_dir() {
     TMP_DIR="$(mktemp -d)"
     if [ -z "$TMP_DIR" -o ! -d "$TMP_DIR" ]; then
@@ -336,10 +345,10 @@ delete_env() {
 }
 
 create_env() {
-    # Create Python 3.7 env for faceswap
+    # Create Python 3.8 env for faceswap
     delete_env
     info "Creating Conda Virtual Environment..."
-    yellow ; "$CONDA_EXECUTABLE" create -n "$ENV_NAME" -q python=3.7 -y
+    yellow ; "$CONDA_EXECUTABLE" create -n "$ENV_NAME" -q python=3.8 -y
 }
 
 
@@ -406,6 +415,7 @@ create_desktop_shortcut () {
 }
 
 check_for_sudo
+check_for_curl
 banner
 user_input
 review

.install/windows/install.nsi

@@ -22,7 +22,7 @@ InstallDir $PROFILE\faceswap
 # Install cli flags
 !define flagsConda "/S /RegisterPython=0 /AddToPath=0 /D=$PROFILE\MiniConda3"
 !define flagsRepo "--depth 1 --no-single-branch ${wwwRepo}"
-!define flagsEnv "-y python=3.7"
+!define flagsEnv "-y python=3.8"
 
 # Folders
 Var ProgramData

.travis.yml

@@ -4,7 +4,7 @@ language: shell
 
 env:
   global:
-    - CONDA_PYTHON=3.7
+    - CONDA_PYTHON=3.8
     - CONDA_BLD_PATH=${HOME}/conda-bld
 
 os:

Dockerfile.cpu

@@ -1,4 +1,4 @@
-FROM tensorflow/tensorflow:1.12.0-py3
+FROM tensorflow/tensorflow:2.2.0-py3
 
 RUN add-apt-repository -y ppa:jonathonf/ffmpeg-4 \
  && apt-get update -qq -y \

Dockerfile.gpu

@@ -1,4 +1,4 @@
-FROM tensorflow/tensorflow:1.15.0-gpu-py3
+FROM tensorflow/tensorflow:2.2.0-gpu-py3
 
 ENV DEBIAN_FRONTEND noninteractive
 

INSTALL.md

@@ -93,8 +93,8 @@ Reboot your PC, so that everything you have just installed gets registered.
 - Select "Create" at the bottom
 - In the pop up:
     - Give it the name: faceswap
-    - **IMPORTANT**: Select python version 3.7
-    - Hit "Create" (NB: This may take a while as it will need to download Python 3.7)
+    - **IMPORTANT**: Select python version 3.8
+    - Hit "Create" (NB: This may take a while as it will need to download Python)
 ![Anaconda virtual env setup](https://i.imgur.com/59RHnLs.png)
 
 #### Entering your virtual environment
@@ -155,7 +155,7 @@ Obtain git for your distribution from the [git website](https://git-scm.com/down
 The recommended install method is to use a Conda3 Environment as this will handle the installation of Nvidia's CUDA and cuDNN straight into your Conda Environment. This is by far the easiest and most reliable way to setup the project.
   - MiniConda3 is recommended: [MiniConda3](https://docs.conda.io/en/latest/miniconda.html)
 
-Alternatively you can install Python (>= 3.6-3.7 64-bit) for your distribution (links below.) If you go down this route and are using an Nvidia GPU you should install CUDA (https://developer.nvidia.com/cuda-zone) and cuDNN (https://developer.nvidia.com/cudnn). for your system. If you do not plan to build Tensorflow yourself, make sure you install no higher than version 10.0 of CUDA and 7.5.x of CUDNN.
+Alternatively you can install Python (>= 3.6-3.8 64-bit) for your distribution (links below.) If you go down this route and are using an Nvidia GPU you should install CUDA (https://developer.nvidia.com/cuda-zone) and cuDNN (https://developer.nvidia.com/cudnn). for your system. If you do not plan to build Tensorflow yourself, make sure you install no higher than version 10.0 of CUDA and 7.5.x of CUDNN.
   - Python distributions:
     - apt/yum install python3 (Linux)
     - [Installer](https://www.python.org/downloads/release/python-368/) (Windows)

_requirements_base.txt

@@ -3,10 +3,8 @@ psutil>=5.7.0
 pathlib==1.0.1
 numpy>=1.18.0
 opencv-python>=4.1.2.0
-scikit-image>=0.16.2
 pillow>=7.0.0
 scikit-learn>=0.22.0
-toposort==1.5
 fastcluster==1.1.26
 matplotlib>=3.0.3
 imageio>=2.8.0
@@ -15,16 +13,5 @@ ffmpy==0.2.3
 # Revert back to nvidia-ml-py3 when windows/system32 patch is implemented
 git+https://github.com/deepfakes/nvidia-ml-py3.git
 #nvidia-ml-py3
-h5py>=2.10.0
-Keras==2.2.4
 pywin32>=227 ; sys_platform == "win32"
 pynvx==1.0.0 ; sys_platform == "darwin"
-
-# tensorflow is included within the docker image.
-# If you are looking for dependencies for a manual install,
-
-# NB: Tensorflow version 1.12 is the minimum supported version of Tensorflow.
-#    If your graphics card support is below Cuda 9.0 you will need to either
-#    compile tensorflow yourself or download a custom version.
-#    Install 1.12.0<=tensorflow-gpu<=1.13.0 for CUDA 9.0
-#    or 1.13.1<=tensorflow-gpu<1.15 for CUDA 10.0

docs/full/lib/gui.rst

@@ -30,6 +30,28 @@ custom\_widgets module
    :undoc-members:
    :show-inheritance:
 
+display module
+==============
+.. automodule:: lib.gui.display
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+
+display\_analysis module
+========================
+.. autoclass:: lib.gui.display_analysis.Analysis
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+popup_configure module
+======================
+.. automodule:: lib.gui.popup_configure
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 project module
 ==============
 

docs/full/lib/logger.rst

@@ -0,0 +1,8 @@
+*************
+logger module
+*************
+
+.. automodule:: lib.logger
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/full/lib/model.rst

@@ -6,6 +6,13 @@ The Model Package handles interfacing with the neural network backend and holds
 .. contents:: Contents
    :local:
 
+model.backup_restore module
+---------------------------
+
+.. automodule:: lib.model.backup_restore
+   :members:
+   :undoc-members:
+   :show-inheritance:
 
 model.initializers module
 -------------------------
@@ -17,6 +24,7 @@ model.initializers module
 
    ~lib.model.initializers.ConvolutionAware
    ~lib.model.initializers.ICNR
+   ~lib.model.initializers.compute_fans
 
 .. automodule:: lib.model.initializers
    :members:
@@ -46,29 +54,44 @@ model.layers module
 model.losses module
 -------------------
 
+The losses listed here are generated from the docstrings in :mod:`lib.model.losses_tf`, however
+the functions are excactly the same for :mod:`lib.model.losses_plaid`. The correct loss module will
+be imported as :mod:`lib.model.losses` depending on the backend in use.
+
 .. rubric:: Module Summary
 
 .. autosummary::
    :nosignatures:
 
-   ~lib.model.losses.DSSIMObjective
-   ~lib.model.losses.PenalizedLoss
-   ~lib.model.losses.gaussian_blur
-   ~lib.model.losses.generalized_loss
-   ~lib.model.losses.gmsd_loss
-   ~lib.model.losses.gradient_loss
-   ~lib.model.losses.l_inf_norm
-   ~lib.model.losses.mask_loss_wrapper
-   ~lib.model.losses.scharr_edges
-
-.. automodule:: lib.model.losses
+   ~lib.model.losses_tf.DSSIMObjective
+   ~lib.model.losses_tf.PenalizedLoss
+   ~lib.model.losses_tf.GeneralizedLoss
+   ~lib.model.losses_tf.GMSDLoss
+   ~lib.model.losses_tf.GradientLoss
+   ~lib.model.losses_tf.LInfNorm
+
+.. automodule:: lib.model.losses_tf
    :members:
    :undoc-members:
    :show-inheritance:
 
 model.nn_blocks module
 ----------------------
 
+.. rubric:: Module Summary
+
+.. autosummary::
+   :nosignatures:
+
+   ~lib.model.nn_blocks.Conv2D
+   ~lib.model.nn_blocks.Conv2DBlock
+   ~lib.model.nn_blocks.Conv2DOutput
+   ~lib.model.nn_blocks.ResidualBlock
+   ~lib.model.nn_blocks.SeparableConv2DBlock
+   ~lib.model.nn_blocks.Upscale2xBlock
+   ~lib.model.nn_blocks.UpscaleBlock
+   ~lib.model.nn_blocks.set_config
+
 .. automodule:: lib.model.nn_blocks
    :members:
    :undoc-members:
@@ -89,14 +112,6 @@ model.normalization module
    :undoc-members:
    :show-inheritance:
 
-model.optimizers module
------------------------
-
-.. automodule:: lib.model.optimizers
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
 model.session module
 ---------------------
 

docs/full/lib/plaidml_stats.rst

@@ -0,0 +1,7 @@
+plaidml\_tools module
+=====================
+
+.. automodule:: lib.plaidml_tools
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/full/lib/utils.rst

@@ -0,0 +1,8 @@
+************
+utils module
+************
+
+.. automodule:: lib.utils
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/full/plugins/extract.rst

@@ -59,6 +59,14 @@ mask._base module
 -----------------
 
 .. automodule:: plugins.extract.mask._base
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+vgg\_face2\_keras module
+------------------------
+
+.. automodule:: plugins.extract.recognition.vgg_face2_keras
    :members:
    :undoc-members:
    :show-inheritance:

docs/full/plugins/train.rst

@@ -4,23 +4,40 @@ train package
 
 The Train Package handles the Model and Trainer plugins for training models in Faceswap.
 
-trainer._base module
-====================
+
+.. contents:: Contents
+   :local:
+
+model._base module
+==================
 
 .. rubric:: Module Summary
 
 .. autosummary::
    :nosignatures:
-
-   ~plugins.train.trainer._base.Batcher
-   ~plugins.train.trainer._base.PingPong
-   ~plugins.train.trainer._base.Samples
-   ~plugins.train.trainer._base.Timelapse
-   ~plugins.train.trainer._base.TrainerBase
-   ~plugins.train.trainer._base.TrainingAlignments
+
+   ~plugins.train.model._base.KerasModel
+   ~plugins.train.model._base.ModelBase
+   ~plugins.train.model._base.State
 
 .. rubric:: Module
 
+.. automodule:: plugins.train.model._base
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+model.original module
+=====================
+
+.. automodule:: plugins.train.model.original
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+trainer._base module
+====================
+
 .. automodule:: plugins.train.trainer._base
    :members:
    :undoc-members:

docs/sphinx_requirements.txt

@@ -1,25 +1,20 @@
 # NB Do not install from this requirements file
 # It is for documentation purposes only
 
-tqdm
-psutil
-pathlib
-numpy==1.16.2
-opencv-python==4.1.1.26
-scikit-image
-Pillow==6.1.0
-scikit-learn
-toposort
-fastcluster
-matplotlib==2.2.2
-imageio==2.5.0
-imageio-ffmpeg
-ffmpy==0.2.2
-# Revert back to nvidia-ml-py3 when windows/system32 patch is implemented
-git+https://github.com/deepfakes/nvidia-ml-py3.git
-#nvidia-ml-py3
-h5py==2.9.0
-Keras==2.2.4
-pywin32 ; sys_platform == "win32"
-pynvx==0.0.4 ; sys_platform == "darwin"
-tensorflow==1.13.1
+tqdm==4.42
+psutil==5.7.0
+pathlib==1.0.1
+numpy==1.18.0
+opencv-python==4.1.2.30
+pillow==7.0.0
+scikit-learn==0.22.0
+fastcluster==1.1.26
+matplotlib==3.0.3
+imageio==2.8.0
+imageio-ffmpeg==0.4.2
+ffmpy==0.2.3
+nvidia-ml-py3
+pywin32==227 ; sys_platform == "win32"
+pynvx==1.0.0 ; sys_platform == "darwin"
+plaidml-keras==0.7.0
+tensorflow==2.2.0