LATS_demo.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "LATS_demo.ipynb",
      "provenance": [],
      "private_outputs": true,
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyPQDFCGCLebkS43l9I5Zr1T",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/royorel/Lifespan_Age_Transformation_Synthesis/blob/master/LATS_demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jny9CGNw1Rjn",
        "colab_type": "text"
      },
      "source": [
        "# Lifespan Age Transformation Synthesis Demo\n",
        "\n",
        "This Colab notebook demonstrates the capabilities of the GAN architecture proposed in our paper. \n",
        "\n",
        "This colab lets you try our method on your own image!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NhbnEdCV2kc-",
        "colab_type": "text"
      },
      "source": [
        "First, let's download the github repository and install all dependencies."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J2uubo7PsvxQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!git clone https://github.com/royorel/Lifespan_Age_Transformation_Synthesis\n",
        "%cd Lifespan_Age_Transformation_Synthesis/\n",
        "!pip3 install -r requirements.txt"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GKKNh_xl3AI2",
        "colab_type": "text"
      },
      "source": [
        "Now let's download the pretrained models for males and females."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3ezAOkaHw4Q_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!python download_models.py"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nKl8INdX3IHv",
        "colab_type": "text"
      },
      "source": [
        "Here, we import libraries and set options."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rQKoh2xrw697",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import os\n",
        "from collections import OrderedDict\n",
        "from options.test_options import TestOptions\n",
        "from data.data_loader import CreateDataLoader\n",
        "from models.models import create_model\n",
        "import util.util as util\n",
        "from util.visualizer import Visualizer\n",
        "\n",
        "opt = TestOptions().parse(save=False)\n",
        "opt.display_id = 0 # do not launch visdom\n",
        "opt.nThreads = 1   # test code only supports nThreads = 1\n",
        "opt.batchSize = 1  # test code only supports batchSize = 1\n",
        "opt.serial_batches = True  # no shuffle\n",
        "opt.no_flip = True  # no flip\n",
        "opt.in_the_wild = True # This triggers preprocessing of in the wild images in the dataloader\n",
        "opt.traverse = True # This tells the model to traverse the latent space between anchor classes\n",
        "opt.interp_step = 0.05 # this controls the number of images to interpolate between anchor classes"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TLOiDzgKtYyi",
        "colab_type": "text"
      },
      "source": [
        "Don't worry about this message above, \n",
        "```\n",
        "ipykernel_launcher.py: error: unrecognized arguments: -f /root/.local/share/jupyter/runtime/kernel-c9d47a98-bdba-4a5f-9f0a-e1437c7228b6.json\n",
        "```\n",
        "everything is perfectly fine..."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4AFlzSi41Kzg",
        "colab_type": "text"
      },
      "source": [
        "Next on, we call the data loader and the visualizer class that generates the video from the network outputs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gxBYeTB18zkZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data_loader = CreateDataLoader(opt)\n",
        "dataset = data_loader.load_data()\n",
        "visualizer = Visualizer(opt)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-AbUVdK74G5p",
        "colab_type": "text"
      },
      "source": [
        "Here, we define our model.\n",
        "\n",
        "NOTE: if you plan to try the method for a female, change opt.name to 'females_model'."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O_0RChfq0YPr",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "opt.name = 'males_model' # change to 'females_model' if you're trying the code on a female image\n",
        "model = create_model(opt)\n",
        "model.eval()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "APhLIBEg4gnk",
        "colab_type": "text"
      },
      "source": [
        "OK, it's time to upload your image.\n",
        "\n",
        "For best results, use images according to the following guidelines:\n",
        "\n",
        "1. The image should contain a single face.\n",
        "2. Image was taken from a digital camera (phone cameras are fine). Old images from film cameras would produce low quality results.\n",
        "3. Pure RGB images only. No black & white, grayscale, sepia, or filtered images (e.g. Instagram filters).\n",
        "4. Person's head should directly face the camera. Looking sideways/downwards/upwards degrades the results.\n",
        "5. The person's face should not be occluded (or partially occluded) by any item.\n",
        "6. Both eyes should be open and visible. (eyeglasses are ok, no sunglasses)\n",
        "\n",
        "Your uploaded images are local to the Colab instance and are not accessible by the paper authors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TSUbmd697Api",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# upload your image (the code supports only a single image at a time)\n",
        "from google.colab import files\n",
        "uploaded = files.upload()\n",
        "for filename in uploaded.keys():\n",
        "  img_path = filename\n",
        "  print('User uploaded file \"{name}\"'.format(name=filename))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AgxXQ4tu45V5",
        "colab_type": "text"
      },
      "source": [
        "Finally, we preprocess the image, run the network, and save the result."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XJ3Az0VY3Fwt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data = dataset.dataset.get_item_from_path(img_path)\n",
        "visuals = model.inference(data)\n",
        "\n",
        "os.makedirs('results', exist_ok=True)\n",
        "out_path = os.path.join('results', os.path.splitext(img_path)[0].replace(' ', '_') + '.mp4')\n",
        "visualizer.make_video(visuals, out_path)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dRCyM2i65ZdB",
        "colab_type": "text"
      },
      "source": [
        "Let's display at the results.\n",
        "\n",
        "NOTE: if you're using chrome, uncomment the lines below. For some reason, mp4 files won't display on chrome browser, so we need to convert to webm."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hwjp6STO3hCz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "use_webm = False\n",
        "# For some unknown reason the mp4 video is not displayed on chrome\n",
        "# If you have chrome, uncomment the following lines to convert the \n",
        "# result to webm for display purposes\n",
        "\n",
        "# !pip3 install webm\n",
        "# webm_out_path = os.path.join('results', os.path.splitext(img_path)[0].replace(' ', '_') + '.webm')\n",
        "# !webm -i $out_path $webm_out_path\n",
        "# use_webm = True\n",
        "\n",
        "from IPython.display import HTML\n",
        "from base64 import b64encode\n",
        "video_path = webm_out_path if use_webm else out_path\n",
        "video_type = \"video/webm\" if use_webm else \"video/mp4\"\n",
        "mp4 = open(video_path,'rb').read()\n",
        "data_url = \"data:video/mp4;base64,\" + b64encode(mp4).decode()\n",
        "HTML(\"\"\"\n",
        "<video width={0} controls>\n",
        "      <source src=\"{1}\" type=\"{2}\">\n",
        "</video>\n",
        "\"\"\".format(opt.fineSize, data_url, video_type))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-RLnTezc59FZ",
        "colab_type": "text"
      },
      "source": [
        "You can download the result if you want"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0W87da7s9FM1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "files.download(out_path)"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}