examples/mnist_colab_gpu.ipynb

{
 "nbformat": 4,
 "nbformat_minor": 0,
 "metadata": {
  "colab": {
   "name": "DeZero with GPU",
   "provenance": []
  },
  "kernelspec": {
   "name": "python3",
   "language": "python",
   "display_name": "Python 3"
  },
  "accelerator": "GPU",
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "source": [],
    "metadata": {
     "collapsed": false
    }
   }
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6KTc8kk5pmyL",
    "colab_type": "text"
   },
   "source": [
    "# DeZero with GPU\n",
    "\n",
    "Google Colabを利用すれば、DeZeroをGPU上で動作させることができます（Google ColabはGPUが無料で利用できます）。ここでは、DeZeroの計算をCPU/GPUと切り替えて実行し、GPUによってどれだけスピードが向上するかを観察します。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g3vUgNs7o__1",
    "colab_type": "text"
   },
   "source": [
    "# DeZeroのインストール\n",
    "\n",
    "まずはDeZeroをインストールします。DeZeroは[PyPI](https://pypi.org/project/dezero/)に登録してあるので、コマンドの`pip install dezero`からインストールすることができます。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "4Zh1CsPofLTs",
    "colab_type": "code",
    "outputId": "0c18c375-6b2d-4d86-9482-eaeca6451c91",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 121
    }
   },
   "source": [
    "pip install dezero"
   ],
   "execution_count": 1,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "Collecting dezero\n",
      "  Downloading https://files.pythonhosted.org/packages/1c/d0/bdc1949ff8bcba4a1cf572174e17cc7971daf30989f278c904f97c91ff3a/dezero-0.0.11-py3-none-any.whl\n",
      "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from dezero) (1.17.4)\n",
      "Installing collected packages: dezero\n",
      "Successfully installed dezero-0.0.11\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4hgp3XrwuDen",
    "colab_type": "text"
   },
   "source": [
    "続いて、DeZeroでGPUが使えるかをチェックしてみます。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "a2UkMP1Sftsn",
    "colab_type": "code",
    "outputId": "4df0740a-6713-4377-e6d1-d1ea18be6ef1",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    }
   },
   "source": [
    "import dezero\n",
    "dezero.cuda.gpu_enable"
   ],
   "execution_count": 0,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "metadata": {
      "tags": []
     },
     "execution_count": 7
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B8nmX5jaudDz",
    "colab_type": "text"
   },
   "source": [
    "`True`であればGPUが使える状態です。先に進みましょう。\n",
    "\n",
    "`False`の場合は、Google ColabのGPU設定が必要になります。それは下記の手順で行います。\n",
    "\n",
    "* メニューの「ランタイム」から「ランタイムのタイプを変更」を選択\n",
    "* 「ハードウェアアクセラレータ」のドロップメニューから「GPU」を選択"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z9NkCfZygLb4",
    "colab_type": "text"
   },
   "source": [
    "# Train MNIST with CPU\n",
    "それでは、DeZeroでMNISTの学習を行ってみます。\n",
    "まずはCPUで計算してみます。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "TavUkl90f83D",
    "colab_type": "code",
    "outputId": "1593a9b6-c4fd-4068-ec3b-c7ac207e7bfd",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 101
    }
   },
   "source": [
    "import time\n",
    "import dezero\n",
    "import dezero.functions as F\n",
    "from dezero import optimizers\n",
    "from dezero import DataLoader\n",
    "from dezero.models import MLP\n",
    "\n",
    "max_epoch = 5\n",
    "batch_size = 100\n",
    "cpu_times = []\n",
    "\n",
    "train_set = dezero.datasets.MNIST(train=True)\n",
    "train_loader = DataLoader(train_set, batch_size)\n",
    "model = MLP((1000, 10))\n",
    "optimizer = optimizers.SGD().setup(model)\n",
    "\n",
    "for epoch in range(max_epoch):\n",
    "    start = time.time()\n",
    "    sum_loss = 0\n",
    "\n",
    "    for x, t in train_loader:\n",
    "        y = model(x)\n",
    "        loss = F.softmax_cross_entropy(y, t)\n",
    "        model.cleargrads()\n",
    "        loss.backward()\n",
    "        optimizer.update()\n",
    "        sum_loss += float(loss.data) * len(t)\n",
    "\n",
    "    elapsed_time = time.time() - start\n",
    "    cpu_times.append(elapsed_time)\n",
    "    print('epoch: {}, loss: {:.4f}, time: {:.4f}[sec]'.format(\n",
    "        epoch + 1, sum_loss / len(train_set), elapsed_time))"
   ],
   "execution_count": 0,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "epoch: 1, loss: 1.9140, time: 7.8949[sec]\n",
      "epoch: 2, loss: 1.2791, time: 7.8918[sec]\n",
      "epoch: 3, loss: 0.9211, time: 7.9565[sec]\n",
      "epoch: 4, loss: 0.7381, time: 7.8198[sec]\n",
      "epoch: 5, loss: 0.6339, time: 7.9302[sec]\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4Xg1lxq3g6nS",
    "colab_type": "text"
   },
   "source": [
    "# Train MNIST on GPU\n",
    "続いてGPUを使って計算してみます。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "lXLb6TPZg1zO",
    "colab_type": "code",
    "outputId": "e7340e63-d531-4aa3-b721-74c47bc2ce62",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 101
    }
   },
   "source": [
    "gpu_times = []\n",
    "\n",
    "# GPU mode\n",
    "train_loader.to_gpu()\n",
    "model.to_gpu()\n",
    "\n",
    "for epoch in range(max_epoch):\n",
    "    start = time.time()\n",
    "    sum_loss = 0\n",
    "\n",
    "    for x, t in train_loader:\n",
    "        y = model(x)\n",
    "        loss = F.softmax_cross_entropy(y, t)\n",
    "        model.cleargrads()\n",
    "        loss.backward()\n",
    "        optimizer.update()\n",
    "        sum_loss += float(loss.data) * len(t)\n",
    "\n",
    "    elapsed_time = time.time() - start\n",
    "    gpu_times.append(elapsed_time)\n",
    "    print('epoch: {}, loss: {:.4f}, time: {:.4f}[sec]'.format(\n",
    "        epoch + 1, sum_loss / len(train_set), elapsed_time))"
   ],
   "execution_count": 0,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "epoch: 1, loss: 0.5678, time: 1.5356[sec]\n",
      "epoch: 2, loss: 0.5227, time: 1.5687[sec]\n",
      "epoch: 3, loss: 0.4898, time: 1.5498[sec]\n",
      "epoch: 4, loss: 0.4645, time: 1.5433[sec]\n",
      "epoch: 5, loss: 0.4449, time: 1.5512[sec]\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nlVfg7bux2w9",
    "colab_type": "text"
   },
   "source": [
    "以上より、結果は次のようになります。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "X8bf_VA6v2Fy",
    "colab_type": "code",
    "outputId": "2a11c0c9-83ae-4a46-9b56-3bf64062d84a",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 67
    }
   },
   "source": [
    "cpu_avg_time = sum(cpu_times) / len(cpu_times)\n",
    "gpu_avg_time = sum(gpu_times) / len(gpu_times)\n",
    "\n",
    "print('CPU: {:.2f}[sec]'.format(cpu_avg_time))\n",
    "print('GPU: {:.2f}[sec]'.format(gpu_avg_time))\n",
    "print('GPU speedup over CPU: {:.1f}x'.format(cpu_avg_time/gpu_avg_time))"
   ],
   "execution_count": 0,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "CPU: 7.90[sec]\n",
      "GPU: 1.55[sec]\n",
      "GPU speedup over CPU: 5.1x\n"
     ],
     "name": "stdout"
    }
   ]
  }
 ]
}