Images Collection and Labelling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8c58fada",
   "metadata": {},
   "source": [
    "### **STEPS TO TAKE BEFORE CREATING THIS NOTEBOOK:**\n",
    "* **1.Create a new virtual environment (cmd)**  \n",
    "open the cmd to create a virtual environment (before creating this notebook) and put yourself in the cd path choice to develop this object detection project, then run in the cmd:\n",
    "  *python -m venv tfdet*  \n",
    "  \n",
    "\n",
    "* **2.Activate the virtual environment (cmd)**\n",
    "be in the same path and then use:   \n",
    " *.\\tfdet\\Scripts\\activate # Win*  \n",
    " \n",
    "\n",
    "* **3.Install dependencies and add virtual environment to the Python Kernel (cmd)**  \n",
    "Ipython --> to associate your virtual env with jupyter notebook  \n",
    " *python -m pip install --upgrade pip*  \n",
    " *pip install ipykernel*  \n",
    " *python -m ipykernel install --user --name=tfdet*  \n",
    " \n",
    "\n",
    "* **4.Create this notebook to make Image Collection**\n",
    "Still in the activated virtualenv, install jupyter and once this is done launch jupyter notebook (*jupyter notebook*) and then create this notebook. Once created specify that you want to use tfobdetection as the kernel.\n",
    "When opening the notebook you must always specify to use tfobdetection as the kernel, instead of Python3 (But if you work in a cloud environment (i.e. Colab) you do not need to make the step of creating a separate environment because they are already designed to be one )\n",
    "\n",
    "#!pip list\n",
    "\n",
    "#Extra:\n",
    "#CUDA and cuDNN ---> GPU BASED ACCELARARION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "bf55f917",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install opencv-python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2aa847b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "wd=os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "372aae6a",
   "metadata": {},
   "source": [
    "### 2. IMAGE COLLECTION AND LABELLING\n",
    "Collecting images using the webcam and labelling using LabelImg  \n",
    "**LABELING**:\n",
    "* case sensitive\n",
    "* prefer little labels (small square)\n",
    "* include pictures of objects at different angles and under different lighting conditions\n",
    "* 10/20 pictures for class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "09902799",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import opencv\n",
    "import cv2 \n",
    "\n",
    "# Import uuid\n",
    "import uuid\n",
    "\n",
    "# Import Operating System\n",
    "import os\n",
    "\n",
    "# Import time\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06b701e2",
   "metadata": {},
   "source": [
    "#### 2.1 Define Images to Collect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "dba43610",
   "metadata": {},
   "outputs": [],
   "source": [
    "#multi-class detection models\n",
    "labels = ['thumbsup', 'thumbsdown','open','rock']\n",
    "number_imgs = 5 #we are going to collect 5 images for each class, but more is better"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bb31728",
   "metadata": {},
   "source": [
    "#### 2.2. Setup Folders "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "fcb56cd9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensorflow\\workspace\\images\\collectedimages\n"
     ]
    }
   ],
   "source": [
    "#We need to create the folder to store the data\n",
    "IMAGES_PATH = os.path.join('Tensorflow', 'workspace', 'images', 'collectedimages')\n",
    "print(IMAGES_PATH) #folder inside IMAGES_PATH"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "39ee9ff7",
   "metadata": {},
   "outputs": [],
   "source": [
    "if not os.path.exists(IMAGES_PATH):\n",
    "    if os.name == 'posix': #linux machine\n",
    "        !mkdir -p {IMAGES_PATH}\n",
    "    if os.name == 'nt': #wind machine\n",
    "         !mkdir {IMAGES_PATH}\n",
    "for label in labels:\n",
    "    path = os.path.join(IMAGES_PATH, label)\n",
    "    if not os.path.exists(path):\n",
    "        !mkdir {path}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a672486b",
   "metadata": {},
   "source": [
    "#### 2.4 Capture Images using the webcam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "c2212ce6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting images for thumbsup\n",
      "Collecting image 0\n",
      "Collecting image 1\n",
      "Collecting image 2\n",
      "Collecting image 3\n",
      "Collecting image 4\n",
      "Collecting images for thumbsdown\n",
      "Collecting image 0\n",
      "Collecting image 1\n",
      "Collecting image 2\n",
      "Collecting image 3\n",
      "Collecting image 4\n",
      "Collecting images for open\n",
      "Collecting image 0\n",
      "Collecting image 1\n",
      "Collecting image 2\n",
      "Collecting image 3\n",
      "Collecting image 4\n",
      "Collecting images for rock\n",
      "Collecting image 0\n",
      "Collecting image 1\n",
      "Collecting image 2\n",
      "Collecting image 3\n",
      "Collecting image 4\n"
     ]
    }
   ],
   "source": [
    "for label in labels:\n",
    "    cap = cv2.VideoCapture(0) #connects to the webcam or caputure device\n",
    "    print('Collecting images for {}'.format(label))\n",
    "    time.sleep(5) #5 second to be in the position\n",
    "    for imgnum in range(number_imgs):\n",
    "        print('Collecting image {}'.format(imgnum))\n",
    "        ret, frame = cap.read()\n",
    "        imgname = os.path.join(IMAGES_PATH,label,label+'.'+'{}.jpg'.format(str(uuid.uuid1()))) #photo name: label+random number+.jpg\n",
    "        cv2.imwrite(imgname, frame) #write the file\n",
    "        cv2.imshow('frame', frame)\n",
    "        time.sleep(5) #5 second capturing time\n",
    "\n",
    "        if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "            break\n",
    "cap.release()\n",
    "cv2.destroyAllWindows() #close everything down"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2953b6dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "#if you want/need to collect images for one specific class, run:\n",
    "#labels=['thumbsup']\n",
    "#for label in labels:\n",
    "    #cap = cv2.VideoCapture(0) #connects to the webcam or caputure device\n",
    "    #print('Collecting images for {}'.format(label))\n",
    "    #time.sleep(5) #5 second to be in the position\n",
    "    #for imgnum in range(number_imgs):\n",
    "    #    print('Collecting image {}'.format(imgnum))\n",
    "    #    ret, frame = cap.read()\n",
    "    #    imgname = os.path.join(IMAGES_PATH,label,label+'.'+'{}.jpg'.format(str(uuid.uuid1()))) #photo name: label+random number+.jpg\n",
    "    #    cv2.imwrite(imgname, frame) #write the file\n",
    "    #    cv2.imshow('frame', frame)\n",
    "    #    */time.sleep(5) #5 second capturing time\n",
    "\n",
    "     #   if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "      #      break\n",
    "#cap.release()\n",
    "#cv2.destroyAllWindows() #close everything down"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6121fd05",
   "metadata": {},
   "outputs": [],
   "source": [
    "#if you do not want to resort to the acquisition of your own images through the PC camera, \n",
    "#you can download the images from google images\n",
    "#from icrawler.builtin import GoogleImageCrawler\n",
    "\n",
    "#google_crawler = GoogleImageCrawler(storage={'root_dir': 'your_image_dir'})\n",
    "\n",
    "#google_crawler = GoogleImageCrawler(storage={'root_dir': r'Tensorflow\\workspace\\images\\collectedimages\\thumbsup'})\n",
    "#google_crawler.crawl(keyword='thumb up hand', max_num=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fc1e6d8",
   "metadata": {},
   "source": [
    "### 3. Images Labelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1b31b467",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install --upgrade pyqt5 lxml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a55ffbae",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install labelImg\n",
    "# run labelImg in the virtual env to open the LabelImg app\n",
    "#the annotations are done by hand and are xml files containing information about the label\n",
    "#(coordinates, name etc.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4159744d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#otherwise you can clone the repository:\n",
    "#path to save the LabelImg\n",
    "#LABELIMG_PATH = os.path.join('Tensorflow', 'labelimg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "0ca83f33",
   "metadata": {},
   "outputs": [],
   "source": [
    "#if not os.path.exists(LABELIMG_PATH): #check if the path exist\n",
    "    #!mkdir {LABELIMG_PATH}\n",
    "    #!git clone https://github.com//tzutalin//labelImg {LABELIMG_PATH}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "bd785054",
   "metadata": {},
   "outputs": [],
   "source": [
    "#if os.name == 'posix':\n",
    " #   !make qt5py3\n",
    "#if os.name =='nt':\n",
    " #   !cd {LABELIMG_PATH} && pyrcc5 -o libs/resources.py resources.qrc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "4624ca53",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!cd {LABELIMG_PATH} && python labelImg.py #open the labelimg app \n",
    "#the annotations are done by hand and are xml files containing information about the label\n",
    "#(coordinates, name etc.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc535139",
   "metadata": {},
   "source": [
    "### 4. Training and Testing Partition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "356f9b6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#I subset manually the images in train e test folder in the following path:\n",
    "wd=os.getcwd()\n",
    "TRAIN_PATH = os.path.join(wd,'Tensorflow', 'workspace', 'images', 'train')\n",
    "TEST_PATH = os.path.join(wd,'Tensorflow', 'workspace', 'images', 'test')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tfdet",
   "language": "python",
   "name": "tfdet"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}