t81_558_class_11_01_spacy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# T81-558: Applications of Deep Neural Networks\n",
    "**Module 11: Natural Language Processing and Speech Recognition**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 11 Material\n",
    "\n",
    "* **Part 11.1: Getting Started with Spacy in Python** [[Video]](https://www.youtube.com/watch?v=bv_iVVrlfbU) [[Notebook]](t81_558_class_11_01_spacy.ipynb)\n",
    "* Part 11.2: Word2Vec and Text Classification [[Video]](https://www.youtube.com/watch?v=qN9hHlZKIL4) [[Notebook]](t81_558_class_11_02_word2vec.ipynb)\n",
    "* Part 11.3: What are Embedding Layers in Keras [[Video]](https://www.youtube.com/watch?v=Ae3GVw5nTYU) [[Notebook]](t81_558_class_11_03_embedding.ipynb)\n",
    "* Part 11.4: Natural Language Processing with Spacy and Keras [[Video]](https://www.youtube.com/watch?v=Ae3GVw5nTYU) [[Notebook]](t81_558_class_11_04_text_nlp.ipynb)\n",
    "* Part 11.5: Learning English from Scratch with Keras and TensorFlow [[Video]](https://www.youtube.com/watch?v=Ae3GVw5nTYU) [[Notebook]](t81_558_class_11_05_english_scratch.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 11.1: Getting Started with Spacy in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will need to ensure that you've installed a language with Spacy.  If you do not, you will get the following error:\n",
    "\n",
    "```\n",
    "OSError: [E050] Can't find model 'en_core_web_sm'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.\n",
    "```\n",
    "\n",
    "To install English, use the following command:\n",
    "\n",
    "```\n",
    "python -m spacy download en\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Apple\n",
      "is\n",
      "looking\n",
      "at\n",
      "buying\n",
      "a\n",
      "U.K.\n",
      "startup\n",
      "for\n",
      "$\n",
      "1\n",
      "billion\n"
     ]
    }
   ],
   "source": [
    "import spacy\n",
    "\n",
    "nlp = spacy.load(\"en_core_web_sm\")\n",
    "doc = nlp(u\"Apple is looking at buying a U.K. startup for $1 billion\")\n",
    "for token in doc:\n",
    "    print(token.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I PRON\n",
      "want VERB\n",
      "an DET\n",
      "iPad PROPN\n",
      ", PUNCT\n",
      "Laptop PROPN\n",
      ", PUNCT\n",
      "and CCONJ\n",
      "a DET\n",
      "dog NOUN\n",
      ". PUNCT\n"
     ]
    }
   ],
   "source": [
    "for word in doc:  \n",
    "    print(word.text,  word.pos_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Apple is like number? False\n",
      "is is like number? False\n",
      "looking is like number? False\n",
      "at is like number? False\n",
      "buying is like number? False\n",
      "a is like number? False\n",
      "U.K. is like number? False\n",
      "startup is like number? False\n",
      "for is like number? False\n",
      "$ is like number? False\n",
      "1 is like number? True\n",
      "billion is like number? True\n"
     ]
    }
   ],
   "source": [
    "for word in doc:\n",
    "    print(f\"{word} is like number? {word.like_num}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sentence Diagramming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/jheaton/miniconda3/envs/tensorflow-2.0/lib/python3.6/runpy.py:193: UserWarning: [W011] It looks like you're calling displacy.serve from within a Jupyter notebook or a similar environment. This likely means you're already running a local web server, so there's no need to make displaCy start another one. Instead, you should be able to replace displacy.serve with displacy.render to show the visualization.\n",
      "  \"__main__\", mod_spec)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html>\n",
       "<html lang=\"en\">\n",
       "    <head>\n",
       "        <title>displaCy</title>\n",
       "    </head>\n",
       "\n",
       "    <body style=\"font-size: 16px; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol'; padding: 4rem 2rem; direction: ltr\">\n",
       "<figure style=\"margin-bottom: 6rem\">\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xml:lang=\"en\" id=\"3a74d43a917e4a229bdf43bcd98d364b-0\" class=\"displacy\" width=\"1450\" height=\"399.5\" direction=\"ltr\" style=\"max-width: none; height: 399.5px; color: #000000; background: #ffffff; font-family: Arial; direction: ltr\">\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"309.5\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"50\">I</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"50\">PRON</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"309.5\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"225\">want</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"225\">VERB</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"309.5\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"400\">an</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"400\">DET</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"309.5\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"575\">iPad,</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"575\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"309.5\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"750\">Laptop,</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"750\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"309.5\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"925\">and</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"925\">CCONJ</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"309.5\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"1100\">a</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"1100\">DET</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"309.5\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"1275\">dog.</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"1275\">NOUN</tspan>\n",
       "</text>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-3a74d43a917e4a229bdf43bcd98d364b-0-0\" stroke-width=\"2px\" d=\"M70,264.5 C70,177.0 215.0,177.0 215.0,264.5\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-3a74d43a917e4a229bdf43bcd98d364b-0-0\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">nsubj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M70,266.5 L62,254.5 78,254.5\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-3a74d43a917e4a229bdf43bcd98d364b-0-1\" stroke-width=\"2px\" d=\"M420,264.5 C420,177.0 565.0,177.0 565.0,264.5\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-3a74d43a917e4a229bdf43bcd98d364b-0-1\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">det</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M420,266.5 L412,254.5 428,254.5\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-3a74d43a917e4a229bdf43bcd98d364b-0-2\" stroke-width=\"2px\" d=\"M245,264.5 C245,89.5 570.0,89.5 570.0,264.5\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-3a74d43a917e4a229bdf43bcd98d364b-0-2\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">dobj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M570.0,266.5 L578.0,254.5 562.0,254.5\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-3a74d43a917e4a229bdf43bcd98d364b-0-3\" stroke-width=\"2px\" d=\"M595,264.5 C595,177.0 740.0,177.0 740.0,264.5\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-3a74d43a917e4a229bdf43bcd98d364b-0-3\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">conj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M740.0,266.5 L748.0,254.5 732.0,254.5\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-3a74d43a917e4a229bdf43bcd98d364b-0-4\" stroke-width=\"2px\" d=\"M770,264.5 C770,177.0 915.0,177.0 915.0,264.5\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-3a74d43a917e4a229bdf43bcd98d364b-0-4\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">cc</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M915.0,266.5 L923.0,254.5 907.0,254.5\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-3a74d43a917e4a229bdf43bcd98d364b-0-5\" stroke-width=\"2px\" d=\"M1120,264.5 C1120,177.0 1265.0,177.0 1265.0,264.5\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-3a74d43a917e4a229bdf43bcd98d364b-0-5\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">det</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1120,266.5 L1112,254.5 1128,254.5\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-3a74d43a917e4a229bdf43bcd98d364b-0-6\" stroke-width=\"2px\" d=\"M770,264.5 C770,2.0 1275.0,2.0 1275.0,264.5\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-3a74d43a917e4a229bdf43bcd98d364b-0-6\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">conj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1275.0,266.5 L1283.0,254.5 1267.0,254.5\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "</svg>\n",
       "</figure>\n",
       "</body>\n",
       "</html>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Using the 'dep' visualizer\n",
      "Serving on http://0.0.0.0:5000 ...\n",
      "\n",
      "Shutting down server on port 5000.\n"
     ]
    }
   ],
   "source": [
    "import spacy\n",
    "from spacy import displacy\n",
    "\n",
    "nlp = spacy.load(\"en_core_web_sm\")\n",
    "doc = nlp(u\"I want an iPad, Laptop, and a dog.\")\n",
    "displacy.serve(doc, style=\"dep\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I want an iPad, Laptop, and a dog.\n"
     ]
    }
   ],
   "source": [
    "print(doc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'the stripe bat be hang on -PRON- foot for good'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import spacy\n",
    "\n",
    "# Initialize spacy 'en' model, keeping only tagger component needed for lemmatization\n",
    "nlp = spacy.load('en', disable=['parser', 'ner'])\n",
    "\n",
    "sentence = \"The striped bats are hanging on their feet for best\"\n",
    "\n",
    "# Parse the sentence using the loaded 'en' model object `nlp`\n",
    "doc = nlp(sentence)\n",
    "\n",
    "# Extract the lemma for each token and join\n",
    "\" \".join([token.lemma_ for token in doc])\n",
    "#> 'the strip bat be hang on -PRON- foot for good'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stop Words\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'doing', 'per', 'whither', 'go', 'too', 'yourself', \"'s\", 'enough', 'part', 'during', 'the', '’re', 'many', 'it', 'others', \"'d\", 'besides', '’m', 'beside', 'out', 'formerly', 'anyway', 'hereafter', 'sometimes', 'would', 'she', 'did', 'seem', 'only', 'had', 'to', 'among', 'six', 'was', 'due', 'we', 'throughout', 'those', 'using', 'down', 'everywhere', 're', 'via', \"'re\", 'each', 'might', 'toward', 'myself', 'mine', 'has', 'same', 'something', 'four', 'last', 'every', 'through', 'without', 'less', 'along', 'have', 'top', 'whereupon', 'anything', 'meanwhile', 'perhaps', 'sometime', 'take', 'after', 'around', 'though', 'former', 'does', 'third', 'eight', 'onto', 'often', 'or', '‘re', 'its', 'your', 'but', 'now', 'whatever', 'therefore', 'became', 'serious', 'three', 'he', 'no', 'empty', 'all', 'none', 'much', 'sixty', 'whether', \"'m\", 'n‘t', 'him', 'between', 'seems', '’ll', 'with', 'any', 'done', '’d', 'otherwise', 'my', 'himself', 'either', 'fifteen', 'towards', 'twelve', 'whereafter', 'can', 'anyone', 'nobody', 'therein', 'were', '‘d', 'her', 'move', 'whom', 'beyond', 'amount', 'full', 'one', 'why', 'and', \"'ll\", 'not', 'back', 'becomes', 'below', 'amongst', 'namely', 'also', 'ourselves', 'since', 'most', 'somehow', 'together', '‘s', 'twenty', 'who', 'under', 'so', 'being', 'nowhere', 'whoever', 'are', 'except', 'above', 'anyhow', 'afterwards', 'hers', 'always', 'another', 'hereby', 'never', 'this', 'be', 'should', 'do', 'some', 'an', 'behind', 'well', 'what', 'first', 'name', 'a', 'them', 'once', 'give', 'ten', '’s', 'beforehand', 'herself', 'whose', 'nevertheless', 'side', 'hereupon', 'where', 'few', 'as', 'latterly', 'please', 'regarding', 'will', 'again', 'made', 'ours', 'bottom', 'anywhere', 'nothing', 'own', 'itself', 'while', 'yet', 'various', 'forty', 'used', 'wherein', 'five', 'moreover', 'mostly', 'herein', 'such', 'becoming', 'there', 'put', 'least', 'thence', 'me', 'that', 'off', 'other', 'yours', 'if', 'themselves', 'here', 'thereby', 'show', 'before', 'just', 'they', 'still', 'call', 'already', 'wherever', 'you', 'thru', 'several', 'thereupon', 'across', 'against', 'in', 'of', \"'ve\", 'n’t', 'although', 'someone', 'from', 'seeming', 'ever', 'up', 'however', 'keep', 'upon', 'indeed', 'whole', 'within', 'become', 'neither', 'us', 'into', 'been', 'further', 'our', 'these', 'am', 'elsewhere', 'everything', 'thus', 'whereas', 'ca', 'front', 'hundred', 'noone', '‘ve', 'nine', 'quite', 'by', 'rather', 'whence', 'whenever', 'than', '’ve', 'get', 'how', 'everyone', 'is', 'until', 'eleven', '‘m', 'somewhere', 'when', 'say', \"n't\", '‘ll', 'very', 'which', 'because', 'seemed', 'make', 'could', 'i', 'almost', 'at', 'on', 'his', 'latter', 'even', 'for', 'cannot', 'really', 'yourselves', 'about', 'their', 'next', 'then', 'thereafter', 'over', 'two', 'both', 'fifty', 'hence', 'unless', 'alone', 'may', 'whereby', 'nor', 'more', 'see', 'else', 'must'}\n"
     ]
    }
   ],
   "source": [
    "from spacy.lang.en.stop_words import STOP_WORDS\n",
    "\n",
    "print(STOP_WORDS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3.6 (tensorflow-2.0)",
   "language": "python",
   "name": "tensorflow-2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}