regex_wwc.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "regex_wwc.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/nt03/Regex_WWC_DC/blob/master/regex_wwc.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PPgZ0IwOx1Nc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#import the python library 're' for regular expressions \n",
        "\n",
        "import re"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PhhVhPqT9JSn",
        "colab_type": "text"
      },
      "source": [
        "Each character in a regular expression is either:\n",
        "\n",
        "- a metacharacter, having a special meaning, or \n",
        "- a regular character that has a literal meaning.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YO35zF4EJp0t",
        "colab_type": "text"
      },
      "source": [
        "#Metacharacters\n",
        "\n",
        "- Python regex has 12 metacharacters\n",
        "- If you want your regex to match them literally, you need to escape them by placing a backslash `(\\)` in front of them.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rlxSOP-LSaF5",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "![alt text](https://www.optimizesmart.com/wp-content/uploads/2010/06/regex-cheatsheet-for-Google-Analytics1.jpg)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5n7jKU1FU0sV",
        "colab_type": "text"
      },
      "source": [
        "#Shorthands\n",
        "\n",
        "Six regex tokens that consist of a backslash and a letter form shorthand character classes: `‹\\d›, ‹\\D›, ‹\\w›, ‹\\W›, ‹\\s› and ‹\\S›`.\n",
        "\n",
        "You can use these both inside and outside character classes. Each lowercase shorthand character has an associated uppercase shorthand character with the opposite meaning.\n",
        "\n",
        "- `‹\\d›` match a single digit.\n",
        "- `‹\\w›` matches a single word character.\n",
        "- `‹\\s›` matches any whitespace character. \n",
        "\n",
        "A word character is a character that can occur as part of a word. That includes letters, digits, and the underscore."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CFOc9xx96fzH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#caret\n",
        "\n",
        "re.search(\"^\\d\", \"number 123\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vt5KZYst-mcD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "re.search(\"^\\d\", \"123\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UJ8vKnbi-sxy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#dollar \n",
        "\n",
        "re.search(\"\\d$\", \"number 123\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nEQpbr_3_M9c",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#dot\n",
        "\n",
        "re.search(\".\", \"number 123\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zj28v_iCF7il",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#asterisk\n",
        "\n",
        "re.findall(\".*\", \"number 123\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jhKoLgkr-yn_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#plus sign\n",
        "\n",
        "re.findall(\".+\", \"number 123\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0-ZBwTT3_RN4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#pipe symbol\n",
        "\n",
        "re.search(\"cat|dog\", \"I have a pet dog\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D-Go6F3Q_7Rx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#escape character\n",
        "\n",
        "s = \"The trademark (®) symbol is a symbol to indicate that the preceding mark is a trademark.\"\n",
        "\n",
        "re.search(\"\\(.\\)\", s)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MP8H6UPkC8oV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#curly brackets\n",
        "\n",
        "s = \"the year was 1990\"\n",
        "\n",
        "#re.search(\"\\d\\d\\d\\d\", s)\n",
        "re.search(\"\\d{4}\", s)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hI285EsUTGUE",
        "colab_type": "text"
      },
      "source": [
        "#Character Class\n",
        "\n",
        "A character class matches a single character out of a list of possible characters. The notation uses square brackets to list the character options. \n",
        "\n",
        "eg. `<[CcKk]ate>` is a regex with character class notation to indicate possible ways the name 'Kate' could be spelled.\n",
        "\n",
        "\n",
        "**By default, regular expressions are case sensitive.\n",
        "\n",
        "ex. ‹regex› matches regex but not Regex, REGEX, or ReGeX. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EGMCupfyUsyu",
        "colab_type": "text"
      },
      "source": [
        "* Inside a character class, only four characters have a special function: ` \\, ^, -, and ]`\n",
        "\n",
        "* All other characters are literals and simply add themselves to the character class. The regular expression ` ‹[$()*+.?{|]› ` matches any one of the nine characters between the square brackets.\n",
        "\n",
        "* A caret `(^)` negates the character class if you place it immediately after the opening bracket. It makes the character class match any character that is not in the list.\n",
        "\n",
        "* A hyphen `(-)` creates a range when it is placed between two characters. eg. `[a-zA-Z]` includes all upper and lowercase alphabets, whereas, `[a-h]` includes only lowercase alphabets from a to h."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EU0UW2D0Dvuv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# character class example\n",
        "\n",
        "s = \"Its easy to misspell Calendar as calender or calander so let's check using regex for all possible spellings and replace\"\n",
        "\n",
        "re.sub(\"[Cc]al[ae]nd[ae]r\", \"calendar\", s)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ohWAIEhHGEba",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#example - hyphen\n",
        "\n",
        "#find all year strings after 1700s in a text using hyphen \n",
        "\n",
        "s = \"US Senate Resolution 155 of 10 November 1997 states that the Declaration of Arbroath, the Scottish Declaration of Independence, was signed on 6 April 1320 and the American Declaration of Independence, 1776 was modelled on that inspirational document\"\n",
        "\n",
        "re.findall(\"[12][7-9][0-9]{2}\", s)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-yIaSBw8nUgS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#example: all characters except alphabets using caret\n",
        "\n",
        "re.findall(\"[^a-zA-Z\\s,]\", s)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KoBDdMqoh9dR",
        "colab_type": "text"
      },
      "source": [
        "# Grouping and Capturing\n",
        "\n",
        "Grouping is done with parentheses. They have the highest precedence of all regex operators. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D1R_ch3sjBIz",
        "colab_type": "text"
      },
      "source": [
        "However, a pair of parentheses isn’t just a group; it’s a capturing group. **Captures** become useful when they cover only part of the regular expression. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dZYRunHn5cow",
        "colab_type": "text"
      },
      "source": [
        "Problem: Create a regular expression that matches any date in yyyy-mm-dd format, and separately captures the year, month, and day."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nB11MmqN8JB8",
        "colab_type": "code",
        "colab": {},
        "cellView": "form"
      },
      "source": [
        "#@title TRY IT HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JXeTTtPs44Mh",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Solution\n",
        "\n",
        "d = \"2007-03-13\"\n",
        "\n",
        "x = re.search(r\"(\\d\\d\\d\\d)-(\\d\\d)-(\\d\\d)\", d)\n",
        "x.group()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KxnYdqao5-UP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(x.group(1), x.group(2), x.group(3))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "u6mp_-RL6JEQ",
        "colab_type": "code",
        "colab": {},
        "cellView": "both"
      },
      "source": [
        "#@title named capture groups\n",
        "\n",
        "d = \"2007-03-13\"\n",
        "\n",
        "x = re.search(r\"\\b(?P<year>\\d\\d\\d\\d)-(?P<month>\\d\\d)-(?P<day>\\d\\d)\\b\", d)\n",
        "\n",
        "print(x.group('year'))\n",
        "print(x.group('month'))\n",
        "print(x.group('day'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k5b5BLjKj0d5",
        "colab_type": "text"
      },
      "source": [
        "#LAZY QUANTIFIERS\n",
        "\n",
        "\n",
        "> You can make any quantifier lazy by placing a question mark after it: `‹*?›, ‹+?›, ‹??›, and ‹{7,42}?›` are all lazy quantifiers.\n",
        "\n",
        "Lazy quantifiers backtrack too, but the other way around. A lazy quantifier repeats as few times as it has to, stores one backtracking position, and allows the regex to continue.\n",
        "\n",
        "\n",
        "The quantifiers `‹*› and ‹*?›` allow all the same regular expression matches. The only difference is the order in which the possible matches are tried. The greedy quantifier will find the longest possible match. The lazy quantifier will find the shortest possible match."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K-4fSvKikeQj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "s = \"<p> The very <em>first</em> task is to find the beginning of a paragraph. </p> <p> Then you have to find the end of the paragraph </p>\"\n",
        "\n",
        "\n",
        "re.search(\"<p>.*</p>\", s).group()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LiPt2izAkjlb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "re.search(\"<p>.*?</p>\", s).group()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Eu_h1vTsR1Xm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "re.findall(\"great!*\", \"the music was great!!!\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XQyGsor2R5Tu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "re.findall(\"great!*?\", \"the music was great!!!\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NBedpDjNQ7u-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "re.findall(\"great!+\", \"the music was great!!!\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_M5cE5NNRVNm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "re.findall(\"great!+?\", \"the music was great!!!\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IscjLUAAudyH",
        "colab_type": "text"
      },
      "source": [
        "#RE Functions:\n",
        "\n",
        "\n",
        "1. **`re.search(pattern, string, flags)`** [link text](https://docs.python.org/3/library/re.html#re.search)\n",
        "\n",
        "> Scan through string looking for the first location where the regular expression pattern produces a match, and return a corresponding match object.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "snY6hlcUwEph",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "s = \"bill list: HR 1234, S 1023, S 234, HR 4505, S 8098\"\n",
        "\n",
        "re.search(\"(HR|S) (\\d{3,4})\", s)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "efA8hTHCyV2S",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "s = \"bill list: HR 1234, S 1023, S 234, HR 4505, S 8098\"\n",
        "\n",
        "re.match(\"(HR|S) (\\d{3,4})\", s)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0bWQQYqayciG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "s = \"bill list: HR 1234, S 1023, S 234, HR 4505, S 8098\"\n",
        "\n",
        "x = re.search(\"(HR|S) (\\d{3,4})\", s)\n",
        "\n",
        "print(x.group(1))\n",
        "print(x.group(2))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yeJr16ZPyvci",
        "colab_type": "text"
      },
      "source": [
        "2. **`re.findall(pattern, string, flags)`**[link text](https://docs.python.org/3/library/re.html#re.findall)\n",
        "\n",
        "> Return all non-overlapping matches of pattern in string, as a list of strings."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Die3bTaXzWcW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "s = \"bill list: HR 1234, S 1023, S 234, HR 4505, S 8098\"\n",
        "\n",
        "re.findall(\"(HR|S) (\\d{3,4})\", s)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PYYtqo9Gzg4a",
        "colab_type": "text"
      },
      "source": [
        "3. **`re.sub(pattern, replacement, string)`**[link text](https://docs.python.org/3/library/re.html#re.sub)\n",
        "\n",
        "Return the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement repl. If the pattern isn’t found, string is returned unchanged."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sx0BWBsGz-Yc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "s = \"bill list: HR 1234, S 1023, S 234, HR 4505, S 8098\"\n",
        "\n",
        "re.sub('\\s', \"\", s)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f3Y1I_9Z5j8j",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "re.sub('4', \"@\", s)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "67RFQnSO5tdd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "re.sub('\\d{4}', \"-\", s)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2bjJo7FF8tYZ",
        "colab_type": "text"
      },
      "source": [
        "#EXERCISES: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2fsELK7n8xS1",
        "colab_type": "code",
        "cellView": "both",
        "colab": {}
      },
      "source": [
        "#@title 1. Write a regex to check that a string contains only a certain set of characters (in this case a-z, A-Z and 0-9)\n",
        "\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jQDmezUrq017",
        "colab_type": "code",
        "cellView": "both",
        "outputId": "dbcafc85-5964-4615-fa5f-9b6620a96a89",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "#@title test block\n",
        "\n",
        "print(bool(re.search(**insert regex**, \"ABCDEFabcdef123450\")))\n",
        "\n",
        "print(bool(re.search(**insert regex**, \"*&%@#!}{\")))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "False\n",
            "True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w0QBqpvUPIIx",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title solution\n",
        "\n",
        "'[a-zA-Z0-9]'\n",
        "\n",
        "\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D_c6hJjRs6O8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#@title 2.Write a regex for a US phone number(10-digit), extract the state code in a named capture group\n",
        "\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ebvTNyMVTx9E",
        "colab_type": "code",
        "cellView": "both",
        "outputId": "0e11b0fe-3531-4778-bded-20718d85ced3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "#@title test block\n",
        "\n",
        "print(bool(re.search(**insert regex**, \"2027893452\")))\n",
        "\n",
        "print(bool(re.search(**insert regex**, \"0902352234\")))\n",
        "\n",
        "print(bool(re.search(**insert regex**, \"2930352234\")))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1nGiH6BkUDcr",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title solution\n",
        "\n",
        "'(?P<statecode>[2-9]\\d{2})[2-9][0-9]{6}'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j_0ByHJNV3N9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#@title 3. write a regex to replace all () in a text to whitespace\n",
        "\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3onKMyoBWBG0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#@title test block\n",
        "\n",
        "print(re.sub(**insert regex**, \"sdkn(huy67)\"))\n",
        "\n",
        "print(re.sub(**insert regex**, \"(x+y+z)= (x+y)+z\"))\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7PkaNxWKWHGx",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title solution\n",
        "\n",
        "\"[\\(\\)]\"\n"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}