concepts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# ![Python logo](data/python.ico) Hello Python!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Python: history and now\n",
    "\n",
    "- Created by [Guido van Rossum](https://en.wikipedia.org/wiki/Guido_van_Rossum) in 1990.  At the time he was a postdoc at CWI, Amsterdam, currently he is at Dropbox.\n",
    "- Named after [**Monty Python**](https://en.wikipedia.org/wiki/Monty_Python%27s_Flying_Circus), Guido is a big fan.\n",
    "- Development guided by the Python Steering Council,\n",
    "  - entirely community led effort,\n",
    "  - supported by the Python Software Foundation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## The Python ecosystem\n",
    "\n",
    "- Ease of use among its primary considerations\n",
    "- Highly extendable, including through extensions written in lower level languages like C/C++.\n",
    "- Natively there are certain performance limitations (e.g. *Global Interpreter Lock* or GIL), but effectively not a limitation as it can be worked around with extensions; e.g. `numpy`, `xarray`, `pandas`, `numba`, `pyarrow`, etc.\n",
    "- Writing *native* extensions is easier with `Cython` (Python + `type` information + ability to by-pass the GIL)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Understanding variables\n",
    "\n",
    "Variables hold a *reference* to a value,\n",
    "- can be *objects* of simple *types* (e.g. numbers, strings, booleans),\n",
    "\n",
    "- user defined types,\n",
    "\n",
    "- *sequences* or *containers* (contains other variables), and others (esp. in Python)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Literals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# values\n",
    "1, 3.14, 0b10, 0x1e, \"string\", b\"bytes\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "a = 2.14  # variable assignment: hold a reference to a value\n",
    "a + 1  # the variable refers to the value later"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "a # what is a?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To refer to an `object` later, you must store a reference to it in a variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "x, y = 3, 3.0\n",
    "x, type(x), y, type(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# alternate notations for readability\n",
    "p, q = 10_000, 1.1e4\n",
    "p, type(p), q, type(q)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Numeric operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "# increment, similar for most other operators: -=, *=, /=\n",
    "a = 40\n",
    "a = a + 1\n",
    "a += 1\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Numeric operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# division\n",
    "1 / 2, 4 / 3, 4 / 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# floor division\n",
    "1 // 2, 4 // 3, 4 // 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# modulo/remainder, exponent\n",
    "4 % 3, 3 ** 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Booleans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "eq = p == 10_000\n",
    "eq, type(eq)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# numbers and strings: 0, empty string, None -> False, everything else -> True\n",
    "bool(1), bool(0), bool(\"\"), bool(\"foo\"), bool(None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# containers: empty -> False, has element -> True\n",
    "bool(list()), bool([1]), bool(set()), bool({1}), bool(dict()), bool({1: 4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "bool(None), bool([]), bool([1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bool([None])  # given the above, what does this evaluate to?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Boolean operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "-2 > 3 or -1 < 0, 2 > 3 and -1 > 3, not True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "##### Truth table\n",
    "\n",
    "<table>\n",
    "<tr><th>Logical OR</th><th>Logical AND</th></tr>\n",
    "<tr><td>\n",
    "\n",
    "| `or` | T | F |\n",
    "|:----:|---|---|\n",
    "| T    | T | T |\n",
    "| F    | T | F |\n",
    "\n",
    "</td><td>\n",
    "\n",
    "| `and` | T | F |\n",
    "|:-----:|---|---|\n",
    "| T     | T | F |\n",
    "| F     | F | F |\n",
    "\n",
    "</td></tr> </table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "txt = \"foo bar baz\"  # 'also valid'\n",
    "txt, type(txt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# escaping, and nested quotes\n",
    "'Don\\'t be an ass', \"Don't be an ass\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "multi = \"First\\nSecond\"\n",
    "multi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "print(multi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Triple quoted strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prose = \"\"\"This is a pre-formatted string.\n",
    "\n",
    "You may have paragraphs, and lists:\n",
    "- an item\n",
    "- no need for \"escaping\"\n",
    "\n",
    "    Or whatever you like\n",
    "\n",
    "\"\"\"\n",
    "print(prose)\n",
    "prose"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### String operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "# concatenation\n",
    "\"foo\" \"bar\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "\"foo\"\\\n",
    "\"bar\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# append\n",
    "a = \"foo\"\n",
    "b = \"bar\"\n",
    "a + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# multiply\n",
    "\"--8<-\" * 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# in\n",
    "\"foo\" in \"foo bar baz\", \"foo\" not in \"foo bar baz\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### User defined types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "class MyClass:\n",
    "    def __init__(self, name):\n",
    "        self.name = name\n",
    "\n",
    "    def __repr__(self):\n",
    "        return f'MyClass(\"{self.name}\")'\n",
    "\n",
    "\n",
    "instance = MyClass(\"foo\")\n",
    "\n",
    "instance, type(instance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Containers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "l = [1, 2.0, 3, 4.0, \"foo\", True]\n",
    "t1 = (1, 3.14, True)\n",
    "\n",
    "type(l), type(t1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# append\n",
    "l + [1, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# multiply\n",
    "t1 * 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Indexing\n",
    "\n",
    "Syntax: `container[index]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "l[0] == t1[0] == 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "l[-1] == True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Slicing\n",
    "\n",
    "Syntax: `container[start:stop:step]` (only one is required)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l, l[:2], l[1:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "l[0:4:2], l[::3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "txt = \"foo bar baz\"\n",
    "txt[4:7]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Mutable and immutable types\n",
    "\n",
    "Certain types, once created cannot be \"edited\"; e.g. `str`, `tuple`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "l  # mutable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "l[0] = 1000\n",
    "l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "t1, txt  # immutable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "t1[0] = 1000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "txt[5] = \"e\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Operators & type conversion\n",
    "\n",
    "- Implicit type conversion on supported operations\n",
    "- Result is the most representative type: support both operands **and** the result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "1 +  3.14, type(1 + 3.14), 1 + True, type(1 + True), 2.16 * False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "\"1\" + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "1 + \"1\"  # same for: \"foo\" - \"foo\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Flow control\n",
    "\n",
    "- conditionals\n",
    "- iteration\n",
    "- callable / functions: reusable routines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Conditionals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if .. else ..\n",
    "txt = \"foo bar baz\"\n",
    "if \"foo\" in txt:\n",
    "    pass\n",
    "elif \"bar\" in txt:  # optional\n",
    "    pass\n",
    "else:  # optional\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# ternary conditional\n",
    "par = None\n",
    "5 if par is None else par"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Iteration\n",
    "\n",
    "#### `for` loops"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# iterate over items\n",
    "t1 = (1, 3.14, True)\n",
    "for i in t1:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# iterate by index\n",
    "for i in range(len(t1)):  # len(..) returns the length of a sequence\n",
    "    print(t1[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# iterate by index\n",
    "for i in range(1, 10, 2):\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `while` loops\n",
    "\n",
    "*What does the following do?*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# flexible iteration\n",
    "txt = \"foo bar baz\"\n",
    "i = 0\n",
    "res = []\n",
    "while i < len(txt):\n",
    "    res += [txt[-1 - i]]\n",
    "    i += 1\n",
    "\"\".join(res)  # calling: str.join(Iterable[str]) -> str"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**Ans:** *Reverse the string `txt`*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `break` and `continue` statements\n",
    "\n",
    "*What does the following do?*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "txt = \"foo bar baz\"\n",
    "i = 0\n",
    "while i < len(txt):\n",
    "    if txt[i] == \"b\":\n",
    "        break  # prematurely ends any iteration\n",
    "    i += 1\n",
    "txt[:i]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**Ans:** *Finds the first b / Shows the string upto the first b*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "*What does the following do?*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "txt = \"foo bar baz\"\n",
    "i = 0\n",
    "res = \"\"\n",
    "while i < len(txt):\n",
    "    char = txt[i]\n",
    "    i += 1\n",
    "    if char == \"b\":\n",
    "        continue  # skip this iteration\n",
    "    res += char\n",
    "res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**Ans:** *Remove all b-s from the string*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Callables / Functions\n",
    "\n",
    "- Wraps a code block that can be reused, arguments act as parameters\n",
    "- Ends in a `return` statement; returns control back to the caller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add(i, j):\n",
    "    return i + j\n",
    "\n",
    "def sub(i, j):\n",
    "    return i - j\n",
    "\n",
    "def op(i, j, operator=add):  # default operator: addition\n",
    "    \"\"\"Applies a binary operator to two numbers\"\"\"  # <- docstring\n",
    "    return operator(i, j)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "op(3, 4)  # addition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "op(3, 4, sub)  # subtraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# multiply w/ anonymous function\n",
    "op(3, 4, lambda i, j: i * j)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "In the absence of a `return` statement, `return None`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def myfunc():\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "res = myfunc()\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "#### Anonymous / `lambda` functions\n",
    "\n",
    "- no statements\n",
    "- only expressions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Positional and keyword arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def myfunc(pos1, pos2, kw1=\"foo\", kw2=\"bar\"):\n",
    "    print(pos1, pos2, kw1, kw2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# what's the print order?\n",
    "myfunc(1, 2, 3, 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "myfunc(1, 2, kw2=\"random\", kw1=\"order\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "myfunc(5, kw1=\"bla\", pos2=99, kw2=\"dibla\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# explain the errors\n",
    "myfunc()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "myfunc(42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "myfunc(kw1=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Variable scope"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "Code blocks, at any indentation level, are in the *same scope*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 0
   },
   "outputs": [],
   "source": [
    "num = 42\n",
    "\n",
    "for i in range(3):\n",
    "    if i == 0:\n",
    "        assert num == 42\n",
    "        num = 5\n",
    "    assert num == 5\n",
    "\n",
    "assert num == 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Functions have access to variables in the current scope *during execution*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 0
   },
   "outputs": [],
   "source": [
    "num = 42\n",
    "\n",
    "def testfn1():\n",
    "    assert num == 42, f\"{num} is not 42\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "testfn1()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "num = 5\n",
    "testfn1()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "del num"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "testfn1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "When 2 variables have the same name, the variable in the inner most scope is said to *shadow* the outer variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_end_of_cell_marker": 2,
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "num = 42\n",
    "\n",
    "def testfn2():\n",
    "    num = 5\n",
    "    assert num == 5, f\"{num} is not 5\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "testfn2()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "assert num == 5, f\"{num} is not 5\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "*Review errors at this point*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<center><strong>&#9646;&#9646;</strong></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### How to help yourself\n",
    "\n",
    "- Use `help(object|\"syntax token\")`\n",
    "- docstrings\n",
    "- function signatures\n",
    "- Library reference, tutorials, HOWTOs, etc @ [Python.org](https://docs.python.org/3.7/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Problem 1\n",
    "\n",
    "- Read a sequence, and count the number of bases of each type.\n",
    "- Print out the counts as a table, or write it to a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tutorial.io import read_file, fasta_seqs\n",
    "\n",
    "# `next` will work only twice, as the file has only two sequences\n",
    "fasta = read_file(\"data/example.fa\")\n",
    "seq_itr = fasta_seqs(fasta)\n",
    "_, seq1 = next(seq_itr)  # work with seq1, it's a string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq1[:12]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Problem 2\n",
    "\n",
    "- Read a sequence, and identify the sequence of *Codons*.\n",
    "- Print out the position and codon as a table, or write it to a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tutorial.seq import CODON_MAP\n",
    "\n",
    "_, seq2 = next(seq_itr)\n",
    "# CODON_MAP[\"START\"], list of starting seq\n",
    "# CODON_MAP[\"STOP\"], list of all stopping seqs\n",
    "# CODON_MAP[\"REST\"], list of all other seqs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "CODON_MAP[\"STOP\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<center><strong>&#9654;</strong></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# ![Python logo](data/python.ico) More Python concepts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Format strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fmt_float = \"{0} {0:.3f} {0:+.3f} {0: .3f} {0:.3e} {0:.3g}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "fmt_float.format(79 / 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fmt_float.format(-79 / 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "fmt = \"{} {} {}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fmt.format(\"foo\", \"bar\", \"baz\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "fmt = \"{a} {b} {c}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fmt.format(a=\"foo\", c=\"bar\", b=\"baz\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "a, b, c = \"foo bar baz\".split()  # calling: str.split() -> List[str]\n",
    "f\"{a} {b} {c} {c.upper()}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Function calls\n",
    "\n",
    "### Argument unpacking"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def myfunc(pos1, pos2, kw1=\"foo\", kw2=\"bar\"):\n",
    "    print(pos1, pos2, kw1, kw2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y, z = (1, 2, 3)\n",
    "x, y, z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "t1 = (1, 2, \"foo\", \"bar\")\n",
    "myfunc(*t1)  # also works for lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "d1 = {\"kw1\": \"foo\", \"kw2\": \"bar\"}\n",
    "myfunc(*t1[:2], **d1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Arbitrary arguments\n",
    "\n",
    "- positional arguments, followed by keyword arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def myflexiblefn(pos1, *args, kw1=\"foo\", **kwargs):\n",
    "    print(pos1, kw1, args, kwargs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "myflexiblefn(1, 2, 3, kw1=\"foo\", kw2=\"bar\", kw3=\"baz\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# argument order: positional, *args, keyword, **kwargs\n",
    "myflexiblefn(**d1, *t1[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Note:** default arguments should never be mutable objects, e.g. empty containers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fn1(a, b=42):  # a-okay!\n",
    "    pass\n",
    "\n",
    "def fn1(a, b=[]):  # not okay!\n",
    "    pass\n",
    "\n",
    "def fn1(a, b=None):\n",
    "    # use this idiom instead\n",
    "    if b is None:\n",
    "        b = []\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Containers & Iteration\n",
    "\n",
    "### List comprehension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[i for i in range(5)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### with conditionals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[i for i in range(10) if i % 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Sets\n",
    "\n",
    "- elements are unique"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set(\"foo bar baz\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### Set comprehension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "{c for c in \"foo bar baz\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Operations with `set`s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a_, b_, c_ = set(\"foo\"), set(\"bar\"), set(\"baz\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "b_.intersection(c_), a_.isdisjoint(b_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "b_.difference(c_), c_.difference(b_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Dictionaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "week = {\n",
    "    \"mon\": 9,\n",
    "    \"tues\": 1,\n",
    "    \"wed\": 2,\n",
    "    \"thurs\": 3,\n",
    "    \"fri\": 4,\n",
    "    \"sat\": 5,\n",
    "    \"sun\": 6,\n",
    "}\n",
    "week"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "week[\"mon\"] = 0\n",
    "week"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Dictionary comprehension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "{k: (v, v < 5) for k, v in week.items()}  # add weekday boolean flag"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Merging tuples, lists, and dictionaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t1 = (1, 2)\n",
    "t2 = (\"foo\", \"bar\", \"baz\")\n",
    "(42, *t1, *t2)  # same for lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "d0 = {\"a\": 1, \"b\": 2}\n",
    "d1 = {\"c\": 3, \"d\": 4}\n",
    "{**d0, **d1}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# later keys have precedence\n",
    "{**d0, **d1, \"a\": 0}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## I/O\n",
    "\n",
    "- text file formats (read using parsers):\n",
    "  - delimited files: `CSV`, `TSV`\n",
    "  - files with nested data: `JSON`, `XML`, `YAML`\n",
    "  - genomics: `FASTA` (DNA sequence), `VCF` (variant calls)\n",
    "- binary formats (dedicated libraries):\n",
    "  - advanced compression support (smaller files)\n",
    "  - faster (optimised for a common task)\n",
    "  - formats: `HDF5` (hierarchichal), `Parquet` (columnar), `Avro` (row)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "1. open file: read, write, text, binary\n",
    "2. do operations: read, write, seek\n",
    "3. close file (otherwise may get corrupted)\n",
    "4. Standard library: `csv`, `json`, `xml`, `yaml`, `zip`, `bz2`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "txt = open(\"/tmp/afile.txt\", mode=\"w\")\n",
    "txt.write(\"foo\")\n",
    "txt.write(\"bar\")\n",
    "txt.write(\"baz\\n\")\n",
    "txt.write(str(1) + \"\\n\")\n",
    "txt.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! cat /tmp/afile.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "txt = open(\"/tmp/afile.txt\", mode=\"r\")  # open\n",
    "for line in txt.readlines():  # work with it\n",
    "    print(line)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "txt.close()  # close"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# why the extra lines?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# use context managers\n",
    "with open(\"/tmp/afile.txt\", mode=\"w\") as txt:\n",
    "    txt.write(\"foo\")\n",
    "    txt.write(\"bar\")\n",
    "    txt.write(\"baz\\n\")\n",
    "    txt.write(str(42) + \"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! cat /tmp/afile.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Advanced example from tutorial helpers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tutorial.io import read_file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "read_file??"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<center><strong>&#9646;&#9646;</strong></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Problem 1, attempt 2\n",
    "\n",
    "- Read a sequence, and count the number of bases of each type.\n",
    "- Print out the counts as a table, or write it to a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Problem 2, attempt 2\n",
    "\n",
    "- Read a sequence, and identify the sequence of *Codons*.\n",
    "- Print out the position and codon as a table, or write it to a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Problem 3\n",
    "\n",
    "- read `data/summary.txt` as a \"table\", and summarise the data\n",
    "- *Note:* you might need to do some cleaning, to interpret the table as numbers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<center><strong>&#9654;</strong></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# ![Python logo](data/python.ico) Advanced Python concepts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Writing scripts\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### *shebang* line:\n",
    "\n",
    "    #!/usr/bin/python\n",
    "    # script continues ...\n",
    "\n",
    "or more portable:\n",
    "\n",
    "    #!/usr/bin/env python\n",
    "    # script continues ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `import` statements\n",
    "\n",
    "**Syntax:** `import module` or `from module import name [, name]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys  # module name\n",
    "from pathlib import Path  # function or class name\n",
    "from functools import chain, accumulate  # multiple names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Argument parsing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "sys.argv  # all command line arguments (including script name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Problem 1, attempt 3\n",
    "\n",
    "- Read a sequence, and count the number of bases of each type.\n",
    "- Print out the counts as a table, or write it to a CSV file.\n",
    "- Write the solution as a script that takes a fasta sequence file as argument"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "*Tips:* use libraries like,\n",
    "- `argparse` (in the standard library, no installation needed),\n",
    "- `click` (external library, more concise *decorator* based API).\n",
    "\n",
    "*Solutions:* see [simple](prob-1-soln-1.py), [more complete](prob-1-soln-2.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Decorators\n",
    "\n",
    "- Allows you to modify behaviour by \"wrapping\" an existing function\n",
    "- They are functions themselves"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_end_of_cell_marker": 2,
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "def make_bold(fn):\n",
    "    def wrapper(*args, **kwargs):\n",
    "        return \"<b>\" + fn(*args, **kwargs) + \"</b>\"\n",
    "    return wrapper\n",
    "\n",
    "@make_bold\n",
    "def hello1():\n",
    "    return \"Hello World!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hello1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "*How does it work?*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@make_bold\n",
    "def hello1(name=\"World\"):\n",
    "    return f\"Hello {name}!\"\n",
    "\n",
    "def hello2(name=\"World\"):\n",
    "    return f\"Hello {name}!\"\n",
    "hello2 = make_bold(hello2)  # equivalent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hello2(\"Foo\"), hello3(\"Bar\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Reimplement retaining the function name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import wraps\n",
    "\n",
    "def makebold(fn):\n",
    "    @wraps(fn)\n",
    "    def wrapper(*args, **kwargs):\n",
    "        return \"<b>\" + fn(*args, **kwargs) + \"</b>\"\n",
    "    return wrapper\n",
    "\n",
    "@makebold\n",
    "def hello2(name=\"World\"):\n",
    "    return f\"Hello {name}!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hello1.__name__, hello2.__name__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Decorators with parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def format_tag(tag):\n",
    "    def decorator(fn):\n",
    "        @wraps(fn)\n",
    "        def wrapper(*args, **kwargs):\n",
    "            return f\"<{tag}>{fn(*args, **kwargs)}</{tag}>\"\n",
    "        return wrapper\n",
    "    return decorator\n",
    "\n",
    "@format_tag(\"b\")\n",
    "def hello1(name=\"World\"):\n",
    "    return f\"Hello {name}!\"\n",
    "\n",
    "@format_tag(\"i\")\n",
    "def hello2(name=\"World\"):\n",
    "    return f\"Hello {name}!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hello1(\"Felix\"), hello2(\"Phoenix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Generators\n",
    "\n",
    "- functions with lazy evaluation\n",
    "- `yield` -> `return`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def myrange(start, end, step=1):\n",
    "    \"\"\"My range implementation\"\"\"\n",
    "    res = start\n",
    "    while res < end:\n",
    "        yield res\n",
    "        res += step"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[i for i in myrange(0, 10, 3)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Iterators & generator expressions\n",
    "\n",
    "- represents a stream of data, but returns one element at a time\n",
    "- can be used in a `for`-loop like a container\n",
    "- more efficient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "lst_iter = iter([1, 2, 3])\n",
    "a = next(lst_iter)  # get next element\n",
    "b, c = lst_iter     # unpack all elements\n",
    "a, b, c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "lst = [i/2 for i in range(1_000_000)]\n",
    "lst_itr = (i/2 for i in range(1_000_000))\n",
    "sys.getsizeof(lst), sys.getsizeof(lst_itr)  # size in bytes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "sys.getsizeof(lst), sys.getsizeof(list(lst_itr))  # size in bytes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Other built-ins to create iterators\n",
    "\n",
    "- `zip` \"ties\" two iterables\n",
    "- `enumerate` with index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "vowels = (\"a\", \"e\", \"i\", \"o\", \"u\")\n",
    "for vowel, num in zip(range(len(vowels)), vowels):\n",
    "    print(vowel, num)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "for index, vowel in enumerate(vowels):\n",
    "    print(index, vowel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Using `lambda` functions with `map`, `all`, `any`\n",
    "\n",
    "*Note:* `map` returns an iterator\n",
    "\n",
    "- split the strings to a pair of numbers: `[\"1,2\", ..]` -> `[[1, 2], ..]`\n",
    "  - flatten the result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pairs = [\"1,2\", \"3,4\", \"5,6\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "res = list(map(lambda i: list(map(int, i.split(\",\"))), pairs))\n",
    "res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "*Hints for flattening:*\n",
    "- `map` and `list.extend` or nested for loop\n",
    "- `itertools.chain` or `itertools.chain.from_iterable` (returns an iterator)\n",
    "- `functools.reduce`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "*Solutions*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flat = []\n",
    "_ = list(map(flat.extend, res))\n",
    "flat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "\n",
    "list(itertools.chain.from_iterable(res))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "from functools import reduce\n",
    "\n",
    "reduce(lambda i, j: i + j, res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "*Advanced variation of reduce*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from itertools import accumulate\n",
    "from operator import add\n",
    "\n",
    "list(accumulate(res, add))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# play around here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- in a random sequence of characters, find\n",
    "  - if any of them are vowels,\n",
    "  - if all of them are vowels,\n",
    "  - count vowels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from random import choices\n",
    "from string import ascii_lowercase\n",
    "\n",
    "vowels = (\"a\", \"e\", \"i\", \"o\", \"u\")\n",
    "population = choices(ascii_lowercase, k=10)  # `k` random characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "is_vowel = [c in vowels for c in population]\n",
    "population, is_vowel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "any(is_vowel), all(is_vowel), sum(is_vowel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Playground\n",
    "\n",
    "- Play around, try different things\n",
    "- apart from the standard library, you can also try `numpy` and `pandas`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "jupytext": {
   "formats": "ipynb,py:light"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "rise": {
   "autolaunch": true,
   "enable_chalkboard": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}