01_Fundamentals.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modules/packages/libraries\n",
    "\n",
    "Definitions:\n",
    "\n",
    "  * Modules:\n",
    "  A module is a file which contains python functions, global variables etc. It is nothing but .py file which has python executable code / statement.\n",
    "\n",
    "  * Packages:\n",
    "  A package is a namespace which contains multiple package/modules. It is a directory which contains a special file `__init__.py`\n",
    "  \n",
    "  * Libraries:\n",
    "  A library is a collection of various packages. There is no difference between package and python library conceptually.\n",
    "  \n",
    "Modules/packages/libraries can be easily \"imported\" and made functional in your python code. A set of libriaries comes with every python installation. Others can be installed locally and then imported. Your own code sitting somewhere else in your local computer can be imported too.\n",
    "\n",
    "Further details (very important!) on packages and how to create them can be found online. We may find the need of creating our own during the course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "###### all the \"stuff\" that is in the math library can be used\n",
    "import math\n",
    "print(math.pi)\n",
    "\n",
    "# you can give math a label for convenience\n",
    "import math as m\n",
    "print (m.pi)\n",
    "\n",
    "# alternatively you can import only a given \"thing\" from the library\n",
    "from math import pi    #you can add several libraries at once, just list them separated by a \", \"\n",
    "print (pi)\n",
    "\n",
    "# or just get everything (very dangerous!!!)\n",
    "#from math import *\n",
    "#print (sqrt(7))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To know which modules are there for you to use just type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print (help('modules') )\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`pip` is a special package. It is used from the command line to install properly (e.g. matching the version of the local packages) new packages. It can also be used from within python to check i.e. the set installed packages and their versions. N.B.: only the installed packages on top of the default ones will be listed "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    # this doesn't work anymore\n",
    "    import pip\n",
    "    sorted([\"%s==%s\" % (i.key, i.version) for i in pip._internal.utils.misc.get_installed_distributions()])\n",
    "\n",
    "except:\n",
    "    # use this instead\n",
    "    import pkg_resources\n",
    "    installed_packages = pkg_resources.working_set\n",
    "    installed_packages_list = sorted([\"%s==%s\" % (i.key, i.version)\n",
    "       for i in installed_packages])\n",
    "    print(installed_packages_list)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def square(x):\n",
    "    \"\"\"Square of x.\"\"\"\n",
    "    return x*x\n",
    "\n",
    "def cube(x):\n",
    "    \"\"\"Cube of x.\"\"\"\n",
    "    return x*x*x\n",
    "\n",
    "# create a dictionary of functions\n",
    "funcs = {\n",
    "    'square': square,\n",
    "    'cube': cube,\n",
    "}\n",
    "\n",
    "x = 3\n",
    "print(square(x))\n",
    "print(cube(x))\n",
    "\n",
    "for func in sorted(funcs):\n",
    "    print (func, funcs[func](x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions arguments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is passsed to a function is a copy of the input. Imagine we have a list *x =[1, 2, 3]*, i.e. a mutable object. If within the function the content of *x* is directly changed (e.g. *x[0] = 999*), then *x* changes outside the funciton as well. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def modify(x):\n",
    "    x[0] = 999\n",
    "    return x\n",
    "\n",
    "x = [1,2,3]\n",
    "print (x)\n",
    "print (modify(x))\n",
    "print (x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, if *x* is reassigned within the function to a new object (e.g. another list), then the copy of the name *x* now points to the new object, but *x* outside the function is unhcanged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def no_modify(x):\n",
    "    x = [4,5,6]\n",
    "    return x\n",
    "\n",
    "x = [1,2,3]\n",
    "print (x)\n",
    "print (no_modify(x))\n",
    "print (x)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if the function tries to modify the value of an immutable object?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Binding of default arguments occurs at function definition:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x = []):\n",
    "    x.append(1)\n",
    "    return x\n",
    "\n",
    "print (f())\n",
    "print (f())\n",
    "print (f(x = [9,9,9]))\n",
    "print (f())\n",
    "print (f())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try to aviod that!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x = None):\n",
    "    if x is None:\n",
    "        x = []\n",
    "    x.append(1)\n",
    "    return x\n",
    "\n",
    "print (f())\n",
    "print (f())\n",
    "print (f(x = [9,9,9]))\n",
    "print (f())\n",
    "print (f())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Higher order functions\n",
    "\n",
    "A function that uses another function as an input argument or returns a function is known as a higher-order function (HOF). The most familiar examples are `map` and `filter`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### map\n",
    "\n",
    "The map function applies a function to each member of a collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = list(map(square, range(5))) \n",
    "print (x)\n",
    "\n",
    "# Note the difference w.r.t python 2. In python 3 map retuns an iterator so you can do stuff like:\n",
    "for i in map(square,range(5)): print(i)\n",
    "\n",
    "# or\n",
    "[i for i in map(square,range(6))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### filter\n",
    "\n",
    "The filter function applies a predicate to each member of a collection, retaining only those members where the predicate is True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def is_even(x):\n",
    "    return x%2 == 0\n",
    "\n",
    "print (list(filter(is_even, range(5))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Combinations in sequence of HOF are obviously possible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(map(square, filter(is_even, range(5))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### reduce\n",
    "\n",
    "The reduce function reduces a collection using a binary operator to combine items two at a time. More often than not reduce can be substituted with a more efficient for loop. It is worth mentioning it for its key role in big-data applications together with map (the map-reduce paradigm). \n",
    "N.B.: it no loger exist as built-in function in python 3, it is now part of the `functools` library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import reduce\n",
    "\n",
    "def my_add(x, y):\n",
    "    return x + y\n",
    "\n",
    "# another implementation of the sum function\n",
    "reduce(my_add, [1,2,3,4,5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### zip\n",
    "\n",
    "zip is useful when you need to iterate over matched elements of multiple lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xs = [1, 2, 3, 4]\n",
    "ys = [10, 20, 30, 40]\n",
    "zs = ['a', 'b', 'c',]\n",
    "\n",
    "for x, y, z in zip(xs, ys, zs):\n",
    "    print (x, y, z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Custom HOF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def custom_sum(xs, transform):\n",
    "    \"\"\"Returns the sum of xs after a user specified transform.\"\"\"\n",
    "    return sum(map(transform, xs))\n",
    "\n",
    "xs = range(5)\n",
    "print (custom_sum(xs, square))\n",
    "print (custom_sum(xs, cube))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Returning a function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_logger(target):\n",
    "    def logger(data):\n",
    "        with open(target, 'a') as f:\n",
    "            f.write(data + '\\n')\n",
    "    return logger\n",
    "\n",
    "foo_logger = make_logger('foo.txt') #foo.txt will be created if not there already\n",
    "foo_logger('Hello')\n",
    "foo_logger('World')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! cat 'foo.txt'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Anonimous functions (lambda)\n",
    "\n",
    "When using functional style, there is often the need to create specific functions that perform a limited task as input to a HOF such as map or filter. In such cases, these functions are often written as anonymous or lambda functions. \n",
    "The syntax is as follows:\n",
    "\n",
    "lambda *arguments* : *expression*\n",
    "\n",
    "\n",
    "If you find it hard to understand what a lambda function is doing, it should probably be rewritten as a regular function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "suml = lambda x,y: x+y\n",
    "suml(3,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(lambda x,y: x+y)(3,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in map(lambda x: x*x, range(5)): print (i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# what does this function do?\n",
    "from functools import reduce\n",
    "s1 = reduce(lambda x, y: x+y, map(lambda x: x**2, range(1,10)))\n",
    "print(s1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recursive functions "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fib1(n):\n",
    "    \"\"\"Fib with recursion.\"\"\"\n",
    "\n",
    "    # base case\n",
    "    if n==0 or n==1:\n",
    "        return 1\n",
    "    # recursive case\n",
    "    else:\n",
    "        return fib1(n-1) + fib1(n-2)\n",
    "\n",
    "    \n",
    "print ([fib1(i) for i in range(10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In Python, a more efficient version that does not use recursion is\n",
    "\n",
    "def fib2(n):\n",
    "    \"\"\"Fib without recursion.\"\"\"\n",
    "    a, b = 0, 1\n",
    "    for i in range(1, n+1):\n",
    "        a, b = b, a+b\n",
    "    return b\n",
    "\n",
    "print ([fib2(i) for i in range(10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check indeed the timing:\n",
    "\n",
    "%timeit fib1(20)\n",
    "%timeit fib2(20)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iterators\n",
    "\n",
    "Iterators represent streams of values. Because only one value is consumed at a time, they use very little memory. Use of iterators is very helpful for working with data sets too large to fit into RAM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Iterators can be created from sequences with the built-in function iter()\n",
    "\n",
    "xs = [1,2,3]\n",
    "x_iter = iter(xs)\n",
    "\n",
    "print (next(x_iter))\n",
    "print (next(x_iter))\n",
    "print (next(x_iter))\n",
    "print (next(x_iter))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Most commonly, iterators are used (automatically) within a for loop\n",
    "# which terminates when it encouters a StopIteration exception\n",
    "\n",
    "x_iter = iter(xs)\n",
    "for x in x_iter:\n",
    "    print (x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More on comprehensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A generator expression\n",
    "\n",
    "print ((x for x in range(10)))\n",
    "\n",
    "# A list comprehesnnion\n",
    "\n",
    "print ([x**2 for x in range(10)])\n",
    "\n",
    "# A set comprehension\n",
    "\n",
    "print ({x for x in range(10)})\n",
    "\n",
    "# A dictionary comprehension\n",
    "\n",
    "print ({x: \"x\" for x in range(10)})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Useful Modules\n",
    "\n",
    "You may want to have a look at the content of the following modules for further usage of (HO) functions:\n",
    "  - [operator](https://docs.python.org/3/library/operator.html)\n",
    "  - [functools](https://docs.python.org/3/library/functools.html)\n",
    "  - [itertools](https://docs.python.org/3/library/itertools.html)\n",
    "  - [toolz](https://pypi.org/project/toolz/)\n",
    "  - [funcy](https://pypi.org/project/funcy/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decorators\n",
    "\n",
    "Decorators are a type of HOF that take a function and return a wrapped function that provides additional useful properties.\n",
    "\n",
    "Examples:\n",
    "\n",
    "  - logging\n",
    "  - profiling\n",
    "  - Just-In-Time (JIT) compilation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_decorator(func):\n",
    "    def wrapper():\n",
    "        print(\"Something is happening before the function is called.\")\n",
    "        func()\n",
    "        print(\"Something is happening after the function is called.\")\n",
    "    return wrapper\n",
    "\n",
    "def say_whee():\n",
    "    print(\"Whee!\")\n",
    "\n",
    "say_whee = my_decorator(say_whee)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "say_whee()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python allows you to use decorators in a simpler way with the @ symbol, sometimes called the “pie” syntax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_decorator(func):\n",
    "    def wrapper():\n",
    "        print(\"Something is happening before the function is called.\")\n",
    "        func()\n",
    "        print(\"Something is happening after the function is called.\")\n",
    "    return wrapper\n",
    "\n",
    "@my_decorator\n",
    "def say_whee():\n",
    "    print(\"Whee!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "say_whee()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classes and Objects\n",
    "\n",
    "Old school object-oriented programming is possible and often used in python. Classes are defined similarly to standard object-oriented languages, with similar functionalities.\n",
    "\n",
    "The main python doc [page](https://docs.python.org/3/tutorial/classes.html) is worth reading through "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Pet:\n",
    "    # the \"constructor\"\n",
    "    def __init__(self, name, age):  #inizialize the elements of the class\n",
    "        self.name=name\n",
    "        self.age=age\n",
    "    # class functions take the \"self\" parameter !!!\n",
    "    def set_name(self,name):\n",
    "        self.name=name\n",
    "    def convert_age(self,factor):\n",
    "        self.age*=factor\n",
    "\n",
    "buddy=Pet(\"buddy\",12)\n",
    "print (buddy.name, buddy.age)\n",
    "buddy.age=3\n",
    "print (buddy.age)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ineritance is straightforward\n",
    "class Dog(Pet):\n",
    "    # the following variables is \"global\", i.e. holds for all \"Dog\" objects\n",
    "    species = \"mammal\"\n",
    "    # functions can be redefined as usual\n",
    "    def convert_age(self):\n",
    "        self.age*=7\n",
    "    def set_species(self, species):\n",
    "        self.species = species\n",
    "        \n",
    "puppy=Dog(\"tobia\",10)\n",
    "print(puppy.name)\n",
    "puppy.convert_age()\n",
    "print(puppy.age)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}