01_Fundamentals.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modules/packages/libraries\n",
    "\n",
    "Definitions:\n",
    "\n",
    "  * Modules:\n",
    "  A module is a file which contains python functions, global variables etc. It is nothing but .py file which has python executable code / statement.\n",
    "\n",
    "  * Packages:\n",
    "  A package is a namespace which contains multiple package/modules. It is a directory which contains a special file `__init__.py`\n",
    "  \n",
    "  * Libraries:\n",
    "  A library is a collection of various packages. There is no difference between package and python library conceptually.\n",
    "  \n",
    "Modules/packages/libraries can be easily \"imported\" and made functional in your python code. A set of libriaries comes with every python installation. Others can be installed locally and then imported. Your own code sitting somewhere else in your local computer can be imported too.\n",
    "\n",
    "Further details (very important!) on packages and how to create them can be found online. We may find the need of creating our own during the course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "###### all the \"stuff\" that is in the math library can be used\n",
    "import math\n",
    "print(math.pi)\n",
    "\n",
    "# you can give math a label for convenience\n",
    "import math as m\n",
    "print (m.pi)\n",
    "\n",
    "# alternatively you can import only a given \"thing\" from the library\n",
    "from math import pi    #you can add several libraries at once, just list them separated by a \", \"\n",
    "print (pi)\n",
    "\n",
    "# or just get everything (very dangerous!!!)\n",
    "#from math import *\n",
    "#print (sqrt(7))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To know which modules are there for you to use just type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print (help('modules') )\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`pip` is a special package. It is used from the command line to install properly (e.g. matching the version of the local packages) new packages. It can also be used from within python to check i.e. the set installed packages and their versions. N.B.: only the installed packages on top of the default ones will be listed "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\"\"\n",
    "try:\n",
    "    # this doesn't work anymore\n",
    "    import pip\n",
    "    sorted([\"%s==%s\" % (i.key, i.version) for i in pip._internal.utils.misc.get_installed_distributions()])\n",
    "    print (\"within try\")\n",
    "\"\"\"\"\" \n",
    "# use this instead\n",
    "import pkg_resources\n",
    "installed_packages = pkg_resources.working_set\n",
    "installed_packages_list = sorted([\"%s==%s\" % (i.key, i.version)\n",
    "    for i in installed_packages])\n",
    "print(installed_packages_list)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Copies and Views\n",
    "\n",
    "#### Copies:\n",
    "* A **copy** creates a new object that is a duplicate of the original one.\n",
    "* Changes made to the copy do not affect the original object, and vice versa.\n",
    "* There are two types of copies:\n",
    "  * Shallow Copy: Only the top-level object is copied. If the original object contains references to other objects (e.g., a list of lists), the references are copied, not the objects themselves.\n",
    "  * Deep Copy: A complete copy is made, including all nested objects. Changes to any part of the deep copy won’t affect the original object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import copy\n",
    "original = [1, 2, [3, 4]]\n",
    "shallow_copy = copy.copy(original)\n",
    "\n",
    "shallow_copy[0] = 100  # This won't affect the original\n",
    "shallow_copy[2][0] = 300  # This will affect the original's nested list\n",
    "\n",
    "print(\"Original:\", original)  # Output: [1, 2, [300, 4]]\n",
    "print(\"Shallow Copy:\", shallow_copy)  # Output: [100, 2, [300, 4]]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Views:\n",
    "* A view provides a reference to the original object without creating a new object.\n",
    "* Changes made through the view directly affect the original object because the view is simply another way of accessing the original data.\n",
    "* We will see in the next classes that NumPy arrays and Pandas DataFrames often deal with views when you slice or manipulate them.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "arr = np.array([1, 2, 3, 4])\n",
    "view = arr[1:3]  # This creates a view, not a copy\n",
    "\n",
    "view[0] = 100  # Modifies the original array\n",
    "\n",
    "print(\"Original array:\", arr)  # Output: [  1 100   3   4]\n",
    "print(\"View:\", view)  # Output: [100   3]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def square(x):\n",
    "    \"\"\"Square of x.\"\"\"\n",
    "    return x*x\n",
    "\n",
    "def cube(x):\n",
    "    \"\"\"Cube of x.\"\"\"\n",
    "    return x*x*x\n",
    "\n",
    "# create a dictionary of functions\n",
    "funcs = {\n",
    "    'square': square,\n",
    "    'cube': cube,\n",
    "}\n",
    "\n",
    "x = 3\n",
    "print(square(x))\n",
    "print(cube(x))\n",
    "\n",
    "for func in sorted(funcs):\n",
    "    print (func, funcs[func](x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions arguments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Python, whether arguments passed to a function are treated as views or copies depends on the data type of the argument and how it is handled inside the function. Here's how Python handles various types of arguments:\n",
    "\n",
    "**Immutable Types** (e.g., integers, strings, tuples):\n",
    "* For immutable types like `int`, `str`, and `tuple`, when you pass them as arguments to a function, Python creates a copy of the reference to the object, not the object itself.\n",
    "* Since these objects cannot be modified, any attempt to change them inside the function results in the creation of a new object, leaving the original object unchanged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def modify(x):\n",
    "    x += 1\n",
    "    print(\"Inside function:\", x)\n",
    "\n",
    "a = 10\n",
    "modify(a)\n",
    "print(\"Outside function:\", a)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Mutable Types** (e.g., lists, dictionaries, sets):\n",
    "* For mutable types like `list`, `dict`, or `set`, Python passes a reference to the original object. This means that any modification made to the object inside the function will affect the original object outside the function, as they both reference the same object (this behaves like a view)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def modify_list(lst):\n",
    "    lst.append(4)\n",
    "    print(\"Inside function:\", lst)\n",
    "\n",
    "my_list = [1, 2, 3]\n",
    "modify_list(my_list)\n",
    "print(\"Outside function:\", my_list)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Higher order functions\n",
    "\n",
    "A function that uses another function as an input argument or returns a function is known as a higher-order function (HOF). The most familiar examples are `map` and `filter`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### map\n",
    "\n",
    "The map function applies a function to each member of a collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = list(map(square, range(5))) \n",
    "print (x)\n",
    "\n",
    "# Note the difference w.r.t python 2. In python 3 map retuns an iterator so you can do stuff like:\n",
    "for i in map(square,range(5)): print(i)\n",
    "\n",
    "# or\n",
    "[i for i in map(square,range(6))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### filter\n",
    "\n",
    "The filter function applies a predicate to each member of a collection, retaining only those members where the predicate is True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def is_even(x):\n",
    "    return x%2 == 0\n",
    "\n",
    "print (list(filter(is_even, range(5))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Combinations in sequence of HOF are obviously possible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(map(square, filter(is_even, range(5))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### reduce\n",
    "\n",
    "The reduce function reduces a collection using a binary operator to combine items two at a time. More often than not reduce can be substituted with a more efficient for loop. It is worth mentioning it for its key role in big-data applications together with map (the map-reduce paradigm). \n",
    "N.B.: it no loger exist as built-in function in python 3, it is now part of the `functools` library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import reduce\n",
    "\n",
    "def my_add(x, y):\n",
    "    return x + y\n",
    "\n",
    "# another implementation of the sum function\n",
    "reduce(my_add, [1,2,3,4,5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### zip\n",
    "\n",
    "zip is useful when you need to iterate over matched elements of multiple lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xs = [1, 2, 3, 4]\n",
    "ys = [10, 20, 30, 40]\n",
    "zs = ['a', 'b', 'c','d']\n",
    "\n",
    "for x, y, z in zip(xs, ys, zs):\n",
    "    print (x, y, z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Custom HOF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def custom_sum(xs, transform):\n",
    "    \"\"\"Returns the sum of xs after a user specified transform.\"\"\"\n",
    "    return sum(map(transform, xs))\n",
    "\n",
    "xs = range(5)\n",
    "print (custom_sum(xs, square))\n",
    "print (custom_sum(xs, cube))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Returning a function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_logger(target):\n",
    "    def logger(data):\n",
    "        with open(target, 'a') as f:\n",
    "            f.write(data + '\\n')\n",
    "    return logger\n",
    "\n",
    "foo_logger = make_logger('foo.txt') #foo.txt will be created if not there already\n",
    "foo_logger('Hello')\n",
    "foo_logger('World')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! cat 'foo.txt'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Anonimous functions (lambda)\n",
    "\n",
    "When using functional style, there is often the need to create specific functions that perform a limited task as input to a HOF such as map or filter. In such cases, these functions are often written as anonymous or lambda functions. \n",
    "The syntax is as follows:\n",
    "\n",
    "lambda *arguments* : *expression*\n",
    "\n",
    "\n",
    "If you find it hard to understand what a lambda function is doing, it should probably be rewritten as a regular function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "suml = lambda x,y: x+y\n",
    "suml(3,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(lambda x,y: x+y)(3,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in map(lambda x: x*x, range(5)): print (i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# what does this function do?\n",
    "from functools import reduce\n",
    "s1 = reduce(lambda x, y: x+y, map(lambda x: x**2, range(1,10)))\n",
    "print(s1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recursive functions "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fib1(n):\n",
    "    \"\"\"Fib with recursion.\"\"\"\n",
    "\n",
    "    # base case\n",
    "    if n==0 or n==1:\n",
    "        return 1\n",
    "    # recursive case\n",
    "    else:\n",
    "        return fib1(n-1) + fib1(n-2)\n",
    "\n",
    "    \n",
    "print ([fib1(i) for i in range(10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In Python, a more efficient version that does not use recursion is\n",
    "\n",
    "def fib2(n):\n",
    "    \"\"\"Fib without recursion.\"\"\"\n",
    "    a, b = 0, 1\n",
    "    for i in range(1, n+1):\n",
    "        a, b = b, a+b\n",
    "    return b\n",
    "\n",
    "print ([fib2(i) for i in range(10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check indeed the timing:\n",
    "\n",
    "%timeit fib1(20)\n",
    "%timeit fib2(20)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iterators\n",
    "\n",
    "Iterators represent streams of values. Because only one value is consumed at a time, they use very little memory. Use of iterators is very helpful for working with data sets too large to fit into RAM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Iterators can be created from sequences with the built-in function iter()\n",
    "\n",
    "xs = [1,2,3]\n",
    "x_iter = iter(xs)\n",
    "\n",
    "print (next(x_iter))\n",
    "print (next(x_iter))\n",
    "print (next(x_iter))\n",
    "#print (next(x_iter))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Most commonly, iterators are used (automatically) within a for loop\n",
    "# which terminates when it encouters a StopIteration exception\n",
    "\n",
    "x_iter = iter(xs)\n",
    "for x in x_iter:\n",
    "    print (x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More on comprehensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A generator expression\n",
    "\n",
    "print ((x for x in range(10)))\n",
    "\n",
    "# A list comprehesnnion\n",
    "\n",
    "print ([x**2 for x in range(10)])\n",
    "\n",
    "# A set comprehension\n",
    "\n",
    "print ({x for x in range(10)})\n",
    "\n",
    "# A dictionary comprehension\n",
    "\n",
    "print ({x: \"x\" for x in range(10) if x%2==0})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Useful Modules\n",
    "\n",
    "You may want to have a look at the content of the following modules for further usage of (HO) functions:\n",
    "  - [operator](https://docs.python.org/3/library/operator.html)\n",
    "  - [functools](https://docs.python.org/3/library/functools.html)\n",
    "  - [itertools](https://docs.python.org/3/library/itertools.html)\n",
    "  - [toolz](https://pypi.org/project/toolz/)\n",
    "  - [funcy](https://pypi.org/project/funcy/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decorators\n",
    "\n",
    "Decorators are a type of HOF that take a function and return a wrapped function that provides additional useful properties.\n",
    "\n",
    "Examples:\n",
    "\n",
    "  - logging\n",
    "  - profiling\n",
    "  - Just-In-Time (JIT) compilation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_decorator(func):\n",
    "    def wrapper():\n",
    "        print(\"Something is happening before the function is called.\")\n",
    "        func()\n",
    "        print(\"Something is happening after the function is called.\")\n",
    "    return wrapper\n",
    "\n",
    "def say_whee():\n",
    "    print(\"Whee!\")\n",
    "\n",
    "say_whee = my_decorator(say_whee)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "say_whee()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python allows you to use decorators in a simpler way with the @ symbol, sometimes called the “pie” syntax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_decorator(func):\n",
    "    def wrapper():\n",
    "        print(\"Something is happening before the function is called.\")\n",
    "        func()\n",
    "        print(\"Something is happening after the function is called.\")\n",
    "    return wrapper\n",
    "\n",
    "@my_decorator\n",
    "def say_whee():\n",
    "    print(\"Whee!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "say_whee()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### JIT\n",
    "\n",
    "A JIT (Just-In-Time) compiler refers to a technique used to improve the performance of code execution by compiling code into machine code at runtime, rather than interpreting it line by line. This helps speed up program execution by reducing the overhead of repeatedly interpreting code.\n",
    "\n",
    "While the standard Python interpreter, CPython, does not include a JIT compiler, there are alternative Python implementations that provide JIT compilation, such as PyPy or [numba](https://numba.pydata.org).\n",
    "\n",
    "JIT combines both interpretation and compilation. It interprets code initially, and as it identifies sections of code that are executed repeatedly, it compiles them to machine code, optimizing performance dynamically.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numba import jit\n",
    "\n",
    "@jit\n",
    "def fast_sum(arr):\n",
    "    total = 0\n",
    "    for i in arr:\n",
    "        total += i\n",
    "    return total\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fast_sum(list(range(100)))\n",
    "\n",
    "# is it really fast? Try to compare with my_add above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classes and Objects\n",
    "\n",
    "Old school object-oriented programming is possible and often used in python. Classes are defined similarly to standard object-oriented languages, with similar functionalities.\n",
    "\n",
    "The main python doc [page](https://docs.python.org/3/tutorial/classes.html) is worth reading through "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Pet:\n",
    "    # the \"constructor\"\n",
    "    def __init__(self, name, age):  #inizialize the elements of the class\n",
    "        self.name=name\n",
    "        self.age=age\n",
    "    # class functions take the \"self\" parameter !!!\n",
    "    def set_name(self,name):\n",
    "        self.name=name\n",
    "    def convert_age(self,factor):\n",
    "        self.age*=factor\n",
    "\n",
    "buddy=Pet(\"buddy\",4)\n",
    "print (buddy.name, buddy.age)\n",
    "buddy.age=3\n",
    "print (buddy.convert_age(4))\n",
    "print (buddy.age)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ineritance is straightforward\n",
    "class Dog(Pet):\n",
    "    # the following variables is \"global\", i.e. holds for all \"Dog\" objects\n",
    "    species = \"mammal\"\n",
    "    # functions can be redefined as usual\n",
    "    def convert_age(self):\n",
    "        self.age*=7\n",
    "    def set_species(self, species):\n",
    "        self.species = species\n",
    "        \n",
    "puppy=Dog(\"tobia\",10)\n",
    "print(puppy.name)\n",
    "puppy.convert_age()\n",
    "print(puppy.age)\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}