Floating_Points.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import bitstring"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 引言 : 浮點數奇怪的精度\n",
    "> [浮點數練習網站](https://float.exposed/0x7effffff)\n",
    "* 0.1 + 0.7 != 0.8\n",
    "* 0.1 + 0.2 != 0.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.1 + 0.7 = 0.7999999999999999\n",
      "0.1 + 0.2 = 0.30000000000000004\n"
     ]
    }
   ],
   "source": [
    "print( f\"0.1 + 0.7 = {0.1 + 0.7}\")\n",
    "print( f\"0.1 + 0.2 = {0.1 + 0.2}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 1 : IEEE 754 32bits format\n",
    "```\n",
    "[ sign (1 bit) ][ exponent (8 bits) ] [ fraction ( 23 bits ) ]\n",
    "exponent have bias +127\n",
    "Ex: 2^4  ---> exponent = binary(127+4)\n",
    "```\n",
    "以 `0.1` 為例 <br>\n",
    "$0.1_{10}$ <br> \n",
    "= $ 0.00010011001100110011001101_{2} $  \n",
    "= $ 1.0011001100110011001101 * 2^{-4}$ <br>\n",
    "= ( bit pattern ) 0 01111011 10011001100110011001101 <br>\n",
    "= 0  (-4 + 127) ( 1.600000023841858 )\n",
    "       \n",
    "$1.600000023841858 = 1 + 10011001100110011001101 (0.600000023841858)$\n",
    "\n",
    "不用存最前面的 1 可以再多存一位精確度 <br>\n",
    "實際上存的是 $ 0.100000001490116119384765625$ 而非精確的 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 01111111 00000000000000000000000\n",
      "exp: 01111111, cuz +127 bias, Why >1 need to add bias? \n"
     ]
    }
   ],
   "source": [
    "# 0.1 Floating points\n",
    "f1 = bitstring.BitArray(float=1.0, length=32)\n",
    "\n",
    "sign = f1.bin[0]   # 1 bit\n",
    "exp  = f1.bin[1:9] # 8 bits\n",
    "frac = f1.bin[9:]  # 23 bits\n",
    "\n",
    "print(sign, exp, frac)\n",
    "print(f\"exp: {exp}, cuz +127 bias, Why >1 need to add bias? \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fraction :  1.600000023841858\n",
      "Value actually stored in float  :  0.10000000149011612\n"
     ]
    }
   ],
   "source": [
    "# 計算 0.1 的二進位\n",
    "frac = \"10011001100110011001101\"\n",
    "value = 0\n",
    "start = 1 # 從 1/2 ^ 1 開始\n",
    "for bit in frac:\n",
    "        value += int(bit) * 0.5 ** start\n",
    "        start += 1\n",
    "print( f\"fraction :  { value + 1 }\")\n",
    "print( f\"Value actually stored in float  :  { (value + 1) * 2 ** ( 123 - 127) }\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Qusetion 1 : 為什麼要加 bias ? \n",
    "Answer : \n",
    "\n",
    "> [Wiki](https://zh.wikipedia.org/wiki/IEEE_754) <br>\n",
    ">採用指數的實際值加上固定的偏移值的辦法表示浮點數的指數，好處是可以用長度為{\\displaystyle e}e個位元的無符號整數來表示所有的指數取值，這使得兩個浮點數的指數大小的比較更為容易，實際上可以按照字典次序比較兩個浮點表示的大小。\n",
    "\n",
    "試想原本沒有加 127, 如果指數有正有負，如果當成 unsigned int 來比較指數位 8 bits, 負的首位元是 1 一定比正的大，我們想要floating points 比較時直接當作 unsigned int 比較，因此原本 -126 ~ +127 (為什麼是 -126 稍後闡述 )，變成 1 (最小的正規數) ~ 254(最大的正規數)\n",
    "\n",
    "\n",
    "問題 : 為何最小指數次方 -126 ? <br>\n",
    "答案 : 因為全部指數 8 bits 都是 0， 代表非正規化數，所以 exponent 應從 00000001 開始，扣掉 127 = -126"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 2 浮點數轉換"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Float number: 0.1   0 01111011 10011001100110011001101 Actually stored Value = 0.10000000149011612\n",
      "Float number: 0.2   0 01111100 10011001100110011001101 Actually stored Value = 0.20000000298023224\n",
      "Float number: 0.3   0 01111101 00110011001100110011010 Actually stored Value = 0.30000001192092896\n"
     ]
    }
   ],
   "source": [
    "def float_2_IEEE754(float_num):\n",
    "        bit_pattern = bitstring.BitArray(float=float_num, length=32)\n",
    "        sign = bit_pattern.bin[0]   # 1 bit\n",
    "        exp  = bit_pattern.bin[1:9] # 8 bits\n",
    "        frac = bit_pattern.bin[9:]  # 23 bits\n",
    "        print(f\"Float number: {float_num}  \", sign, exp, frac, \n",
    "              f\"Actually stored Value = {bit_pattern.float}\")\n",
    "\n",
    "float_2_IEEE754(0.1)\n",
    "float_2_IEEE754(0.2)\n",
    "float_2_IEEE754(0.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.30000001192092896"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def bit_2_float(bit_string):\n",
    "        sign = bit_string[0]   # 1 bit\n",
    "        exp  = bit_string[1:9] # 8 bits\n",
    "        frac = bit_string[9:]  # 23 bits\n",
    "\n",
    "        power = int(exp, 2) - 127\n",
    "        value = 0\n",
    "        start = 1 # 從 1/2 ^ 1 開始\n",
    "        for bit in frac:\n",
    "                value += int(bit) * 0.5 ** start\n",
    "                start += 1\n",
    "        \n",
    "        return ( 1 + value) * 2 ** power\n",
    "\n",
    "test_bitstr = \"00111110100110011001100110011010\" # 0.3\n",
    "bit_2_float(test_bitstr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Float number: 0.0   0 00000000 00000000000000000000000 Actually stored Value = 0.0\n",
      "Float number: -0.0   1 00000000 00000000000000000000000 Actually stored Value = -0.0\n"
     ]
    }
   ],
   "source": [
    "# +0, -0\n",
    "float_2_IEEE754(+0.0)\n",
    "float_2_IEEE754(-0.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Float number: 1.0   0 01111111 00000000000000000000000 Actually stored Value = 1.0\n",
      "Float number: -1.0   1 01111111 00000000000000000000000 Actually stored Value = -1.0\n"
     ]
    }
   ],
   "source": [
    "float_2_IEEE754(+1.0) # 1.0 * 2 ** 0  (127 - 127)\n",
    "float_2_IEEE754(-1.0) # -1.0 * 2 ** 0 (127 - 127)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Float number: nan   0 11111111 10000000000000000000000 Actually stored Value = nan\n",
      "Float number: inf   0 11111111 00000000000000000000000 Actually stored Value = inf\n"
     ]
    }
   ],
   "source": [
    "# not a number (nan) : 指數全為 1 而 分數不等於 0\n",
    "float_2_IEEE754(np.nan)\n",
    "# nan != nan\n",
    "\n",
    "\n",
    "# infinite (inf) : 指數全為 1 而 分數等於 0\n",
    "float_2_IEEE754(np.inf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 3 : 浮點數的安全範圍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Float number: 123456789   0 10011001 11010110111100110100011 Actually stored Value = 123456792.0\n"
     ]
    }
   ],
   "source": [
    "float_2_IEEE754(123456789)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在指定 32 bits 浮點數為 123456789 時，可看到實際儲存不是123456789，而是123456792"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "111010110111100110100010101 , length is 27\n"
     ]
    }
   ],
   "source": [
    "# 123456789 --> 轉成 binary\n",
    "test_str = bin(123456789)[2:] # remove 0b prefix\n",
    "print(test_str, \", length is\", len(test_str))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$111010110111100110100010101$ <br>\n",
    "= $1.11010110111100110100010101 * 2^{26} $ <br>\n",
    "\n",
    "因為後面有 26 bits，但是 32bits floating points 的 fraction 只有 23 bits，所以要 **向上捨入** 3 bits <br>\n",
    "= $1.11010110111100110100011 (捨入進位) * 2^{26} $ <br>\n",
    "\n",
    "接著，將此字串依照 IEEE 754 格式 <br>\n",
    "exp  = 26 + 127 = 153 ( 10011001 ) <br>\n",
    "frac = 7043491 ( 11010110111100110100010 ) <br>\n",
    "\n",
    "123456789.0 = 0 10011001 11010110111100110100011 \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "123456792.0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bit_2_float(\"01001100111010110111100110100011\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，由於向上捨入 3 bits，造成一些精度被丟失，所以無法正確儲存 123456789 <br>\n",
    "因此，我們可以知道，\n",
    "### 浮點數其實是 **離散** 的，在一定範圍內是連續的，範圍外是離散的\n",
    "\n",
    "Question : 範圍是 ? <br>\n",
    "Ans : 只要能在 23 bits 內表示的話 ( fraction 23 bits )，那就不會被捨入而丟失精度，因此，單精度安全範圍內最大整數是後面 23 bits 全 1 ( 8388607 ), $ 1.11111111111111111111111*2^{23} = 16777215$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8388607"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "int('1' * 23, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16777215"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "int('1' * 24, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16777212 : 16777212.0\n",
      "16777213 : 16777213.0\n",
      "16777214 : 16777214.0\n",
      "16777215 : 16777215.0\n",
      "16777216 : 16777216.0\n",
      "16777217 : 16777216.0\n",
      "16777218 : 16777218.0\n",
      "16777219 : 16777220.0\n",
      "16777220 : 16777220.0\n",
      "16777221 : 16777220.0\n"
     ]
    }
   ],
   "source": [
    "for i in range(10):\n",
    "        value = 16777212 + i\n",
    "        bit_pattern = bitstring.BitArray(float=value, length=32)\n",
    "        store_value = bit_2_float(bit_pattern.bin)\n",
    "        print(value, \":\", store_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到只要 > 16777215 後就開始不連續了，逐漸發散"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " # Chapter  4 : 非正規浮點數\n",
    " > * 如果浮點數的指數部分的 **編碼值是0，分數部分非零**，那麼這個浮點數將被稱為非正規形式的浮點數\n",
    " > * 一般是某個數字相當接近零時才會使用非規約型式來表示。 \n",
    " > * IEEE 754標準規定：非正規形式的浮點數的指數偏移值比正規形式的浮點數的指數偏移值小1。\n",
    " > * 漸進式下溢位（gradual underflow）\n",
    " > * 用來解決填補絕對值意義下最小規格數與零的距離。（舉例說，正數下，最大的非規格數等於最小的規格數。而一個浮點數編碼中，如果exponent=0，且尾數部分不為零，那麼就按照非規約浮點數來解析）\n",
    " > * [Stack Overflows](https://stackoverflow.com/questions/46732247/ieee-754-32-bit-single-precision-exponent-126-instead-of-127)\n",
    " > * [IEEE 754 標準](http://ieee-754.blogspot.com/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[揭密浮點數](https://zhuanlan.zhihu.com/p/89320102) 一文所述\n",
    "定義非正規化浮點數的目的 : 是希望最小正規數和 0 之間 **更平滑**，因此可以儲存更多的很小的數"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Float number: 1.1754943508222875e-38   0 00000001 00000000000000000000000 Actually stored Value = 1.1754943508222875e-38\n",
      "Float number: 3.4028234663852886e+38   0 11111110 11111111111111111111111 Actually stored Value = 3.4028234663852886e+38\n"
     ]
    }
   ],
   "source": [
    "# 由上文可知，正規浮點數最小的值為 1.0 * 2^-126\n",
    "# min normalize number\n",
    "float_2_IEEE754(2**-126)\n",
    "\n",
    "# max normalize number\n",
    "max_normal_float = bit_2_float(\"01111111011111111111111111111111\")\n",
    "float_2_IEEE754(max_normal_float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "小於 1.1754943508222875e-3 underflow, 大於 3.4028234663852886e+38 為 overflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[IEEE 754 single-precision](https://www.quora.com/Why-is-2-149-the-smallest-normalized-gap-in-the-IEEE-754-single-precision-format)\n",
    "\n",
    "* 最小的正號正規浮點數 : E = 1, M = 0 ---> 2 ^ -126\n",
    "* 次小的正號正規浮點數 : E = 1, M = 1 ---> 2 ^ -126 * ( 1 + 2 ^ -23 ) 大約等於 2 ^ -126 + 2 ^ -149\n",
    "  和原本相差 2^ -149\n",
    "\n",
    "```\n",
    "   0                          2 ^ -126         2 ^ -126 + 2 ^ -149\n",
    "   |------------------------------|---------------------|-------\n",
    "   a           2^-126             b       2^-149        c\n",
    "    \n",
    "    ab\n",
    "   ----- = 2 ^ -23  --> 非常急促的降到 0 ， 有沒有更優雅的作法 ? <Ans> 漸進式下溢位\n",
    "    bc\n",
    "\n",
    "```\n",
    "\n",
    "最小非正規化 : 0 00000000 00000000000000000000001 ( 2^-149 )\n",
    "最大非正規化 : 0 00000000 11111111111111111111111 ( (1 - 2^-23) * 2 ^-126 )\n",
    "\n",
    "```\n",
    "    最小正規化   :  1.0000 0000 0000 0000 0000 000 (1      )  * 2 ^ -126\n",
    "-)  最大非正規化 :  0.0000 0000 0000 0000 0000 001 (2 ^ -23)  * 2 ^ -126\n",
    "---------------------------------------------------------------------------------\n",
    "    最大非正規化 :  0.1111 1111 1111 1111 1111 111 (1-2^-23)  * 2 ^ -126\n",
    "\n",
    "(1 - 2 ^-23 ) * 2^ -126 = 2^-126 - 2^-149 因此最小正規化數跟前後距離變得一樣了   2^-149 ~~ ( 漸進式下降 )\n",
    "\n",
    "而 0 和 最小非正規化浮點數 2 ^-149 也一樣差了 2 ^ -149\n",
    "```\n",
    "\n",
    "**有了非正規化浮點數 再也不會驟降至0了。又可以儲存更多更小的小數**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " # Chapter 5 : 浮點數的密度與分布"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![img](https://pic2.zhimg.com/80/v2-75cebc365c88cd9b8862e04ad3c52601_720w.png)\n",
    "![img](https://pic2.zhimg.com/v2-96e6fafa938a2578e22496b8daf82691_r.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$F=(-1)^{S} * 2^{(E-127)}* (1 + \\frac{M}{2^{23}})$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD7CAYAAABpJS8eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAZsklEQVR4nO3de3DU5aH/8c93k4BcT4DuZoXD5dRYgijFylDgj9A4nCSQpMGIgmLDqRqVqUIZBxuvjAoVW9qoYJlO2ylHB0fC7/xKpApaOULHQqWkKihgRQmaCJvNxYaEQC77nD/SLCy5bCALSfZ5v/7pfJ/nm+8+n7188uVhsY4xxggAEPVcPb0AAMDlQeEDgCUofACwBIUPAJag8AHAEhQ+AFiCwgcAS8T29AI6U11dp0AgMv9MYMSIwaqsrI3ItXpKNGSQoiMHGXoHMoRyuRwNGzaow/leXfiBgIlY4bder6+LhgxSdOQgQ+9Ahq5jSwcALEHhA4AlKHwAsASFDwCWoPABwBIUPgBYgsIHAEtQ+ABgCQofACxB4QOAJSh8ALAEhQ8AlqDwAcASFD4AWILCBwBLUPgAYAkKHwAsQeEDgCUofACwBIUPAJag8AHAEhQ+AFiCwgcAS3Sp8Gtra5WZmanS0tI2c4cOHVJOTo7S0tL06KOPqqmpKWT+4MGDuvbaayOzWgDARQtb+B9++KFuu+02lZSUtDu/fPlyPfHEE3rzzTdljFFhYWFwrr6+Xk8//bQaGxsjtmAAwMUJW/iFhYVasWKFPB5Pm7mysjKdPn1akydPliTl5ORo+/btwfnVq1dr0aJFEVwuAOBixYY7YdWqVR3OlZeXy+12B4/dbrd8Pp8kaceOHTp9+rTS09MjsEwAQHeFLfzOBAIBOY4TPDbGyHEc+f1+rV+/Xhs2bOjW4kaMGNytnz+f2z0kotfrCdGQQYqOHGToHcjQdd0qfK/XK7/fHzyuqKiQx+PRzp079fXXX2vhwoXBuezsbG3cuFGDB3e9xCsraxUImO4sMcjtHiK//2RErtVToiGDFB05yNA7kCGUy+V0eqPcrcIfNWqU+vfvr+LiYt1www0qKipScnKybrnlFt1yyy3B88aPH6+ioqLuPBQAoJsu6nv4eXl5OnDggCRpzZo1euaZZ5Senq5Tp04pNzc3ogsEAESGY4yJzJ7JJcCWTqhoyCBFRw4y9A5kCBVuS4d/aQsAlqDwAcASFD4AWILCBwBLUPgAYAkKHwAsQeEDgCUofACwBIUPAJag8AHAEhQ+AFiCwgcAS1D4AGAJCh8ALEHhA4AlKHwAsASFDwCWoPABwBIUPgBYgsIHAEtQ+ABgCQofACxB4QOAJSh8ALAEhQ8AlqDwAcASFD4AWILCBwBLUPgAYAkKHwAs0aXCr62tVWZmpkpLS9vMHTp0SDk5OUpLS9Ojjz6qpqYmSVJxcbHmzZun7OxsLVq0SGVlZZFdOQDggoQt/A8//FC33XabSkpK2p1fvny5nnjiCb355psyxqiwsDA4vnLlShUVFSkrK0srV66M6MIBABcmbOEXFhZqxYoV8ng8bebKysp0+vRpTZ48WZKUk5Oj7du3q6GhQUuXLlVSUpIkafz48Tp+/HiElw4AuBCx4U5YtWpVh3Pl5eVyu93BY7fbLZ/Pp379+ik7O1uSFAgEtG7dOs2aNSsCywUAXKywhd+ZQCAgx3GCx8aYkOOGhgbl5+erqalJ99577wVff8SIwd1ZXhtu95CIXq8nREMGKTpykKF3IEPXdavwvV6v/H5/8LiioiK49VNXV6fFixcrPj5e69evV1xc3AVfv7KyVoGA6c4Sg9zuIfL7T0bkWj0lGjJI0ZGDDL0DGUK5XE6nN8rd+lrmqFGj1L9/fxUXF0uSioqKlJycLKnlL23Hjh2r5557Tv369evOwwAAIuCiCj8vL08HDhyQJK1Zs0bPPPOM0tPTderUKeXm5urgwYPasWOH/v73v+umm25Sdna28vLyIrpwAMCFcYwxkdkzuQTY0gkVDRmk6MhBht6BDKEu6ZYOAKDvoPABwBIUPgBYgsIHAEtQ+ABgCQofACxB4QOAJSh8ALAEhQ8AlqDwAcASFD4AWILCBwBLUPgAYAkKHwAsQeEDgCUofACwBIUPAJag8AHAEhQ+AFiCwgcAS1D4AGAJCh8ALEHhA4AlKHwAsASFDwCWoPABwBIUPgBYgsIHAEtQ+ABgCQofACxB4QOAJbpU+LW1tcrMzFRpaWmbuUOHDiknJ0dpaWl69NFH1dTUJEn66quvtHDhQqWnp2vx4sWqq6uL7MoBABckNtwJH374oR577DGVlJS0O798+XKtXLlSkydP1iOPPKLCwkLdfvvtevLJJ3X77bcrIyNDL774on71q19p+fLlkV5/0La/HtO4K4dqwthhwbH/3nZYRtJ/zU7qs2Pb/npMLpejQMAoN+vaiF6/9Tnbe9AnIylh2IDgY5VX18tIcqTg+a0/292xKwbEaegVsR0+VsKwAfrky6918lSDhgzsp5OnGjR1QoL2HvKp/kxzMMu3RsfLkfTJl193OFZb3yBJ8gwbqPLqU2puNoqNdan+TJPiYlrud043nr2mMWfXEQmt14qNcRQX41JDU0D9Ys/eZ135jUEqrz4lSeoXG6OGpmbd8C2PjKR/nJNr/Oj4NmMD+seEPC+eYQN08lSDxniGaOo1Cfp/O49ojGdI8LldNDtJG7YdDq5pwBVxmp9yVchYe+c5kjznvDd81fVhz+9o7NCxapUcrwleY9E5783W8y/n2Lmfr9nTxvbYOjoaa32+WtfWXWELv7CwUCtWrNBDDz3UZq6srEynT5/W5MmTJUk5OTl64YUXdMstt+hvf/ubXnzxxeD4HXfccUkLf9yVQ7V+y0daPPdaTRg7TIeOVWvv4XIZY/TdaxLkdg9pM9beeb1tzOVytOl/j2j+jYmSFNHrj7tyqNb+z34ZY+Q4jqZNTNDO97/S964fqb2Hy9XUHJDLkRzH0ZUjBkZsLCbGpakTPB0+1rSJCdr/WaVcLikQkFwu6ejxk4pxOWoOnK1i/9f1crkcNTYFOh2TpNr6mrMHDS0F39TcrPZEquzPvVZTswk+Xn3D2cf9/Ktz1qWWPx3v/viEAgETkrWy5nSbsRiXo6PHTwafpxNVpxQb4+irijrt/viEHEf6qqJOASPFxrjkHTFQew/5ZIxRwEhxsS7FD4oLGWvvvNgYl757zTmvVxfO72js9T3HlDF9rPYe8slxHE099715EWPBz/VFXq/dz1eE1haJDK2dFilhC3/VqlUdzpWXl8vtdgeP3W63fD6fqqurNXjwYMXGxoaMX0oTxg7T4rnXav2Wj5Ry/Si9836Z7s+5TpK0fstH+qKiTq+/ezRkrL3zeuPY/BsT9fqeY1KMK+IZHMeRJF1/tVs73/9K14wbpp3vf6XpExP0wZFKGWN0/dVubfrfIxEb++5Er94pLu3wsVrXcbCkOlhmktQcMCG/BJr/VYDhxgKh3d+ruVwK/rI6N0N7Y63lf37OpuaW52D6RK/e/9SvWJejyYkj/vU6nB2bOtHbZqy98yYnjjjvvRH+/M7GXt9zTA/cPElS6HvzYsZaP9fduV7r5+vU6aaLXselytB6AxspYQu/M4FAIFgYkoJ3iq3/e67zj7tixIjBF3S+2z1EX1TUadOf/qH5//ktJU8ZI0l9fuyO9AlSjOuSXV+SNv3pH7rmP4br4NEqXfMfw7XnY1+buUiPdfZYrXMHj1aFvMaBgNqMd3Wst+tOrvPHW57XE+28Dhc/dvb16v51bfp8RWIsUrpV+F6vV36/P3hcUVEhj8ej4cOH6+TJk2publZMTIz8fr88Hs8FX7+yslaBQNf/gH3oWLVef/eosmaM0+vvHtWYbwySJL3+7lHN/89vtRlr77zeOKbmgF7fc+ySZHjtz5/LmJa7wT0fn2i5sz5apekTE9rMRWos5YZ/D97hd/RYrXPn36G7XGoz3tWx3q47uVrHY2McGaN/Pa9eFe36TI7jaPrEBO352Bcy9r0b/l07i0vDntc6dvb16tr5HY1F8j0cic9E6+erpz7n4TJcyB2+y+V0eqPcrcIfNWqU+vfvr+LiYt1www0qKipScnKy4uLiNGXKFL3xxhvKysrSli1blJyc3J2HCuvc/a4JY4cpaewwrfv/B2SM0QM3T1LylDEa841BIWPtndfbxgZeERvcY7wjfUJEM0jSn/Z9KWOM3v/Ur+9dPzK4T/vewbP76u9/6tf8GxP12l9KIjL23scnOn2s1rnzt2XO3cNvPT53v76jsb4kEGjZWz93v76jsdbn4/zyj41peQ6K/1EuRy1bPx8cqdT8GxNV9O5RGdNynb0fn2gz1t55HxypDHm9/vqxL+z5HY217uGv/Z/9chxH9+dcF3xvXsxY6+f6Yq937ucrbeqYi17HpcpwbqdFwkUVfl5enpYsWaLrrrtOa9as0WOPPaba2lpNnDhRubm5kqQVK1YoPz9f69ev15VXXqlf/vKXEVlwR0qO14Q8MRPGDtPUpJZvO/TlsUDAaP6NicE/6UTy+tv+2rKX2votHfe/DQg+Vut5rd+ySJs6RicqT0VkrPVbOh09lvvfBmjSVSP4lk6Ev6WTNnWMjleeCvmWzvlj7Z3n6Lz3xoSEsOd3NDYmYYhKjtcErxHy3uyBsXY/X71kba1/L1lyvCZihe8YYyL5pYSIutAtnc643UPk95+MyLV6SjRkkKIjBxl6BzKECrelw7+0BQBLUPgAYAkKHwAsQeEDgCUofACwBIUPAJag8AHAEhQ+AFiCwgcAS1D4AGAJCh8ALEHhA4AlKHwAsASFDwCWoPABwBIUPgBYgsIHAEtQ+ABgCQofACxB4QOAJSh8ALAEhQ8AlqDwAcASFD4AWILCBwBLUPgAYAkKHwAsQeEDgCUofACwBIUPAJboUuFv3bpVc+bMUWpqqjZu3NhmfteuXcrKylJWVpYefPBB1dXVSZJKS0u1cOFCZWdn6wc/+IHKysoiu3oAQJeFLXyfz6eCggK98sor2rJlizZt2qQjR44E52tqapSfn6+CggJt3bpVSUlJKigokCQ9//zzysjIUFFRkVJTU4PjAIDLL2zh7969W9OmTVN8fLwGDhyotLQ0bd++PThfUlKikSNHKjExUZKUkpKit99+W5IUCARUW1srSaqvr9cVV1xxKTIAALogNtwJ5eXlcrvdwWOPx6P9+/cHj8eNG6cTJ07o8OHDSkpK0rZt21RRUSFJWrp0qRYsWKCXX35ZjY2N2rRp0yWIAADoirCFHwgE5DhO8NgYE3I8dOhQPfvss3r88ccVCAR06623Ki4uTpL0k5/8RE899ZRmzZqlN998U/fff79ee+21kJ/vzIgRgy80T6fc7iERvV5PiIYMUnTkIEPvQIauC1v4Xq9X+/btCx77/X55PJ7gcXNzs7xerzZv3ixJ2r9/v0aPHq2qqip9/vnnmjVrliQpLS1NK1asUHV1tYYPH96lxVVW1ioQMBcUqCNu9xD5/Scjcq2eEg0ZpOjIQYbegQyhXC6n0xvlsHv4M2bM0J49e1RVVaX6+nq99dZbSk5ODs47jqM777xTPp9Pxhht2LBBc+bM0bBhw9S/f//gL4vi4mINGjSoy2UPAIissHf4CQkJWrZsmXJzc9XY2Kh58+Zp0qRJysvL05IlS3Tdddfpqaee0t13362GhgZNnz5dd911lxzH0bp16/T000/r9OnTGjRokNauXXs5MgEA2uEYYyKzZ3IJsKUTKhoySNGRgwy9AxlCdXtLBwAQHSh8ALAEhQ8AlqDwAcASFD4AWILCBwBLUPgAYAkKHwAsQeEDgCUofACwBIUPAJag8AHAEhQ+AFiCwgcAS1D4AGAJCh8ALEHhA4AlKHwAsASFDwCWoPABwBIUPgBYgsIHAEtQ+ABgCQofACxB4QOAJSh8ALAEhQ8AlqDwAcASFD4AWILCBwBLUPgAYIkuFf7WrVs1Z84cpaamauPGjW3md+3apaysLGVlZenBBx9UXV2dJKm8vFz33HOP5s6dqwULFqi0tDSyqwcAdFnYwvf5fCooKNArr7yiLVu2aNOmTTpy5EhwvqamRvn5+SooKNDWrVuVlJSkgoICSdJDDz2klJQUbdmyRdnZ2VqzZs2lSwIA6FTYwt+9e7emTZum+Ph4DRw4UGlpadq+fXtwvqSkRCNHjlRiYqIkKSUlRW+//baqqqp0+PBhLViwQJJ0880368c//vEligEACCds4ZeXl8vtdgePPR6PfD5f8HjcuHE6ceKEDh8+LEnatm2bKioq9OWXX2rkyJFavXq1br75Zi1ZskRxcXGXIAIAoCtiw50QCATkOE7w2BgTcjx06FA9++yzevzxxxUIBHTrrbcqLi5OTU1NOnjwoB544AE9/PDD2rx5s/Lz8/Xyyy93eXEjRgy+wDidc7uHRPR6PSEaMkjRkYMMvQMZui5s4Xu9Xu3bty947Pf75fF4gsfNzc3yer3avHmzJGn//v0aPXq03G63Bg0apJSUFElSZmamVq5ceUGLq6ysVSBgLuhnOuJ2D5HffzIi1+op0ZBBio4cZOgdyBDK5XI6vVEOu6UzY8YM7dmzR1VVVaqvr9dbb72l5OTk4LzjOLrzzjvl8/lkjNGGDRs0Z84cjRkzRl6vV7t27ZIkvfPOO5o4cWIEIgEALkbYwk9ISNCyZcuUm5uruXPnKjMzU5MmTVJeXp4OHDggl8ulp556SnfffbfS09M1dOhQ3XXXXZKktWvX6re//a0yMzP10ksv6ac//eklDwQAaJ9jjInMnsklwJZOqGjIIEVHDjL0DmQI1e0tHQBAdKDwAcASFD4AWILCBwBLUPgAYAkKHwAsQeEDgCUofACwBIUPAJag8AHAEhQ+AFiCwgcAS1D4AGAJCh8ALEHhA4AlKHwAsASFDwCWoPABwBIUPgBYgsIHAEtQ+ABgCQofACxB4QOAJWJ7egGdcbmcXn29nhANGaToyEGG3oEMXb+OY4wxEXkkAECvxpYOAFiCwgcAS1D4AGAJCh8ALEHhA4AlKHwAsASFDwCWoPABwBIUPgBYIuoLf9++fcrJyVFWVpbuu+8+/fOf/5Qk1dTU6J577tHs2bO1cOFC+f3+Hl5px4qLizVv3jxlZ2dr0aJFKisrk9S3MrR67rnntHbt2uBxX8uwdetWzZkzR6mpqdq4cWNPL+eC1NbWKjMzU6WlpZKk3bt3KysrS6mpqSooKOjh1YW3bt06ZWRkKCMjQz/72c8k9b0Mzz//vObMmaOMjAz9/ve/l3SZM5goN2vWLPPpp58aY4z5+c9/bn7xi18YY4x58sknza9//WtjjDF/+MMfzNKlS3tsjeGkpKSYQ4cOGWOM2bx5s7nvvvuMMX0rQ01NjXn44YfNpEmTzAsvvBAc70sZTpw4YVJSUkx1dbWpq6szWVlZwfdWb/fBBx+YzMxMM3HiRPPll1+a+vp6M3PmTPPFF1+YxsZGc+edd5qdO3f29DI79Je//MXMnz/fnDlzxjQ0NJjc3FyzdevWPpXhvffeMwsWLDCNjY2mvr4++Lm+nBmi/g7/jTfeUGJiohobG+Xz+TR06FBJ0s6dO5WVlSVJyszM1J///Gc1Njb25FLb1dDQoKVLlyopKUmSNH78eB0/flxS38kgSTt27NC4ceP0wx/+MGS8L2XYvXu3pk2bpvj4eA0cOFBpaWnavn17Ty+rSwoLC7VixQp5PB5J0v79+zV27FiNHj1asbGxysrK6tVZ3G638vPz1a9fP8XFxemqq65SSUlJn8owdepUvfTSS4qNjVVlZaWam5tVU1NzWTNEfeHHxcXpk08+0cyZM/Xee+8pIyNDklReXi632y1Jio2N1eDBg1VVVdWTS21Xv379lJ2dLUkKBAJat26dZs2aJanvZJCkuXPn6p577lFMTEzIeF/KcO5aJcnj8cjn8/Xgirpu1apVmjJlSvC4r2W5+uqrNXnyZElSSUmJtm3bJsdx+lQGqaWPXnjhBWVkZGj69OmX/XXo1f955Auxbds2PfPMMyFj3/zmN7VhwwaNHz9eu3fv1quvvqply5bp1VdfbfPzxhi5XD37+6+zDA0NDcrPz1dTU5Puvffedn++t2foit6QoSOBQECOc/Y/P2uMCTnuS/pqlk8//VT33nuvHnroIcXExKikpCQ411cyLFmyRHl5ebrvvvtUUlJyWV+HqCn82bNna/bs2SFjZ86c0dtvvx28I/7+97+vZ599VlLLb9KKigp5vV41NTWprq5O8fHxl33d52ovgyTV1dVp8eLFio+P1/r16xUXFyepb2XoSG/M0BGv16t9+/YFj/1+f3CLpK/xer0hf0HeF7IUFxdryZIleuSRR5SRkaG9e/f2qQyfffaZGhoaNGHCBA0YMECpqanavn17yJ96L3WG3nkrFSGxsbF68skn9dFHH0lqufv8zne+I0maOXOmtmzZIqlln3/KlCnBIu1tli9frrFjx+q5555Tv379guN9KUNH+lKGGTNmaM+ePaqqqlJ9fb3eeustJScn9/SyLsq3v/1tHT16VMeOHVNzc7P++Mc/9uosx48f149+9COtWbMmuC3b1zKUlpbqscceU0NDgxoaGrRjxw4tWLDgsmaImjv89sTExKigoEBPPPGEmpublZCQoFWrVkmSli5dqvz8fGVkZGjIkCFas2ZND6+2fQcPHtSOHTuUmJiom266SVLLXfFvfvObPpOhM30pQ0JCgpYtW6bc3Fw1NjZq3rx5mjRpUk8v66L0799fq1ev1gMPPKAzZ85o5syZSk9P7+lldeh3v/udzpw5o9WrVwfHFixY0KcyzJw5U/v379fcuXMVExOj1NRUZWRkaPjw4ZctA/+PVwBgiaje0gEAnEXhA4AlKHwAsASFDwCWoPABwBIUPgBYgsIHAEtQ+ABgif8DpW0AwexWTREAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "significand_bits = 4\n",
    "\n",
    "expon_max, expon_min = 4, -3\n",
    "fp_numbers = []\n",
    "\n",
    "for sign_bit in [1, 0]:\n",
    "        for exp in range(expon_min, expon_max + 1):\n",
    "                for bits in range(0, 2**significand_bits):\n",
    "                        significand = (-1)**(sign_bit) * ( 1 + bits / 2**significand_bits ) * 2 ** exp\n",
    "                        fp_numbers.append(significand)\n",
    "\n",
    "fp_numbers = np.array(fp_numbers)\n",
    "\n",
    "sns.set()\n",
    "plt.plot(fp_numbers, np.ones_like(fp_numbers), \"x\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![img](https://pic1.zhimg.com/v2-53e4a2b1c2d91a2581004f7511d78a50_r.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}