DataFrameAccess.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accessing your data in DataFrames\n",
    "\n",
    "*Note: There is also a longer, more complete version of this information in the notebook [FullAccessingDataFrames](FullAccessingDataFrames.ipynb)*\n",
    "\n",
    "One big problem with Pandas is that, sometimes for historical reasons, there are multiple ways of doing the same thing. **The syntax for accessing DataFrames has always been one of the most confusing aspects for me!**\n",
    "\n",
    "Let's make a toy DataFrame to see some of the features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "*To preserve the mystery, select from the notebook menus*\n",
    "\n",
    "`Edit -> Clear All Outputs`\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dict = {'letters':['A','B','c','D','eee'], \n",
    "             'hundreds':[100,200,300,400,500], \n",
    "             'tens':[10.0,20.0,30.0,40.0,50.0],\n",
    "             'boolean':[True,False,True,True,False]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>letters</th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>boolean</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>200</td>\n",
       "      <td>20.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>D</td>\n",
       "      <td>400</td>\n",
       "      <td>40.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>eee</td>\n",
       "      <td>500</td>\n",
       "      <td>50.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  letters  hundreds  tens  boolean\n",
       "0       A       100  10.0     True\n",
       "1       B       200  20.0    False\n",
       "2       c       300  30.0     True\n",
       "3       D       400  40.0     True\n",
       "4     eee       500  50.0    False"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(data_dict)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Each column has a data \"type\"\n",
    "\n",
    "- **object** is how Pandas refers to strings of text\n",
    "- **int64** is a 64-bit integer (whole number). The number of bits is just the amount of internal storage used for that number. *For integers it limits how big the number can be.* **Note that the default int64 can not store NaN/Null values!** See the [integer NA documentation](https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html) for more information.\n",
    "- **float64** is a \"floating point\" number (number with decimal places). *For floats the number of bits limits the precision of the number.*\n",
    "- **bool** is a booleal value, which is just True/False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "letters      object\n",
       "hundreds      int64\n",
       "tens        float64\n",
       "boolean        bool\n",
       "dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DataFrame index\n",
    "\n",
    "Notice the column of sequential integers off to the left-hand side of the DataFrame output. That is the DataFrame's **index**. \n",
    "\n",
    "- **The index contains the names of the rows**\n",
    "- Because we didn't explicitly specify an index column, Pandas created one for us"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=5, step=1)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DataFrame columns\n",
    "\n",
    "There is a separate index of column names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['letters', 'hundreds', 'tens', 'boolean'], dtype='object')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## `df[]` with a name for column indexing/selecting\n",
    "\n",
    "The most common, and concise, way of selecting a column out of a DataFrame is just using square brackets with the column name inside – similar to how you access a dictionary value using it's key."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Single name gives single column\n",
    "\n",
    "This returns a Series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    100\n",
       "1    200\n",
       "2    300\n",
       "3    400\n",
       "4    500\n",
       "Name: hundreds, dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['hundreds']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List of column names inside the square brackets gives multiple columns\n",
    "\n",
    "You can select multiple columns by putting a *list of column names* inside the square brackets. This returns a DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tens</th>\n",
       "      <th>hundreds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.0</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>20.0</td>\n",
       "      <td>200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>30.0</td>\n",
       "      <td>300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>40.0</td>\n",
       "      <td>400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>50.0</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   tens  hundreds\n",
       "0  10.0       100\n",
       "1  20.0       200\n",
       "2  30.0       300\n",
       "3  40.0       400\n",
       "4  50.0       500"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['tens','hundreds']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boolean Series as a selector\n",
    "\n",
    "**It's very common to want all the rows from your DataFrame which pass a certain test**, or set of criteria. You can write the test itself in a very straightforward way, returning a `Series` of True/False boolean values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1     True\n",
       "2     True\n",
       "3    False\n",
       "4    False\n",
       "Name: tens, dtype: bool"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['tens'] < 35"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `df[]` with boolean Series for multiple rows, all columns\n",
    "\n",
    "**Already things are getting screwy – If we use a single bracket with a boolean Series inside, we get back rows instead of columns!**\n",
    "\n",
    "*This can be confusing, but it's convenient enough that you'll see it quite often. Note that you can easily use the `df.loc[,:]` notation below to do the same thing, which is somewhat more clear and readable.*\n",
    "\n",
    "### Rows are returned where boolean Series == True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>letters</th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>boolean</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>200</td>\n",
       "      <td>20.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  letters  hundreds  tens  boolean\n",
       "0       A       100  10.0     True\n",
       "1       B       200  20.0    False\n",
       "2       c       300  30.0     True"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['tens'] < 35]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>letters</th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>boolean</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>D</td>\n",
       "      <td>400</td>\n",
       "      <td>40.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  letters  hundreds  tens  boolean\n",
       "0       A       100  10.0     True\n",
       "2       c       300  30.0     True\n",
       "3       D       400  40.0     True"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['boolean']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More complicated conditionals\n",
    "\n",
    "You can combine multiple conditions if you put them in parentheses and put a logical operator between.\n",
    "\n",
    "- `&` = \"and\"\n",
    "- `|` = \"or\"  *(It's called a \"pipe\" character, and it's `shift-\\` above the Enter/Return key)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>letters</th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>boolean</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  letters  hundreds  tens  boolean\n",
       "2       c       300  30.0     True"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[(df['tens'] < 35) & (df['hundreds'] > 200)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "*To try the exercise below, if you haven't been executing the cells above, select this cell and from the Jupyter menus choose*\n",
    "\n",
    "`Run -> Run All Above Selected Cell`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EXERCISE\n",
    "\n",
    "**Return all rows in `df` where the \"hundreds\" column value is greater than or equal to 400**\n",
    "\n",
    "*Note: Type instead of using copy/paste for better retention*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Row Index doesn't have to be integers!\n",
    "\n",
    "The Index values aren't \"row numbers\". Instead, **the Index values are the names of the rows, so you can use things like strings or dates for the Index**. Let's start by making a new column.\n",
    "\n",
    "*Note that if we creating a new column from a list we need to make sure we don't make any mistakes in the order of the DataFrame rows compared to the order in our list! A more reliable solution would be to create a Series and assign to the new column from that.*\n",
    "\n",
    "*Better:* `df['spelled_out] = pd.Series(['One','Two','Three','Four','Five'])`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>letters</th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>boolean</th>\n",
       "      <th>spelled_out</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "      <td>True</td>\n",
       "      <td>One</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>200</td>\n",
       "      <td>20.0</td>\n",
       "      <td>False</td>\n",
       "      <td>Two</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>True</td>\n",
       "      <td>Three</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>D</td>\n",
       "      <td>400</td>\n",
       "      <td>40.0</td>\n",
       "      <td>True</td>\n",
       "      <td>Four</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>eee</td>\n",
       "      <td>500</td>\n",
       "      <td>50.0</td>\n",
       "      <td>False</td>\n",
       "      <td>Five</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  letters  hundreds  tens  boolean spelled_out\n",
       "0       A       100  10.0     True         One\n",
       "1       B       200  20.0    False         Two\n",
       "2       c       300  30.0     True       Three\n",
       "3       D       400  40.0     True        Four\n",
       "4     eee       500  50.0    False        Five"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['spelled_out'] = ['One','Two','Three','Four','Five']\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>letters</th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>boolean</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>A</td>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>B</td>\n",
       "      <td>200</td>\n",
       "      <td>20.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>c</td>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Four</th>\n",
       "      <td>D</td>\n",
       "      <td>400</td>\n",
       "      <td>40.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Five</th>\n",
       "      <td>eee</td>\n",
       "      <td>500</td>\n",
       "      <td>50.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            letters  hundreds  tens  boolean\n",
       "spelled_out                                 \n",
       "One               A       100  10.0     True\n",
       "Two               B       200  20.0    False\n",
       "Three             c       300  30.0     True\n",
       "Four              D       400  40.0     True\n",
       "Five            eee       500  50.0    False"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = df.set_index('spelled_out')\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Every column's Series has this same Index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spelled_out\n",
       "One        A\n",
       "Two        B\n",
       "Three      c\n",
       "Four       D\n",
       "Five     eee\n",
       "Name: letters, dtype: object"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2['letters']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spelled_out\n",
       "One      10.0\n",
       "Two      20.0\n",
       "Three    30.0\n",
       "Four     40.0\n",
       "Five     50.0\n",
       "Name: tens, dtype: float64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2['tens']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## `df.loc[row,col]` for label-based, multi-axis indexing\n",
    "\n",
    "**This is the best method!** It lets you select first along rows, and then along columns, in both directions simultaneoulsy using row and column \"labels\", which are the row index and column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'A'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.loc['One','letters']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Colon `:` for whole or slices\n",
    "\n",
    "A notation that comes from accessing lists in Python is the \"slice\" operator, which is specified with a colon between two values. **The colon by itself denotes all Index entries in a row or column.** So, here we grab a single column and all rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spelled_out\n",
       "One      100\n",
       "Two      200\n",
       "Three    300\n",
       "Four     400\n",
       "Five     500\n",
       "Name: hundreds, dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.loc[:,'hundreds']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "*To try the exercise below, if you haven't been executing the cells above, select this cell and from the Jupyter menus choose*\n",
    "\n",
    "`Run -> Run All Above Selected Cell`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EXERCISE\n",
    "\n",
    "**Do you know how to use the slice notation to specify just the rows up to and including \"Three\"?**\n",
    "\n",
    "*Note: Type instead of using copy/paste for better retention*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lists for combinations\n",
    "\n",
    "Lists of values work the same as with the `df[]` notation. *Note that again, order matters for what is returned!*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tens</th>\n",
       "      <th>letters</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>10.0</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>20.0</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>30.0</td>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Four</th>\n",
       "      <td>40.0</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Five</th>\n",
       "      <td>50.0</td>\n",
       "      <td>eee</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             tens letters\n",
       "spelled_out              \n",
       "One          10.0       A\n",
       "Two          20.0       B\n",
       "Three        30.0       c\n",
       "Four         40.0       D\n",
       "Five         50.0     eee"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.loc[:,['tens','letters']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Single rows are a Series, too\n",
    "\n",
    "Remember, any 1D result, row or column, will be a Series in Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "letters        c\n",
       "hundreds     300\n",
       "tens          30\n",
       "boolean     True\n",
       "Name: Three, dtype: object"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.loc['Three',:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Boolean Series can be used for selecting True rows or columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>letters</th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>boolean</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>A</td>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>B</td>\n",
       "      <td>200</td>\n",
       "      <td>20.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>c</td>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            letters  hundreds  tens  boolean\n",
       "spelled_out                                 \n",
       "One               A       100  10.0     True\n",
       "Two               B       200  20.0    False\n",
       "Three             c       300  30.0     True"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.loc[df2['tens']<35,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "*To try the exercise below, if you haven't been executing the cells above, select this cell and from the Jupyter menus choose*\n",
    "\n",
    "`Run -> Run All Above Selected Cell`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EXERCISE\n",
    "\n",
    "**Using the `df2.loc[]` notation, return all columns for all rows in which the \"boolean\" column is True**\n",
    "\n",
    "*Note: Type instead of using copy/paste for better retention*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I've skipped the integer selection methods\n",
    "\n",
    "Indexing by integers using the `df.iloc[]` method is good to know about, but frankly, I hardly ever find myself using that method – I prefer to use labels. \n",
    "\n",
    "*I never use integers inside the `df[]` notation – it is just a historical leftover that should be avoided!*\n",
    "\n",
    "See the [FullAccessingDataFrames notebook](FullAccessingDataFrames.ipynb) to learn more."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## SettingWithCopyWarning\n",
    "\n",
    "If you use the \"list of column names\" notation and assign to a new variable as a way to keep just a couple columns of your DataFrame, you will run into a **SettingWithCopyWarning** if you try to change that new DataFrame. \n",
    "\n",
    "*Because of some Pandas inner-workings, when you slice or index into a DataFrame it's not actually clear whether it will create a \"view\" into the original DataFrame or a copy of the data!* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>200</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Four</th>\n",
       "      <td>400</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Five</th>\n",
       "      <td>500</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             hundreds  tens\n",
       "spelled_out                \n",
       "One               100  10.0\n",
       "Two               200  20.0\n",
       "Three             300  30.0\n",
       "Four              400  40.0\n",
       "Five              500  50.0"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_nums = df2[['hundreds','tens']]\n",
    "df_nums"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### It does the operation but gives you a confusing warning\n",
    "\n",
    "You don't want these warnings all over the place in your notebook or you'll miss really important errors and warnings, and there's a chance you might be doing something wrong, anyway!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-23-677717d8156d>:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_nums['sums'] = df_nums['hundreds'] + df_nums['tens']\n"
     ]
    }
   ],
   "source": [
    "df_nums['sums'] = df_nums['hundreds'] + df_nums['tens']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create an explicit `.copy()`\n",
    "\n",
    "- **If you are assigning a subset of a DataFrame to a new variable**, with the intention of creating a copy that you'll work on independently of the original, **create an explicit copy by chaining the `.copy()` method to the end!**\n",
    "- If you're trying to assign values to a subset of your original DataFrame, **use the `df.loc[,] =` notation** detailed below.\n",
    "\n",
    "Additional learning resources:\n",
    "- [The best explanation of the SettingWithCopyWarning I've seen](https://www.dataquest.io/blog/settingwithcopywarning/)\n",
    "- [A complicated explanation in the Pandas documentation](https://pandas.pydata.org/pandas-docs/version/0.22/indexing.html#indexing-view-versus-copy)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>sums</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "      <td>110.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>200</td>\n",
       "      <td>20.0</td>\n",
       "      <td>220.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>330.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Four</th>\n",
       "      <td>400</td>\n",
       "      <td>40.0</td>\n",
       "      <td>440.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Five</th>\n",
       "      <td>500</td>\n",
       "      <td>50.0</td>\n",
       "      <td>550.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             hundreds  tens   sums\n",
       "spelled_out                       \n",
       "One               100  10.0  110.0\n",
       "Two               200  20.0  220.0\n",
       "Three             300  30.0  330.0\n",
       "Four              400  40.0  440.0\n",
       "Five              500  50.0  550.0"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_nums = df2[['hundreds','tens']].copy()\n",
    "df_nums['sums'] = df_nums['hundreds'] + df_nums['tens']\n",
    "df_nums"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## `df.loc[row,col]` for setting values\n",
    "\n",
    "So far we've been accessing values in DataFrames to read/get what's already there, but **the same methods can be used for setting values on DataFrame subsets**!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tens</th>\n",
       "      <th>hundreds</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>10.0</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>20.0</td>\n",
       "      <td>200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>30.0</td>\n",
       "      <td>300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Four</th>\n",
       "      <td>40.0</td>\n",
       "      <td>400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Five</th>\n",
       "      <td>50.0</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             tens  hundreds\n",
       "spelled_out                \n",
       "One          10.0       100\n",
       "Two          20.0       200\n",
       "Three        30.0       300\n",
       "Four         40.0       400\n",
       "Five         50.0       500"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_set = df2.loc[:,['tens','hundreds']].copy()\n",
    "df_set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tens</th>\n",
       "      <th>hundreds</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>10.0</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>20.0</td>\n",
       "      <td>200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>50.0</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Four</th>\n",
       "      <td>40.0</td>\n",
       "      <td>400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Five</th>\n",
       "      <td>50.0</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             tens  hundreds\n",
       "spelled_out                \n",
       "One          10.0       100\n",
       "Two          20.0       200\n",
       "Three        50.0        50\n",
       "Four         40.0       400\n",
       "Five         50.0       500"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_set.loc['Three',:] = 50\n",
    "df_set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tens</th>\n",
       "      <th>hundreds</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>10.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>20.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>50.0</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Four</th>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Five</th>\n",
       "      <td>50.0</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             tens  hundreds\n",
       "spelled_out                \n",
       "One          10.0         0\n",
       "Two          20.0         0\n",
       "Three        50.0        50\n",
       "Four         40.0         0\n",
       "Five         50.0       500"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_set.loc[df_set['tens'] <= 45.0, 'hundreds'] = 0\n",
    "df_set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "# Selection method you'll see, but don't really need!\n",
    "\n",
    "A very good article on more advanced Pandas features where there are multiple ways of doing similar things, and [Ted Petrou's](https://medium.com/@petrou.theodore) \n",
    "opinions on which to use, is \n",
    "[Minimally Sufficient Pandas](https://medium.com/dunder-data/minimally-sufficient-pandas-a8e67f2a2428). \n",
    "\n",
    "*These pieces are taken from that article, but there's a lot more great content there that we don't have time to cover.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selecting a single column with the \"dot\" notation\n",
    "\n",
    "A very common alternative to selecting a single column with the `df['name']` bracket notation you'll see all the time, is what's called the \"dot\" notation, where you follow the dataframe name by a dot and the column name, `df.name`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spelled_out\n",
       "One      100\n",
       "Two      200\n",
       "Three    300\n",
       "Four     400\n",
       "Five     500\n",
       "Name: hundreds, dtype: int64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.hundreds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Issues with the \"dot\" notation\n",
    "\n",
    "There are three issues with using dot notation. It doesn’t work in the following situations:\n",
    "\n",
    "- When there are spaces in the column name\n",
    "- When the column name is the same as a DataFrame method\n",
    "- When the name of a column you want to access is stored in a variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Spaces in the columns name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>letters</th>\n",
       "      <th>hundreds</th>\n",
       "      <th>tens</th>\n",
       "      <th>boolean</th>\n",
       "      <th>more letters</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>spelled_out</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>One</th>\n",
       "      <td>A</td>\n",
       "      <td>100</td>\n",
       "      <td>10.0</td>\n",
       "      <td>True</td>\n",
       "      <td>AA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Two</th>\n",
       "      <td>B</td>\n",
       "      <td>200</td>\n",
       "      <td>20.0</td>\n",
       "      <td>False</td>\n",
       "      <td>BB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Three</th>\n",
       "      <td>c</td>\n",
       "      <td>300</td>\n",
       "      <td>30.0</td>\n",
       "      <td>True</td>\n",
       "      <td>cc</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Four</th>\n",
       "      <td>D</td>\n",
       "      <td>400</td>\n",
       "      <td>40.0</td>\n",
       "      <td>True</td>\n",
       "      <td>DD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Five</th>\n",
       "      <td>eee</td>\n",
       "      <td>500</td>\n",
       "      <td>50.0</td>\n",
       "      <td>False</td>\n",
       "      <td>eeeeee</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            letters  hundreds  tens  boolean more letters\n",
       "spelled_out                                              \n",
       "One               A       100  10.0     True           AA\n",
       "Two               B       200  20.0    False           BB\n",
       "Three             c       300  30.0     True           cc\n",
       "Four              D       400  40.0     True           DD\n",
       "Five            eee       500  50.0    False       eeeeee"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2['more letters'] = df2['letters'] + df2['letters']\n",
    "df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spelled_out\n",
       "One          AA\n",
       "Two          BB\n",
       "Three        cc\n",
       "Four         DD\n",
       "Five     eeeeee\n",
       "Name: more letters, dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2['more letters']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-31-95d1872b366d>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-31-95d1872b366d>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    df2.more letters\u001b[0m\n\u001b[0m             ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "df2.more letters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Column name is the same as a DataFrame method\n",
    "\n",
    "`sum` can be the name of one of our columns, and it's easy to access those values with the \"quoted name in square brackets\" notation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spelled_out\n",
       "One      110.0\n",
       "Two      220.0\n",
       "Three    330.0\n",
       "Four     440.0\n",
       "Five     550.0\n",
       "Name: sum, dtype: float64"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2['sum'] = df2['tens'] + df2['hundreds']\n",
    "df2['sum']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**But if we try to access that column's values with the \"dot\" notation, there is a problem.** The output here will seem confusing, but it's basically saying that `.sum` is a method (built-in function) of the DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bound method DataFrame.sum of             letters  hundreds  tens  boolean more letters    sum\n",
       "spelled_out                                                     \n",
       "One               A       100  10.0     True           AA  110.0\n",
       "Two               B       200  20.0    False           BB  220.0\n",
       "Three             c       300  30.0     True           cc  330.0\n",
       "Four              D       400  40.0     True           DD  440.0\n",
       "Five            eee       500  50.0    False       eeeeee  550.0>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.sum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The column name is stored in a variable\n",
    "\n",
    "It's not uncommon to want to access a column who's name string has been stored in a variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spelled_out\n",
       "One      10.0\n",
       "Two      20.0\n",
       "Three    30.0\n",
       "Four     40.0\n",
       "Five     50.0\n",
       "Name: tens, dtype: float64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "column_name = 'tens'\n",
    "df2[column_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**But, again, that doesn't work with the \"dot\" notation**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'DataFrame' object has no attribute 'column_name'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-35-33864dc204f6>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdf2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumn_name\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/lib/python3.8/site-packages/pandas/core/generic.py\u001b[0m in \u001b[0;36m__getattr__\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m   5137\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_info_axis\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_can_hold_identifiers_and_holds_name\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5138\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 5139\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mobject\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__getattribute__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   5140\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5141\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__setattr__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mAttributeError\u001b[0m: 'DataFrame' object has no attribute 'column_name'"
     ]
    }
   ],
   "source": [
    "df2.column_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lots of Pandas is written with the dot notation. Why?\n",
    "\n",
    "Many tutorials make use of the dot notation to select a single column of data. Why is this done when the brackets seem to be clearly superior? \n",
    "\n",
    "- It might be because **the official documentation contains plenty of examples that use it**\n",
    "- It also ***uses three fewer characters which entices the very laziest amongst us***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}