examples/tutorial_gender_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bias in Image based Automatic Gender Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recent studies have shown that the machine learning models for gender classification task from face images perform differently across groups defined by skin tone. In this tutorial, we will demonstrate the use of the aif360 toolbox to study the the differential performance of a custom classifier. We use a bias mitigiating algorithm available in aif360 with the aim of improving a classfication model in terms of the fairness metrics. We will work with the UTK dataset for this tutorial. This can be downloaded from here:\n",
    "https://susanqq.github.io/UTKFace/\n",
    "\n",
    "In a nutshell, we will follow these steps:\n",
    " - Process images and load them as a aif360 dataset\n",
    " - Learn a baseline classifier and obtain fairness metrics\n",
    " - Call the `Reweighing` algorithm to obtain obtain instance weights\n",
    " - Learn a new classifier with the instance weights and obtain updated fairness metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Call the import statements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import glob\n",
    "from skimage import io\n",
    "from skimage.transform import resize\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib import gridspec\n",
    "from IPython.display import Markdown, display\n",
    "\n",
    "import torch\n",
    "import torch.utils.data\n",
    "from torch.autograd import Variable\n",
    "import torch.nn as nn\n",
    "from torchsummary import summary\n",
    "\n",
    "import pandas as pd\n",
    "import sys\n",
    "sys.path.append(\"../\")\n",
    "\n",
    "from aif360.datasets import BinaryLabelDataset\n",
    "from aif360.metrics import BinaryLabelDatasetMetric\n",
    "from aif360.metrics import ClassificationMetric\n",
    "from aif360.algorithms.preprocessing.reweighing import Reweighing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x10ad85d10>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(99)\n",
    "torch.manual_seed(99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 1: Load and Process Images\n",
    "The first step is to to download the images `Aligned&cropped` images at this location mentioned above.\n",
    "\n",
    "After unzipping the downloaded file, point the location of the folder in the `image_dir` variable below. \n",
    "The file name has the following format `age`-`gender`-`race`-`date&time`.jpg\n",
    "\n",
    "    age: indicates the age of the person in the picture and can range from 0 to 116.\n",
    "    gender: indicates the gender of the person and is either 0 (male) or 1 (female).\n",
    "    race: indicates the race of the person and can from 0 to 4, denoting White, Black, Asian, Indian, and Others (like Hispanic, Latino, Middle Eastern).\n",
    "    date&time: indicates the date and time an image was collected in the UTK dataset.\n",
    "\n",
    "For this tutorial we will restict the images to contrain `White` and `Others` races. We need to specify the unprivileged and previledged groups to obtain various metrics from the aif360 toolbox. We set `White` as the previledged group and `Others` as the unpreviledged group for computing the results with gender as the outcome variable that need to be predicted. We set prediction as `female (1)` as the unfavorable label and `male (0)` as favorable label for the purpose of computing metrics and does not have any special meaning in the context of gender prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "races_to_consider = [0,4]\n",
    "unprivileged_groups = [{'race': 4.0}]\n",
    "privileged_groups = [{'race': 0.0}]\n",
    "favorable_label = 0.0 \n",
    "unfavorable_label = 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Update the `image_dir` with the downloaded and extracted images location and specify the desired image size.\n",
    "The images and loaded and resized usign opencv library. The following code creates three key numpy arrays each containing the raw images, the race attributes and the gender labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "image_dir = '/path/to/UTKFace/'\n",
    "img_size = 64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "protected_race = []\n",
    "outcome_gender = []\n",
    "feature_image = []\n",
    "feature_age = []\n",
    "\n",
    "for i, image_path in enumerate(glob.glob(image_dir + \"*.jpg\")):\n",
    "    try:\n",
    "        age, gender, race = image_path.split('/')[-1].split(\"_\")[:3]\n",
    "        age = int(age)\n",
    "        gender = int(gender)\n",
    "        race = int(race)\n",
    "        \n",
    "        if race in races_to_consider:\n",
    "            protected_race.append(race)\n",
    "            outcome_gender.append(gender)\n",
    "            feature_image.append(resize(io.imread(image_path), (img_size, img_size)))\n",
    "            feature_age.append(age)\n",
    "    except:\n",
    "        print(\"Missing: \" + image_path)\n",
    "\n",
    "feature_image_mat = np.array(feature_image)\n",
    "outcome_gender_mat =  np.array(outcome_gender)\n",
    "protected_race_mat =  np.array(protected_race)\n",
    "age_mat = np.array(feature_age)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 2: Learn a Baseline Classifier\n",
    "Lets build a simple convolutional neural network (CNN) with $3$ convolutional layers and $2$ fully connected layers using the `pytorch` framework. \n",
    "![CNN](images/cnn_arch.png)\n",
    "Each convolutional layer is followed by a maxpool layer. The final layer provides the logits for the binary gender predicition task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "class ThreeLayerCNN(torch.nn.Module):\n",
    "    \"\"\"\n",
    "    Input: 128x128 face image (eye aligned).\n",
    "    Output: 1-D tensor with 2 elements. Used for binary classification.\n",
    "    Parameters:\n",
    "        Number of conv layers: 3\n",
    "        Number of fully connected layers: 2       \n",
    "    \"\"\"\n",
    "    def __init__(self):\n",
    "        super(ThreeLayerCNN,self).__init__()\n",
    "        self.conv1 = torch.nn.Conv2d(3,6,5)\n",
    "        self.pool = torch.nn.MaxPool2d(2,2)\n",
    "        self.conv2 = torch.nn.Conv2d(6,16,5)\n",
    "        self.conv3 = torch.nn.Conv2d(16,16,6)\n",
    "        self.fc1 = torch.nn.Linear(16*4*4,120)\n",
    "        self.fc2 = torch.nn.Linear(120,2)\n",
    "\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(torch.nn.functional.relu(self.conv1(x)))\n",
    "        x = self.pool(torch.nn.functional.relu(self.conv2(x)))\n",
    "        x = self.pool(torch.nn.functional.relu(self.conv3(x)))\n",
    "        x = x.view(-1,16*4*4)\n",
    "        x = torch.nn.functional.relu(self.fc1(x))\n",
    "        x = self.fc2(x)\n",
    "        return x\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the dataset into train and test\n",
    "\n",
    "Let us rescale the pixels to lie between $-1$ and $1$ and split the complete dataset into train and test sets. \n",
    "We use $70$-$30$ percentage for train and test, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "feature_image_mat_normed = 2.0 *feature_image_mat.astype('float32') - 1.0\n",
    "\n",
    "N = len(feature_image_mat_normed)\n",
    "ids = np.random.permutation(N)\n",
    "train_size=int(0.7 * N)\n",
    "X_train = feature_image_mat_normed[ids[0:train_size]]\n",
    "y_train = outcome_gender_mat[ids[0:train_size]]\n",
    "X_test = feature_image_mat_normed[ids[train_size:]]\n",
    "y_test = outcome_gender_mat[ids[train_size:]]\n",
    "\n",
    "p_train = protected_race_mat[ids[0:train_size]]\n",
    "p_test = protected_race_mat[ids[train_size:]]\n",
    "\n",
    "age_train = age_mat[ids[0:train_size]]\n",
    "age_test = age_mat[ids[train_size:]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": true
   },
   "source": [
    "Next, we will create the pytorch train and test data loaders after transposing and converting the images and labels. The batch size is set to $64$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 64\n",
    "\n",
    "X_train = X_train.transpose(0,3,1,2)\n",
    "X_test = X_test.transpose(0,3,1,2)\n",
    "\n",
    "train = torch.utils.data.TensorDataset(Variable(torch.FloatTensor(X_train.astype('float32'))), Variable(torch.LongTensor(y_train.astype('float32'))))\n",
    "train_loader = torch.utils.data.DataLoader(train, batch_size=batch_size, shuffle=True)\n",
    "test = torch.utils.data.TensorDataset(Variable(torch.FloatTensor(X_test.astype('float32'))), Variable(torch.LongTensor(y_test.astype('float32'))))\n",
    "test_loader = torch.utils.data.DataLoader(test, batch_size=batch_size, shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Plain Model\n",
    "In the next few steps, we will create and intialize a model with the above described architecture and train it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------\n",
      "        Layer (type)               Output Shape         Param #\n",
      "================================================================\n",
      "            Conv2d-1            [-1, 6, 60, 60]             456\n",
      "         MaxPool2d-2            [-1, 6, 30, 30]               0\n",
      "            Conv2d-3           [-1, 16, 26, 26]           2,416\n",
      "         MaxPool2d-4           [-1, 16, 13, 13]               0\n",
      "            Conv2d-5             [-1, 16, 8, 8]           9,232\n",
      "         MaxPool2d-6             [-1, 16, 4, 4]               0\n",
      "            Linear-7                  [-1, 120]          30,840\n",
      "            Linear-8                    [-1, 2]             242\n",
      "================================================================\n",
      "Total params: 43,186\n",
      "Trainable params: 43,186\n",
      "Non-trainable params: 0\n",
      "----------------------------------------------------------------\n",
      "Input size (MB): 0.05\n",
      "Forward/backward pass size (MB): 0.32\n",
      "Params size (MB): 0.16\n",
      "Estimated Total Size (MB): 0.53\n",
      "----------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n",
    "model = ThreeLayerCNN().to(device)\n",
    "summary(model, (3,img_size,img_size))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the network\n",
    "Next,  we will train the model summarized above. `num_epochs` specifies the number of epochs used for  training. The learning rate is set to $0.001$. We will use the `Adam` otimizer to minimze the standard cross-entropy loss for classification tasks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [1/5], Step [100/129], Loss: 0.4650\n",
      "Epoch [2/5], Step [100/129], Loss: 0.4970\n",
      "Epoch [3/5], Step [100/129], Loss: 0.3691\n",
      "Epoch [4/5], Step [100/129], Loss: 0.2505\n",
      "Epoch [5/5], Step [100/129], Loss: 0.4929\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 5\n",
    "learning_rate = 0.001\n",
    "print_freq = 100\n",
    "\n",
    "# Specify the loss and the optimizer\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)\n",
    "\n",
    "# Start training the model\n",
    "num_batches = len(train_loader)\n",
    "for epoch in range(num_epochs):\n",
    "    for idx, (images, labels) in enumerate(train_loader):\n",
    "        images = images.to(device)\n",
    "        labels = labels.to(device)\n",
    "        \n",
    "        outputs = model(images)\n",
    "        loss = criterion(outputs, labels)\n",
    "\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        \n",
    "        if (idx+1) % print_freq == 0:\n",
    "            print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}' .format(epoch+1, num_epochs, idx+1, num_batches, loss.item()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Measure Fairness Metrics\n",
    "Let's get the predictions of this trained model on the test and use them to compute various fariness metrics available in the aif360 toolbox. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Run model on test set in eval mode.\n",
    "model.eval()\n",
    "y_pred = []\n",
    "with torch.no_grad():\n",
    "    for images, labels in test_loader:\n",
    "        images = images.to(device)\n",
    "        labels = labels.to(device)\n",
    "        outputs = model(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        y_pred += predicted.tolist()\n",
    "y_pred = np.array(y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The wrapper function defined below can be used to convert the numpy arrays and the related meta data into a aif360 dataset. This will ease the process of computing metrics and comparing two datasets. The wrapper consumes the outcome array, the protected attribute array, information about unprivileged_groups and privileged_groups; and the favorable and unfavorable label to produce an instance of aif360's `BinaryLabelDataset`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def dataset_wrapper(outcome, protected, unprivileged_groups, privileged_groups,\n",
    "                          favorable_label, unfavorable_label):\n",
    "    \"\"\" A wraper function to create aif360 dataset from outcome and protected in numpy array format.\n",
    "    \"\"\"\n",
    "    df = pd.DataFrame(data=outcome,\n",
    "                      columns=['outcome'])\n",
    "    df['race'] = protected\n",
    "    \n",
    "    dataset = BinaryLabelDataset(favorable_label=favorable_label,\n",
    "                                       unfavorable_label=unfavorable_label,\n",
    "                                       df=df,\n",
    "                                       label_names=['outcome'],\n",
    "                                       protected_attribute_names=['race'],\n",
    "                                       unprivileged_protected_attributes=unprivileged_groups)\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "original_traning_dataset = dataset_wrapper(outcome=y_train, protected=p_train, \n",
    "                                                 unprivileged_groups=unprivileged_groups, \n",
    "                                                 privileged_groups=privileged_groups,\n",
    "                                                 favorable_label=favorable_label,\n",
    "                                          unfavorable_label=unfavorable_label)\n",
    "original_test_dataset = dataset_wrapper(outcome=y_test, protected=p_test, \n",
    "                                              unprivileged_groups=unprivileged_groups, \n",
    "                                              privileged_groups=privileged_groups,\n",
    "                                                 favorable_label=favorable_label,\n",
    "                                          unfavorable_label=unfavorable_label)\n",
    "plain_predictions_test_dataset = dataset_wrapper(outcome=y_pred, protected=p_test, \n",
    "                                                       unprivileged_groups=unprivileged_groups,\n",
    "                                                       privileged_groups=privileged_groups,\n",
    "                                                 favorable_label=favorable_label,\n",
    "                                          unfavorable_label=unfavorable_label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Obtaining the Classification Metrics\n",
    "We use the `ClassificationMetric` class from the aif360 toolbox for computing metrics based on two BinaryLabelDatasets. The first dataset is the original one and the second is the output of the classification transformer (or similar). Later on we will use `BinaryLabelDatasetMetric` which computes based on a single `BinaryLabelDataset`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "classified_metric_nodebiasing_test = ClassificationMetric(original_test_dataset, \n",
    "                                                 plain_predictions_test_dataset,\n",
    "                                                 unprivileged_groups=unprivileged_groups,\n",
    "                                                 privileged_groups=privileged_groups)\n",
    "TPR = classified_metric_nodebiasing_test.true_positive_rate()\n",
    "TNR = classified_metric_nodebiasing_test.true_negative_rate()\n",
    "bal_acc_nodebiasing_test = 0.5*(TPR+TNR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "#### Plain model - without debiasing - classification metrics"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test set: Classification accuracy = 0.864911\n",
      "Test set: Balanced classification accuracy = 0.867417\n",
      "Test set: Statistical parity difference = -0.084532\n",
      "Test set: Disparate impact = 0.830321\n",
      "Test set: Equal opportunity difference = -0.072537\n",
      "Test set: Average odds difference = -0.033736\n",
      "Test set: Theil index = 0.112105\n",
      "Test set: False negative rate difference = 0.072537\n"
     ]
    }
   ],
   "source": [
    "display(Markdown(\"#### Plain model - without debiasing - classification metrics\"))\n",
    "print(\"Test set: Classification accuracy = %f\" % classified_metric_nodebiasing_test.accuracy())\n",
    "print(\"Test set: Balanced classification accuracy = %f\" % bal_acc_nodebiasing_test)\n",
    "print(\"Test set: Statistical parity difference = %f\" % classified_metric_nodebiasing_test.statistical_parity_difference())\n",
    "print(\"Test set: Disparate impact = %f\" % classified_metric_nodebiasing_test.disparate_impact())\n",
    "print(\"Test set: Equal opportunity difference = %f\" % classified_metric_nodebiasing_test.equal_opportunity_difference())\n",
    "print(\"Test set: Average odds difference = %f\" % classified_metric_nodebiasing_test.average_odds_difference())\n",
    "print(\"Test set: Theil index = %f\" % classified_metric_nodebiasing_test.theil_index())\n",
    "print(\"Test set: False negative rate difference = %f\" % classified_metric_nodebiasing_test.false_negative_rate_difference())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 3: Apply the Reweighing algorithm to tranform the dataset\n",
    "Reweighing is a preprocessing technique that weights the examples in each (group, label) combination differently to ensure fairness before classification [1]. This is one of the very few pre-processing method we are aware of that could tractably be applied to multimedia data (since it does not work with the features).\n",
    "\n",
    "    References:\n",
    "    [1] F. Kamiran and T. Calders,\"Data Preprocessing Techniques for Classification without Discrimination,\" Knowledge and Information Systems, 2012."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "RW = Reweighing(unprivileged_groups=unprivileged_groups,\n",
    "               privileged_groups=privileged_groups)\n",
    "RW.fit(original_traning_dataset)\n",
    "transf_traning_dataset = RW.transform(original_traning_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "metric_orig_train = BinaryLabelDatasetMetric(original_traning_dataset, \n",
    "                                             unprivileged_groups=unprivileged_groups,\n",
    "                                             privileged_groups=privileged_groups)\n",
    "metric_tranf_train = BinaryLabelDatasetMetric(transf_traning_dataset, \n",
    "                                             unprivileged_groups=unprivileged_groups,\n",
    "                                             privileged_groups=privileged_groups)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "#### Original training dataset"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference in mean outcomes between privileged and unprivileged groups = -0.103949\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "#### Transformed training dataset"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference in mean outcomes between privileged and unprivileged groups = 0.000000\n"
     ]
    }
   ],
   "source": [
    "display(Markdown(\"#### Original training dataset\"))\n",
    "print(\"Difference in mean outcomes between privileged and unprivileged groups = %f\" % metric_orig_train.mean_difference())\n",
    "display(Markdown(\"#### Transformed training dataset\"))\n",
    "print(\"Difference in mean outcomes between privileged and unprivileged groups = %f\" % metric_tranf_train.mean_difference())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "#### Original testing dataset"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference in mean outcomes between privileged and unprivileged groups = -0.071139\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "#### Transformed testing dataset"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference in mean outcomes between privileged and unprivileged groups = 0.032808\n"
     ]
    }
   ],
   "source": [
    "metric_orig_test = BinaryLabelDatasetMetric(original_test_dataset, \n",
    "                                             unprivileged_groups=unprivileged_groups,\n",
    "                                             privileged_groups=privileged_groups)\n",
    "transf_test_dataset = RW.transform(original_test_dataset)\n",
    "metric_transf_test = BinaryLabelDatasetMetric(transf_test_dataset, \n",
    "                                             unprivileged_groups=unprivileged_groups,\n",
    "                                             privileged_groups=privileged_groups)\n",
    "display(Markdown(\"#### Original testing dataset\"))\n",
    "print(\"Difference in mean outcomes between privileged and unprivileged groups = %f\" % metric_orig_test.mean_difference())\n",
    "display(Markdown(\"#### Transformed testing dataset\"))\n",
    "print(\"Difference in mean outcomes between privileged and unprivileged groups = %f\" % metric_transf_test.mean_difference())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 4: Learn a New Classfier using the Instance Weights\n",
    "We can see that the reweighing was able to reduce the difference in mean outcomes between privileged and unprivileged groups. This was done by learning appropriate weights for each training instance. In the current step, we will use these learned instance weights to train a network. We  will create coustom pytorch loss called `InstanceWeighetedCrossEntropyLoss` that uses the instances weights to produce the loss value for a batch of data samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "tranf_train = torch.utils.data.TensorDataset(Variable(torch.FloatTensor(X_train.astype('float32'))),\n",
    "                                             Variable(torch.LongTensor(transf_traning_dataset.labels.astype('float32'))),\n",
    "                                            Variable(torch.FloatTensor(transf_traning_dataset.instance_weights.astype('float32'))),)\n",
    "tranf_train_loader = torch.utils.data.DataLoader(tranf_train, batch_size=64, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "class InstanceWeighetedCrossEntropyLoss(nn.Module):\n",
    "    \"\"\"Cross entropy loss with instance weights.\"\"\"\n",
    "    def __init__(self):\n",
    "        super(InstanceWeighetedCrossEntropyLoss, self).__init__()\n",
    "\n",
    "    def forward(self, logits, target, weights):\n",
    "        loss = log_sum_exp(logits) - select_target_class(logits, target.squeeze(1))\n",
    "        loss = loss * weights\n",
    "        return loss.mean()\n",
    "\n",
    "#Helper functions\n",
    "def select_target_class(logits, target):\n",
    "    batch_size, num_classes = logits.size()\n",
    "    mask = torch.autograd.Variable(torch.arange(0, num_classes)\n",
    "                                               .long()\n",
    "                                               .repeat(batch_size, 1)\n",
    "                                               .to(device)\n",
    "                                               .eq(target.data.repeat(num_classes, 1).t()))\n",
    "    return logits.masked_select(mask)\n",
    "\n",
    "def log_sum_exp(x):\n",
    "    c, _ = torch.max(x, 1)\n",
    "    y = c + torch.log(torch.exp(x - c.unsqueeze(dim=1).expand_as(x)).sum(1))\n",
    "    return y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "tranf_model = ThreeLayerCNN().to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [1/5], Step [100/129], Loss: 0.6278\n",
      "Epoch [2/5], Step [100/129], Loss: 0.2547\n",
      "Epoch [3/5], Step [100/129], Loss: 0.3406\n",
      "Epoch [4/5], Step [100/129], Loss: 0.1545\n",
      "Epoch [5/5], Step [100/129], Loss: 0.1986\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 5\n",
    "learning_rate = 0.001\n",
    "print_freq = 100\n",
    "\n",
    "# Specify the loss and the optimizer\n",
    "criterion = InstanceWeighetedCrossEntropyLoss()\n",
    "optimizer = torch.optim.Adam(tranf_model.parameters(), lr=learning_rate)\n",
    "\n",
    "# Start training the new model\n",
    "num_batches = len(tranf_train_loader)\n",
    "for epoch in range(num_epochs):\n",
    "    for idx, (images, labels, weights) in enumerate(tranf_train_loader):\n",
    "\n",
    "        images = images.to(device)\n",
    "        labels = labels.to(device)\n",
    "        \n",
    "        outputs = tranf_model(images)\n",
    "        loss = criterion(outputs, labels, weights)\n",
    "        \n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        \n",
    "        if (idx+1) % print_freq == 0:\n",
    "            print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}' .format(epoch+1, num_epochs, idx+1, num_batches, loss.item()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Test the model\n",
    "tranf_model.eval()\n",
    "y_pred_transf = []\n",
    "with torch.no_grad():\n",
    "    for images, labels in test_loader:\n",
    "        images = images.to(device)\n",
    "        labels = labels.to(device)\n",
    "        outputs = tranf_model(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        y_pred_transf += predicted.tolist()\n",
    "y_pred_transf = np.array(y_pred_transf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us repeat the same steps as before to convert the predictions into aif360 dataset and obtain various metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "transf_predictions_test_dataset = dataset_wrapper(outcome=y_pred_transf, protected=p_test, \n",
    "                                                  unprivileged_groups=unprivileged_groups,\n",
    "                                                  privileged_groups=privileged_groups,\n",
    "                                                 favorable_label=favorable_label,\n",
    "                                                  unfavorable_label=unfavorable_label\n",
    "                                                 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "classified_metric_debiasing_test = ClassificationMetric(original_test_dataset, \n",
    "                                                 transf_predictions_test_dataset,\n",
    "                                                 unprivileged_groups=unprivileged_groups,\n",
    "                                                 privileged_groups=privileged_groups)\n",
    "TPR = classified_metric_debiasing_test.true_positive_rate()\n",
    "TNR = classified_metric_debiasing_test.true_negative_rate()\n",
    "bal_acc_debiasing_test = 0.5*(TPR+TNR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "#### Plain model - without debiasing - classification metrics"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test set: Classification accuracy = 0.864911\n",
      "Test set: Balanced classification accuracy = 0.867417\n",
      "Test set: Statistical parity difference = -0.084532\n",
      "Test set: Disparate impact = 0.830321\n",
      "Test set: Equal opportunity difference = -0.072537\n",
      "Test set: Average odds difference = -0.033736\n",
      "Test set: Theil_index = 0.112105\n",
      "Test set: False negative rate difference = 0.072537\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "#### Model - with debiasing - classification metrics"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test set: Classification accuracy = 0.872557\n",
      "Test set: Balanced classification accuracy = 0.870874\n",
      "Test set: Statistical parity difference = -0.079233\n",
      "Test set: Disparate impact = 0.858805\n",
      "Test set: Equal opportunity difference = -0.066377\n",
      "Test set: Average odds difference = -0.027919\n",
      "Test set: Theil_index = 0.081592\n",
      "Test set: False negative rate difference = 0.066377\n"
     ]
    }
   ],
   "source": [
    "display(Markdown(\"#### Plain model - without debiasing - classification metrics\"))\n",
    "print(\"Test set: Classification accuracy = %f\" % classified_metric_nodebiasing_test.accuracy())\n",
    "print(\"Test set: Balanced classification accuracy = %f\" % bal_acc_nodebiasing_test)\n",
    "print(\"Test set: Statistical parity difference = %f\" % classified_metric_nodebiasing_test.statistical_parity_difference())\n",
    "print(\"Test set: Disparate impact = %f\" % classified_metric_nodebiasing_test.disparate_impact())\n",
    "print(\"Test set: Equal opportunity difference = %f\" % classified_metric_nodebiasing_test.equal_opportunity_difference())\n",
    "print(\"Test set: Average odds difference = %f\" % classified_metric_nodebiasing_test.average_odds_difference())\n",
    "print(\"Test set: Theil_index = %f\" % classified_metric_nodebiasing_test.theil_index())\n",
    "print(\"Test set: False negative rate difference = %f\" % classified_metric_nodebiasing_test.false_negative_rate_difference())\n",
    "\n",
    "display(Markdown(\"#### Model - with debiasing - classification metrics\"))\n",
    "print(\"Test set: Classification accuracy = %f\" % classified_metric_debiasing_test.accuracy())\n",
    "print(\"Test set: Balanced classification accuracy = %f\" % bal_acc_debiasing_test)\n",
    "print(\"Test set: Statistical parity difference = %f\" % classified_metric_debiasing_test.statistical_parity_difference())\n",
    "print(\"Test set: Disparate impact = %f\" % classified_metric_debiasing_test.disparate_impact())\n",
    "print(\"Test set: Equal opportunity difference = %f\" % classified_metric_debiasing_test.equal_opportunity_difference())\n",
    "print(\"Test set: Average odds difference = %f\" % classified_metric_debiasing_test.average_odds_difference())\n",
    "print(\"Test set: Theil_index = %f\" % classified_metric_debiasing_test.theil_index())\n",
    "print(\"Test set: False negative rate difference = %f\" % classified_metric_debiasing_test.false_negative_rate_difference())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us break down these numbers by age to understand how these bias differs across age groups. For demonstration, we dividee all the samples into these age groups: 0-10, 10-20, 20-40, 40-60 and 60-150. For this we will create aif360 datasets using the subset of samples that fall into each of the age groups. The plot below shows how the `Equal opportunity difference` metric varies across age groups before and after applying the bias mitigating reweighing algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEICAYAAABbOlNNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAH8VJREFUeJzt3XuYFdWd7vHvK6DgXQGvqGDEeHlGEfHCeBm8AXoMeozMYIyoRyUmMRPHjBEnk0iMmZNozpijR0XGG0bFC0ZCPCpeEA3ekaijooKIYytRBEQhYgB/80etJlWb3d27e++mu+H9PE89XbVqrapVq9eu365Ve+9SRGBmZlZvg7augJmZtS8ODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwLAOkXSLpMvauh5rm6Sxkn7cStseJKkut/yapEFpXpJulrRY0vMp7duSPpS0VFL31qhTa5M0T9LRbV0Pazud27oC6yJJ84BtgVW55Fsi4ry2qVH7IukWoC4i/rUW24uIc3PbHgTcFhG9arHtMvvaO7d4KHAM0CsilknqAvw7cHBEvNwa+zdbGxwYWs/XIuLRtq5EeyOpU1vXoYZ2AeZFxLK0vC3QFXitJRuT1CkiVjWd06ohqXNErGzrerRnHkpayyR1kvQrSR9Lmivpu5JCUue0vnAZL2mMpNtyy/dI+pOkJZKelLR3uf2U2e8Gkv5V0ruSPpJ0q6Qt0rreqQ6jJH0gab6kH5TUYaKkuyR9JmmmpH1z6/eUNE3SJ2moZVhu3S2SrpP0gKRlwFnAqcAP03DL71O+kLRbSbnL0vwgSXWSfpDqPl/SmaV5JW0CPAjskLa9VNIOkv6cH9aRtL+kBekdfmk7dUvbWyzpdeCAkvXzJB0t6SzgBmBg2s8E4M2U7RNJU1P+PSQ9ImmRpDcl/X0jbXOEpI1S//ivNCQ1VlK3Ctuhm6T/k/7HSyRNz5U9WNLT6X/0crqyaswBkl5P7XCzpK5pO69K+lpun11SX+5Xpi23knR/auvFab5Xbn2f1Ic/k/SopGtK+nrFdZbUX9If07buSX21tP9cJOlPwM0p/RxJc9L/ZrKkHVJ6/euhc2770ySdnebPkPSUpKtTO78h6agm2rNDcWBY+84Bjgf2AwYAJzez/INAX2AbYCZwe4XlzkjTEcCuwKbA/yvJc0Ta9mBgtIrjzCcA9wBbA3cAk9JJoQvwe+DhVKfvAbdL+mqu7DeAnwObAbemOl8eEZtGxNeozHbAFsCOZMHlGklb5TOkd+7HAh+kbW8aER8A04C/z2X9JnBnRKwos59LgK+kaQhwernKRMSNwLnAM2k/pwD1QXrLiDgyBapHyNprG+AU4FoVg3m+baYDvwR2B/oBu6Xj/UmF7fArYH/gb8n+Tz8EvpS0I/D/gctS+j8D90rqWe7YklPT8X8l1ad+2O9WsvardxwwPyJeKrONDchOwrsAOwOfU+xzdwDPA92BMcBp9SuaU2dJGwL3AbekvBOA/1mSbbu0bhdglKQjgf9N1i+2B94F7izfFGUdBMwFepD1md9K2roZ5du3iPBU4wmYBywFPslN56R1U4Fzc3kHAwF0zpU9Ord+DNmYebn9bJnKbpGWbwEuayDvY8B3cstfBVaQDSf2TtvZI7f+cuDGXB2eza3bAJgPHJamPwEb5NZPAMbk6nRrSV3WqGfa/27l8gCDyE4qnXPrPyIbyy+Xt65k2/8APJXmO6X6HthAO80FhuaWR+W3l///kAXa6bl19e3YObffP5Rs/3rgknJtAwhYBnwllzYQeKepdkj/k8+Bfcsc00XAb0rSpgCnN9J/8330OODtNL8D8BmweVqeCPywwtdFP2Bxmt8ZWAlsnFt/G6mvN6fOwOHA+4ByadNL+sRfgK659TeSvTmpX96U7PXQu/T/mNZPA87O/d8/KNnf88BplbRDR5h8j6H1nBjl7zHsALyXW3630g0qG5//OTAc6Al8mVb1AJY0UXyHkn29SxYUts2lldbrb8qti4gvlX1SZ4f6dRHxZUnZHRvYbkstjOK48J/JXsyV+B0wVtKuZO9+l0TE8w3kbfH/p4xdgIMkfZJL6wz8Jrec31dPYGPgRUn1aSILZvUaaoceZPc33m6gHsPzQ0BAF+DxRupe2gY7AETEB5KeAr4u6T6yK7Tvl9uApI2BK4GhQP1VzWapH+8ALIqIP5fsc6cW1HkH4P1IZ+gy9QdYEBHLS8rMrF+IiKWSFpL12/fLHU+J0v2tbqN1gQPD2jefv3Z+yN455S0jOznU2y43/w2yIZ2jyd7VbQEsJjt5NOUDshdbfr8rgQ+B+nHfnYA3cus/yOVfXWdJG6Qy9et3krRBLjjsDLyVK1v6E77lftL3z6x53HVl8jVljW1HxHJJd5MNj+xB8cRcqv7/U38DufT/0xzvAU9ExDGN5MnX92Oyd/17R0QlJ6e8j4HlZEM/pZ+Ieo/s3fc5zdheaR/N94XxwNlk549nGqnrD8iuTA+KiD+l+xB/JOuv84GtJW2cCw75fTanzvOBHSUpd7LeiWKQLO0XhddDGvbrThYU6j9MsDHwaZrfjqLS/e0MTK6grh2C7zGsfXcD/yipVxobHl2y/iVgRBq/L70HsRnwBbCQrNP+WzP2OwH4p3TDb9NU9q6Sd58/lrRxGgM/E7grt25/SSelG3Lnp3o8CzxH9kL6YarzIOBrND5e+yHZfY7S4/6GspvzQ4G/a8axlW67u9KN9ZxbyYYAhpENWTTkbuDidOO0F9k9k5a6H9hd0mn192MkHSBpz3KZU2D9D+BKSdtANtYuaUhTO0plbwL+XdkN906SBkraiOx4vyZpSErvmm7INvaR3u+mPro18C8U+8IkoD/ZlcKtjWxjM7JA90naziW5+r4LzADGSNpQ0kCyflOvOXV+huyj4edJ6izpBODARuoF2f2NMyX1S230b8BzETEvIhaQBYhvpn3/L7KAm7cN2eu4i6ThwJ7AA03ss8NwYGg9v9dfPxmzNF12Q/bCn0L2rm4m8NuScj8m64SLgZ+SdeB6t5Jdsr4PvE52Yq7UTWTvlJ8E3iF7d1l60nsCmEN2P+JXEfFwbt3vyMbMF5PdJDwpIlZExF/ITrbHkr1rvRYYGRFv0LAbgb3Sp00mpbTvk50YPiF7Zz+pocKNSfudAMxN268fAnmKbOhtZkTMa2QTPyVr43fIbqg3dnXRVF0+I7uHNILsHeqfyG4ub9RIsYvI/gfPSvoUeJTsXXcl/hn4T+AFYFHa1wYR8R7Zlea/AAvI3o1fSOOv/zvIjn9umlZ/cTIiPgfuBfqwZv/N+zXQjaxfPAs8VLL+VLJ7KAvT9u8ie8NBc+qc+uBJZDfjPyG7OX5//bbKiYjHyF5r95JdcXyF7P9U75y0v4VkHyp4umQTz5F9UONjsuHdkyNiYUP762hUHCaztU1Sb7KTUJdoo89WN1UHSWPIbgx/s3RdR6LsI6R3RMQNbV2Xjk7ST4Dda9knJN0FvBERlzSZueltPQeMjYibq6/ZGts+g+xG9KG13nZ74SsGWy9IOoBs+OOupvJa49Kw0FnAuCq3c4Ckryj7js1QsiuEFl0pSvo7SduloaTTgX1Y8wrFKuTAYOs8SePJhmTOT8M71kKSziEb1nkwIp6scnPbkX0MdClwFfDtiPhjC7f1VbLh2SVkN71Pjoj5VdZvveWhJDMzK/AVg5mZFXTI7zH06NEjevfu3dbVMDPrUF588cWPI6Kxn0IBOmhg6N27NzNmzGjrapiZdSiSKvomv4eSzMyswIHBzMwKHBjMzKygQ95jKGfFihXU1dWxfPnypjOvB7p27UqvXr3o0mWNZ9GYmTVqnQkMdXV1bLbZZvTu3ZvcTxavlyKChQsXUldXR58+fdq6OmbWwawzQ0nLly+ne/fu631QAJBE9+7dffVkZi2yzgQGwEEhx21hZi21TgUGMzOr3robGKTaThXo1KkT/fr1Y99996V///48/XTpT7iv6aqrrmLPPffk1FNPrfaIK1frtmlGG5lZ+7fO3HxuD7p168ZLL70EwJQpU7j44ot54oknGi1z7bXX8uCDD1Z8k3jlypV07ux/m5m1nnX3iqGNffrpp2y11Varl6+44goOOOAA9tlnHy65JHsOybnnnsvcuXMZNmwYV155JYsWLeLEE09kn3324eCDD+aVV14BYMyYMYwaNYrBgwczcuRIVq1axYUXXrh6e9dff32bHKOZrZv81rOGPv/8c/r168fy5cuZP38+U6dOBeDhhx9m9uzZPP/880QEw4YN48knn2Ts2LE89NBDPP744/To0YPvfe977LfffkyaNImpU6cycuTI1VcgL774ItOnT6dbt26MGzeOLbbYghdeeIEvvviCQw45hMGDB/ujqWZWEw4MNZQfSnrmmWcYOXIkr776Kg8//DAPP/ww++23HwBLly5l9uzZHH744YXy06dP59577wXgyCOPZOHChSxZsgSAYcOG0a1bNyALNK+88goTJ04EYMmSJcyePduBwcxqwoGhlQwcOJCPP/6YBQsWEBFcfPHFfOtb32q0TLmHJtV/7HSTTTYp5Lv66qsZMmRIbSttZobvMbSaN954g1WrVtG9e3eGDBnCTTfdxNKlSwF4//33+eijj9Yoc/jhh3P77bcDMG3aNHr06MHmm2++Rr4hQ4Zw3XXXsWLFCgDeeustli1b1opHY2brk3X3iqENHllaf48h230wfvx4OnXqxODBg5k1axYDBw4EYNNNN+W2225jm222KZQfM2YMZ555Jvvssw8bb7wx48ePL7ufs88+m3nz5tG/f38igp49ezJpUoueoW5mtoYO+cznAQMGROmDembNmsWee+7ZRjVqnxpsk9b6zkEH7Etm6xNJL0bEgKbyeSjJzMwKHBjMzKzAgcHMzAocGMzMrMCBwczMChwYzMysYJ39HsO0abX9SOagQZV9FPO+++7jpJNOYtasWeyxxx4AXHjhhTzwwAMcd9xxHHLIIey+++7stddeNa2fmVmt+IqhxiZMmMChhx7KnXfeuTrt+uuvZ+bMmVxxxRVMmjSJ119/vVnbXLlyZa2raWbWIAeGGlq6dClPPfUUN9544+rAMGzYMJYtW8ZBBx3ET3/6UyZPnsyFF15Iv379ePvtt3n77bcZOnQo+++/P4cddhhvvPEGAGeccQYXXHABRxxxBBdddFFbHpaZrWfW2aGktjBp0iSGDh3K7rvvztZbb83MmTOZPHkym2666epfXX3nnXc4/vjjOfnkkwE46qijGDt2LH379uW5557jO9/5zuqf637rrbd49NFH6dSpU5sdk5mtfxwYamjChAmcf/75AIwYMYIJEybQv3//BvMvXbqUp59+muHDh69O++KLL1bPDx8+3EHBzNY6B4YaWbhwIVOnTuXVV19FEqtWrUISl19+eYNlvvzyS7bccsvVVxOl8j+1bWa2tvgeQ41MnDiRkSNH8u677zJv3jzee+89+vTpw/Tp0wv5NttsMz777DMANt98c/r06cM999wDZL/I+vLLL6/1upuZ5a2zVwyVfry0ViZMmMDo0aMLaV//+te54447CmkjRozgnHPO4aqrrmLixIncfvvtfPvb3+ayyy5jxYoVjBgxgn333XdtVt3MrMA/u70O889umzWitV4H0G5fC2v1Z7clDZX0pqQ5kkaXWX+4pJmSVko6uWTd6ZJmp+n0WtTHzMxarurAIKkTcA1wLLAXcIqk0q/1/hdwBnBHSdmtgUuAg4ADgUskbVVtnczMrOVqccVwIDAnIuZGxF+AO4ET8hkiYl5EvAJ8WVJ2CPBIRCyKiMXAI8DQllakIw6LtRa3hZm1VC0Cw47Ae7nlupTW2mULunbtysKFC31CJAsKCxcupGvXrm1dFTPrgGrxqaRyd3AqPTtXXFbSKGAUwM4777zG+l69elFXV8eCBQsq3PW6rWvXrvTq1autq2FmHVAtAkMdsFNuuRfwQTPKDiopO61cxogYB4yD7FNJpeu7dOlCnz59KtytmZk1pBZDSS8AfSX1kbQhMAKYXGHZKcBgSVulm86DU5qZmbWRqgNDRKwEziM7oc8C7o6I1yRdKmkYgKQDJNUBw4HrJb2Wyi4CfkYWXF4ALk1pZmbWRtaZL7hZM/gLbmb+glsj/FtJZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgWd27oCZmbrmmnT1CrbHTQoWmW7pWpyxSBpqKQ3Jc2RNLrM+o0k3ZXWPyepd0rvLelzSS+laWwt6mNmZi1X9RWDpE7ANcAxQB3wgqTJEfF6LttZwOKI2E3SCOCXwD+kdW9HRL9q62FmZrVRiyuGA4E5ETE3Iv4C3AmcUJLnBGB8mp8IHCWpda61zMysKrUIDDsC7+WW61Ja2TwRsRJYAnRP6/pI+qOkJyQd1tBOJI2SNEPSjAULFtSg2mZmVk4tAkO5d/6ld0gayjMf2Dki9gMuAO6QtHm5nUTEuIgYEBEDevbsWVWFzcysYbX4VFIdsFNuuRfwQQN56iR1BrYAFkVEAF8ARMSLkt4Gdgdm1KBetpa11icxYO19GsPManPF8ALQV1IfSRsCI4DJJXkmA6en+ZOBqRERknqmm9dI2hXoC8ytQZ3MzKyFqr5iiIiVks4DpgCdgJsi4jVJlwIzImIycCPwG0lzgEVkwQPgcOBSSSuBVcC5EbGo2jqZmVnL1eQLbhHxAPBASdpPcvPLgeFlyt0L3FuLOpiZWW34JzHMzKzAgcHMzAocGMzMrMCBwczMCvzrqmbro9b8RZrwd046Ol8xmJlZgQODmZkVODCYmVmBA4OZmRU4MJiZWYE/lWRmNeVf2e34fMVgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBTUJDJKGSnpT0hxJo8us30jSXWn9c5J659ZdnNLflDSkFvUxM7OWqzowSOoEXAMcC+wFnCJpr5JsZwGLI2I34Ergl6nsXsAIYG9gKHBt2p6ZmbWRWlwxHAjMiYi5EfEX4E7ghJI8JwDj0/xE4ChJSul3RsQXEfEOMCdtz8zM2kjnGmxjR+C93HIdcFBDeSJipaQlQPeU/mxJ2R3L7UTSKGAUwM4779zy2kotL9uEaY+3znYHDYrabjBqvL1kUKtstRV0wD4ANe4HrdQHoAP1A7dBg2pxxVDuVVba4g3lqaRslhgxLiIGRMSAnj17NrOKZmZWqVoEhjpgp9xyL+CDhvJI6gxsASyqsKyZma1FtQgMLwB9JfWRtCHZzeTJJXkmA6en+ZOBqRERKX1E+tRSH6Av8HwN6mRmZi1U9T2GdM/gPGAK0Am4KSJek3QpMCMiJgM3Ar+RNIfsSmFEKvuapLuB14GVwHcjYlW1dTIzs5arxc1nIuIB4IGStJ/k5pcDwxso+3Pg57Woh5mZVc/ffDYzswIHBjMzK3BgMDOzAgcGMzMrcGAwM7MCBwYzMytwYDAzswIHBjMzK3BgMDOzAgcGMzMrcGCw9U9E601m6wAHBjMzK3BgMDOzAgcGMzMrcGAwM7MCBwYzMytwYDAzswIHBjMzK3BgMDOzgpo889nMMoMG+Utu1vH5isHMzAocGMzMrMCBwczMChwYzMyswIHBzMwKHBjMzKzAgcHMzAocGMzMrMCBwczMChwYzMyswIHBzMwKHBjMzKzAgcHMzAocGMzMrKCqwCBpa0mPSJqd/m7VQL7TU57Zkk7PpU+T9Kakl9K0TTX1MTOz6lV7xTAaeCwi+gKPpeUCSVsDlwAHAQcCl5QEkFMjol+aPqqyPmZmVqVqA8MJwPg0Px44sUyeIcAjEbEoIhYDjwBDq9yvmZm1kmoDw7YRMR8g/S03FLQj8F5uuS6l1bs5DSP9WJIa2pGkUZJmSJqxYMGCKqttZmYNafLRnpIeBbYrs+pHFe6j3Mm+/vmHp0bE+5I2A+4FTgNuLbeRiBgHjAMYMGCAn59oZtZKmgwMEXF0Q+skfShp+4iYL2l7oNw9gjpgUG65FzAtbfv99PczSXeQ3YMoGxjMzGztqHYoaTJQ/ymj04HflckzBRgsaat003kwMEVSZ0k9ACR1AY4HXq2yPmZmVqVqA8MvgGMkzQaOSctIGiDpBoCIWAT8DHghTZemtI3IAsQrwEvA+8B/VFkfMzOrUpNDSY2JiIXAUWXSZwBn55ZvAm4qybMM2L+a/ZuZWe35m89mZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlbgwGBmZgUODGZmVuDAYGZmBQ4MZmZW4MBgZmYFDgxmZlaw/gWGiNabzMzWAetfYDAzs0Y5MJiZWUFVgUHS1pIekTQ7/d2qgXwPSfpE0v0l6X0kPZfK3yVpw2rqY2Zm1etcZfnRwGMR8QtJo9PyRWXyXQFsDHyrJP2XwJURcaekscBZwHVV1qnNDBrk+wxm1vFVO5R0AjA+zY8HTiyXKSIeAz7Lp0kScCQwsanyZma29lQbGLaNiPkA6e82zSjbHfgkIlam5Tpgx4YySxolaYakGQsWLGhxhc3MrHFNDiVJehTYrsyqH1W5b5VJa3AsJiLGAeMABgwY4DEbM7NW0mRgiIijG1on6UNJ20fEfEnbAx81Y98fA1tK6pyuGnoBHzSjvJmZtYJqh5ImA6en+dOB31VaMCICeBw4uSXlzcysdVQbGH4BHCNpNnBMWkbSAEk31GeS9AfgHuAoSXWShqRVFwEXSJpDds/hxirrY2ZmVarq46oRsRA4qkz6DODs3PJhDZSfCxxYTR3MzKy2/M1nMzMrcGAwM7MCRQf8VVBJC4B327oeZfQg+7TV+mp9P35wG4DbANpvG+wSET2bytQhA0N7JWlGRAxo63q0lfX9+MFtAG4D6Pht4KEkMzMrcGAwM7MCB4baGtfWFWhj6/vxg9sA3AbQwdvA9xjMzKzAVwxmZlbgwGBmZgUODBWQNFTSm5LmpCfVlcuzTj2+VNJNkj6S9GourdJHud6e2uvVtJ0uKV2Srkrt+Iqk/mvreJpL0k6SHpc0S9Jrkr6f0itqg9x2rpa0NLe8UeoHc1K/6N26R1I9SZ0k/bG+b1fapyVtKGmcpLckvSHp6ym93beBpC0lTUz1niVpYDP6/3np2EJSj1z6IElLJL2Upp/k1jV5jlmbHBiaIKkTcA1wLLAXcIqkvcpkvQI4rUx6/eNL+wKLyR5f2hHcAgwtSat/lGtf4LG0XM7twB7A3wDd+OvvZh0L9E3TKNr3Y1xXAj+IiD2Bg4Hvpv97pW2ApAHAliXJZwGLI2I34Eqy/tHefR+YlVuutE//CPgoInYne+08kdI7Qhv8X+ChiNgD2Jfs+Cv93z8FHE35L+H+ISL6pelSaNY5Zu2JCE+NTMBAYEpu+WLg4gbyDgLuzy2L7NuPncttq71PQG/g1dzym8D2aX574M0KtvFPwM/T/PXAKeW2194nsp+EP6bSNgA6kf2s/PbA0lz6FGBgmu+c+ofa+vgaOe5eZCfBI4H7m9OngfeATcqkt+s2ADYH3imtU3P7PzAP6JFbLpwfcukVn2PW1uQrhqbtSNbB6zX6CNISzXp8aQfQrEe5piGk04CHUlI1bdlm0lDHfsBzVN4G5wGT6/PmrG6D1C+WkPWT9urXwA+BL9NyRX1aUv2V0s8kzZR0j6RtU1p7b4NdgQXAzWkI7QZJm1Ddo4zrDZT0sqQHJe2d0trd68KBoWnNegRpDcuuC64FnoyIP6TlDtcekjYF7gXOj4hPKyyzAzAcuLrc6jJp7bINJB1PNhT0Yj65TNZy9e9MdrXxVET0B54BftXMbbSVzkB/4LqI2A9YRiNDhs0wk+y3ivYl6xuTUnq7aw8HhqbVATvllnsBH+duIA1rpOzqx5fmynbkx5d+qOwRrij3KFdJU1Jb5B/OdAnQE7ggV75cW7bb9khXPPcCt0fEb1NyJW2wH7AbMEfSPGBjZQ+jglwbpH6xBbBoLR1Scx0CDEvHcCfZcNKvKdOn0w3q+tfEpcBC4M/AfSnfPWQnW2j/bVAH1EXEc2l5IlndK+7/5UTEpxGxNM0/AHRJN6fb3+uircfz2vtE9u5hLtAH2BB4Gdi7gbyDKBlDJHtBjEjzY4HvtPUxNePYe1O8x3AFMDrNjwYub6Dc2cDTQLeS9P8BPEj2Dulg4Pm2PsZGjl3ArcCvS9IraoOSMvl7DN8Fxqb5EcDdbX2sFbbH6r5daZ8mBZM0fwZwT0dpA+APwFfT/Jj0f2/W/5417zFsx1+/VHwg8F+pn1V8jllrx9/W/4COMAHHAW8BbwM/aqQjLQA+J3sHMCSl7wo8D8xJL6iN2vp4KjzmCcB8YEU6nrPIxoEfA2anv1s3UHZlaquX0vSTlC6yT1+8DfwnMKCtj7OR4z+U7HL+ldxxHFdpG5RsKx8YuqZ+MCf1i13b+lgrbI98YKioTwO7AE+mNnwM2LmjtAHQD5iR6j4J2KoZ/f8f02tmJdk7/xtS+nnAa+nE/yzwt7kyTZ5j1ubkn8QwM7MC32MwM7MCBwYzMytwYDAzswIHBjMzK3BgMDOzAgcGMzMrcGAwM7OC/wYN0pQbIk9/rgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Metrics sliced by age\n",
    "age_range_intervals = [0, 10, 20, 40, 60, 150]\n",
    "nodebiasing_perf = []\n",
    "debiasing_perf = []\n",
    "\n",
    "for idx in range(len(age_range_intervals)-1):\n",
    "    start = age_range_intervals[idx]\n",
    "    end = age_range_intervals[idx+1]\n",
    "    ids = np.where((age_test>start) & (age_test<end))\n",
    "    true_dataset = dataset_wrapper(outcome=y_test[ids], protected=p_test[ids],\n",
    "                                   unprivileged_groups=unprivileged_groups,\n",
    "                                   privileged_groups=privileged_groups,\n",
    "                                   favorable_label=favorable_label,\n",
    "                                unfavorable_label=unfavorable_label)\n",
    "    transf_pred_dataset = dataset_wrapper(outcome=y_pred_transf[ids], protected=p_test[ids],\n",
    "                                          unprivileged_groups=unprivileged_groups,\n",
    "                                          privileged_groups=privileged_groups,\n",
    "                                          favorable_label=favorable_label,\n",
    "                                          unfavorable_label=unfavorable_label)\n",
    "    pred_dataset = dataset_wrapper(outcome=y_pred[ids], protected=p_test[ids],\n",
    "                                   unprivileged_groups=unprivileged_groups,\n",
    "                                   privileged_groups=privileged_groups,\n",
    "                                   favorable_label=favorable_label,\n",
    "                                   unfavorable_label=unfavorable_label)\n",
    " \n",
    "    classified_metric_nodebiasing_test = ClassificationMetric(true_dataset, \n",
    "                                                 pred_dataset,\n",
    "                                                 unprivileged_groups=unprivileged_groups,\n",
    "                                                 privileged_groups=privileged_groups)\n",
    "    classified_metric_debiasing_test = ClassificationMetric(true_dataset, \n",
    "                                                 transf_pred_dataset,\n",
    "                                                 unprivileged_groups=unprivileged_groups,\n",
    "                                                 privileged_groups=privileged_groups)\n",
    "    nodebiasing_perf.append(classified_metric_nodebiasing_test.equal_opportunity_difference())\n",
    "    debiasing_perf.append(classified_metric_debiasing_test.equal_opportunity_difference())\n",
    "\n",
    "N = len(age_range_intervals)-1\n",
    "fig, ax = plt.subplots()\n",
    "ind = np.arange(N)\n",
    "width = 0.35\n",
    "p1 = ax.bar(ind, nodebiasing_perf, width, color='r')\n",
    "p2 = ax.bar(ind + width, debiasing_perf, width,\n",
    "            color='y')\n",
    "ax.set_title('Equal opportunity difference by age group')\n",
    "ax.set_xticks(ind + width / 2)\n",
    "ax.set_xticklabels([str(age_range_intervals[idx])+'-'+str(age_range_intervals[idx+1]) for idx in range(N)])\n",
    "\n",
    "ax.legend((p1[0], p2[0]), ('Before', 'After'))\n",
    "ax.autoscale_view()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusions\n",
    "In this tutorial, we have examined fairness in the scenario of binary classification with face images. We discussed methods to process several attributes of images, outcome variables, and protected attributes and create aif360 ready dataset objects on which many bias mititation algorithms can be easily applied and fairness metrics can be easliy computed. We used the reweighing algorithm with the aim of improving the algorithmic fairness of the learned classifiers. The empirical results show slight improvement in the case of debiased model over the vanilla model. When sliced by age group, the results appear to be mixed bag and thus has scope for further improvements by considering age group while learning models. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}