Fraud_Prediction_P3_notebook.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.10"
    },
    "colab": {
      "name": "Fraud Prediction - P3 notebook.ipynb",
      "provenance": [],
      "toc_visible": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yz1kCIRiQmzO",
        "colab_type": "text"
      },
      "source": [
        "# **Note That This Notebook is Generated Using IBM Watson AutoAI Experiments**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "OWoLUw38DI2I",
        "colab_type": "text"
      },
      "source": [
        "################################################################################\n",
        "#Licensed Materials - Property of IBM\n",
        "#(C) Copyright IBM Corp. 2020\n",
        "#US Government Users Restricted Rights - Use, duplication disclosure restricted\n",
        "#by GSA ADP Schedule Contract with IBM Corp.\n",
        "################################################################################\n",
        "\n",
        "The auto-generated notebooks are subject to the International License Agreement for Non-Warranted Programs (or equivalent) and License Information document for Watson Studio Auto-generated Notebook (License Terms), such agreements located in the link below.\n",
        "Specifically, the Source Components and Sample Materials clause included in the License Information document for\n",
        "Watson Studio Auto-generated Notebook applies to the auto-generated notebooks. \n",
        "By downloading, copying, accessing, or otherwise using the materials, you agree to the License Terms.\n",
        "http://www14.software.ibm.com/cgi-bin/weblap/lap.pl?li_formnum=L-AMCU-BHU2B7&title=IBM%20Watson%20Studio%20Auto-generated%20Notebook%20V2.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": false,
        "pycharm": {
          "name": "#%% md\n"
        },
        "id": "NTW4BAQsDI2L",
        "colab_type": "text"
      },
      "source": [
        "### IBM AutoAI Auto-Generated Notebook v1.13.1\n",
        "\n",
        "**Note:** Notebook code generated using AutoAI will execute successfully. If code is modified or reordered,   \n",
        "there is no guarantee it will successfully execute. This pipeline is optimized for the original dataset.  \n",
        "The pipeline may fail or produce sub-optimium results if used with different data. For different data,  \n",
        "please consider returning to AutoAI Experiments to generate a new pipeline. Please read our documentation   \n",
        "for more information:   \n",
        "(Cloud Platform) https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/autoai-notebook.html .\n",
        "(Cloud Pak For Data) https://www.ibm.com/support/knowledgecenter/SSQNUZ_3.0.0/wsj/analyze-data/autoai-notebook.html .\n",
        "\n",
        "\n",
        "Before modifying the pipeline or trying to re-fit the pipeline, consider:   \n",
        "The notebook converts dataframes to numpy arrays before fitting the pipeline   \n",
        "(a current restriction of the preprocessor pipeline). The known_values_list is passed by reference   \n",
        "and populated with categorical values during fit of the preprocessing pipeline. Delete its members before re-fitting."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hsJZRDpYDI2O",
        "colab_type": "text"
      },
      "source": [
        "### Representing Pipeline_3 \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E1VSSiZqDI2P",
        "colab_type": "text"
      },
      "source": [
        "### 1. Set Up\n",
        "If lightgbm or xgboost installation fails, please follow:\n",
        " - [lightgbm docs](https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html)\n",
        " - [xgboost docs](https://xgboost.readthedocs.io/en/latest/build.html)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OmSLH8LkDI2R",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "outputId": "20b20c59-2d78-420e-f3ca-4a9d9d460d5c"
      },
      "source": [
        "\n",
        "try:\n",
        "    import autoai_libs\n",
        "except Exception as e:\n",
        "    import subprocess\n",
        "    out = subprocess.check_output('pip install autoai-libs'.split(' '))\n",
        "    for line in out.splitlines():\n",
        "        print(line)\n",
        "    import autoai_libs\n",
        "import sklearn\n",
        "try:\n",
        "    import xgboost\n",
        "except:\n",
        "    print('xgboost, if needed, will be installed and imported later')\n",
        "try:\n",
        "    import lightgbm\n",
        "except:\n",
        "    print('lightgbm, if needed, will be installed and imported later')\n",
        "from sklearn.cluster import FeatureAgglomeration\n",
        "import numpy\n",
        "from numpy import inf, nan, dtype, mean\n",
        "from autoai_libs.sklearn.custom_scorers import CustomScorers\n",
        "import sklearn.ensemble\n",
        "from autoai_libs.cognito.transforms.transform_utils import TExtras, FC\n",
        "from autoai_libs.transformers.exportable import *\n",
        "from autoai_libs.utils.exportable_utils import *\n",
        "from sklearn.pipeline import Pipeline\n",
        "known_values_list=[]"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "b'Collecting autoai-libs'\n",
            "b'  Downloading https://files.pythonhosted.org/packages/08/79/2e418d224b1da532f0e98e80c62c874363105a5d1eac2d3fb6a8e6b273ef/autoai_libs-1.11.1-40-cp36-cp36m-manylinux1_x86_64.whl (4.1MB)'\n",
            "b'Requirement already satisfied: numpy>=1.16.4 in /usr/local/lib/python3.6/dist-packages (from autoai-libs) (1.18.5)'\n",
            "b'Requirement already satisfied: scikit-learn<0.24.0,>=0.20.3 in /usr/local/lib/python3.6/dist-packages (from autoai-libs) (0.22.2.post1)'\n",
            "b'Collecting pandas<1.0.0,>=0.24.2'\n",
            "b'  Downloading https://files.pythonhosted.org/packages/52/3f/f6a428599e0d4497e1595030965b5ba455fd8ade6e977e3c819973c4b41d/pandas-0.25.3-cp36-cp36m-manylinux1_x86_64.whl (10.4MB)'\n",
            "b'Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn<0.24.0,>=0.20.3->autoai-libs) (0.16.0)'\n",
            "b'Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.6/dist-packages (from scikit-learn<0.24.0,>=0.20.3->autoai-libs) (1.4.1)'\n",
            "b'Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas<1.0.0,>=0.24.2->autoai-libs) (2018.9)'\n",
            "b'Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.6/dist-packages (from pandas<1.0.0,>=0.24.2->autoai-libs) (2.8.1)'\n",
            "b'Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.6.1->pandas<1.0.0,>=0.24.2->autoai-libs) (1.15.0)'\n",
            "b'Installing collected packages: pandas, autoai-libs'\n",
            "b'  Found existing installation: pandas 1.0.5'\n",
            "b'    Uninstalling pandas-1.0.5:'\n",
            "b'      Successfully uninstalled pandas-1.0.5'\n",
            "b'Successfully installed autoai-libs-1.11.1 pandas-0.25.3'\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2tjbqPApDI2d",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# compose a decorator to assist pipeline instantiation via import of modules and installation of packages\n",
        "def decorator_retries(func):\n",
        "    def install_import_retry(*args, **kwargs):\n",
        "        retries = 0\n",
        "        successful = False\n",
        "        failed_retries = 0\n",
        "        while retries < 100 and failed_retries < 10 and not successful:\n",
        "            retries += 1\n",
        "            failed_retries += 1\n",
        "            try:\n",
        "                result = func(*args, **kwargs)\n",
        "                successful = True\n",
        "            except Exception as e:\n",
        "                estr = str(e)\n",
        "                if estr.startswith('name ') and estr.endswith(' is not defined'):\n",
        "                    try:\n",
        "                        import importlib\n",
        "                        module_name = estr.split(\"'\")[1]\n",
        "                        module = importlib.import_module(module_name)\n",
        "                        globals().update({module_name: module})\n",
        "                        print('import successful for ' + module_name)\n",
        "                        failed_retries -= 1\n",
        "                    except Exception as import_failure:\n",
        "                        print('import of ' + module_name + ' failed with: ' + str(import_failure))\n",
        "                        import subprocess\n",
        "                        if module_name == 'lightgbm':\n",
        "                            try:\n",
        "                                print('attempting pip install of ' + module_name)\n",
        "                                process = subprocess.Popen('pip install ' + module_name, shell=True)\n",
        "                                process.wait()\n",
        "                            except Exception as E:\n",
        "                                print(E)\n",
        "                                try:\n",
        "                                    import sys\n",
        "                                    print('attempting conda install of ' + module_name)\n",
        "                                    process = subprocess.Popen('conda install --yes --prefix {sys.prefix} -c powerai ' + module_name, shell = True)\n",
        "                                    process.wait()\n",
        "                                except Exception as lightgbm_installation_error:\n",
        "                                    print('lightgbm installation failed!' + lightgbm_installation_error)\n",
        "                        else:\n",
        "                            print('attempting pip install of ' + module_name)\n",
        "                            process = subprocess.Popen('pip install ' + module_name, shell=True)\n",
        "                            process.wait()\n",
        "                        try:\n",
        "                            print('re-attempting import of ' + module_name)\n",
        "                            module = importlib.import_module(module_name)\n",
        "                            globals().update({module_name: module})\n",
        "                            print('import successful for ' + module_name)\n",
        "                            failed_retries -= 1\n",
        "                        except Exception as import_or_installation_failure:\n",
        "                            print('failure installing and/or importing ' + module_name + ' error was: ' + str(\n",
        "                                import_or_installation_failure))\n",
        "                            raise (ModuleNotFoundError('Missing package in environment for ' + module_name +\n",
        "                                                       '? Try import and/or pip install manually?'))\n",
        "                elif type(e) is AttributeError:\n",
        "                    if 'module ' in estr and ' has no attribute ' in estr:\n",
        "                        pieces = estr.split(\"'\")\n",
        "                        if len(pieces) == 5:\n",
        "                            try:\n",
        "                                import importlib\n",
        "                                print('re-attempting import of ' + pieces[3] + ' from ' + pieces[1])\n",
        "                                module = importlib.import_module('.' + pieces[3], pieces[1])\n",
        "                                failed_retries -= 1\n",
        "                            except:\n",
        "                                print('failed attempt to import ' + pieces[3])\n",
        "                                raise (e)\n",
        "                        else:\n",
        "                            raise (e)\n",
        "                else:\n",
        "                    raise (e)\n",
        "        if successful:\n",
        "            print('Pipeline successfully instantiated')\n",
        "        else:\n",
        "            raise (ModuleNotFoundError(\n",
        "                'Remaining missing imports/packages in environment? Retry cell and/or try pip install manually?'))\n",
        "        return result\n",
        "    return install_import_retry\n"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2KB-anMYDI2i",
        "colab_type": "text"
      },
      "source": [
        "### 2. Compose Pipeline"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nYONLTNSDI2j",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105
        },
        "outputId": "37a6bf5c-e758-41f7-c8ef-8ecedb6a8e4d"
      },
      "source": [
        "# metadata necessary to replicate AutoAI scores with the pipeline\n",
        "_input_metadata = {'separator': ',', 'excel_sheet': 0, 'target_label_name': 'Fraud_Risk', 'learning_type': 'classification', 'subsampling': None, 'pos_label': 1, 'pn': 'P3', 'cv_num_folds': 3, 'holdout_fraction': 0.1, 'optimization_metric': 'accuracy', 'random_state': 33, 'data_source': ''}\n",
        "\n",
        "# define a function to compose the pipeline, and invoke it\n",
        "@decorator_retries\n",
        "def compose_pipeline():\n",
        "    import numpy\n",
        "    from numpy import nan, dtype, mean\n",
        "    #\n",
        "    # composing steps for toplevel Pipeline\n",
        "    #\n",
        "    _input_metadata = {'separator': ',', 'excel_sheet': 0, 'target_label_name': 'Fraud_Risk', 'learning_type': 'classification', 'subsampling': None, 'pos_label': 1, 'pn': 'P3', 'cv_num_folds': 3, 'holdout_fraction': 0.1, 'optimization_metric': 'accuracy', 'random_state': 33, 'data_source': ''}\n",
        "    steps = []\n",
        "    #\n",
        "    # composing steps for preprocessor Pipeline\n",
        "    #\n",
        "    preprocessor__input_metadata = None\n",
        "    preprocessor_steps = []\n",
        "    #\n",
        "    # composing steps for preprocessor_features FeatureUnion\n",
        "    #\n",
        "    preprocessor_features_transformer_list = []\n",
        "    #\n",
        "    # composing steps for preprocessor_features_categorical Pipeline\n",
        "    #\n",
        "    preprocessor_features_categorical__input_metadata = None\n",
        "    preprocessor_features_categorical_steps = []\n",
        "    preprocessor_features_categorical_steps.append(('cat_column_selector', autoai_libs.transformers.exportable.NumpyColumnSelector(columns=[0, 1, 2, 3, 4, 8, 9, 10, 11])))\n",
        "    preprocessor_features_categorical_steps.append(('cat_compress_strings', autoai_libs.transformers.exportable.CompressStrings(activate_flag=True, compress_type='hash', dtypes_list=['int_num', 'int_num', 'int_num', 'int_num', 'int_num', 'int_num', 'int_num', 'int_num', 'int_num'], missing_values_reference_list=['', '-', '?', nan], misslist_list=[[], [], [], [], [], [], [], [], []])))\n",
        "    preprocessor_features_categorical_steps.append(('cat_missing_replacer', autoai_libs.transformers.exportable.NumpyReplaceMissingValues(filling_values=nan, missing_values=[])))\n",
        "    preprocessor_features_categorical_steps.append(('cat_unknown_replacer', autoai_libs.transformers.exportable.NumpyReplaceUnknownValues(filling_values=nan, filling_values_list=[nan, nan, nan, nan, nan, nan, nan, nan, nan], known_values_list=[[0, 1], [0, 1], [0, 1, 2, 3], [0, 1], [0, 1], [12, 14, 36, 60, 68, 84, 107, 109, 120, 159, 160, 178, 180, 197, 214, 215, 218, 240, 250, 281, 295, 296, 300, 306, 316, 323, 324, 328, 337, 338, 341, 343, 346, 360, 404, 418, 421, 459, 460, 465, 466, 476, 480], [0, 1], [0, 1], [1, 2, 3]], missing_values_reference_list=['', '-', '?', nan])))\n",
        "    preprocessor_features_categorical_steps.append(('boolean2float_transformer', autoai_libs.transformers.exportable.boolean2float(activate_flag=True)))\n",
        "    preprocessor_features_categorical_steps.append(('cat_imputer', autoai_libs.transformers.exportable.CatImputer(activate_flag=True, missing_values=nan, sklearn_version_family='20', strategy='most_frequent')))\n",
        "    preprocessor_features_categorical_steps.append(('cat_encoder', autoai_libs.transformers.exportable.CatEncoder(activate_flag=True, categories='auto', dtype=numpy.float64, encoding='ordinal', handle_unknown='error', sklearn_version_family='20')))\n",
        "    preprocessor_features_categorical_steps.append(('float32_transformer', autoai_libs.transformers.exportable.float32_transform(activate_flag=True)))\n",
        "    # assembling preprocessor_features_categorical_ Pipeline\n",
        "    preprocessor_features_categorical_pipeline = sklearn.pipeline.Pipeline(steps=preprocessor_features_categorical_steps)\n",
        "    preprocessor_features_transformer_list.append(('categorical', preprocessor_features_categorical_pipeline))\n",
        "    #\n",
        "    # composing steps for preprocessor_features_numeric Pipeline\n",
        "    #\n",
        "    preprocessor_features_numeric__input_metadata = None\n",
        "    preprocessor_features_numeric_steps = []\n",
        "    preprocessor_features_numeric_steps.append(('num_column_selector', autoai_libs.transformers.exportable.NumpyColumnSelector(columns=[5, 6, 7])))\n",
        "    preprocessor_features_numeric_steps.append(('num_floatstr2float_transformer', autoai_libs.transformers.exportable.FloatStr2Float(activate_flag=True, dtypes_list=['int_num', 'int_num', 'int_num'], missing_values_reference_list=[])))\n",
        "    preprocessor_features_numeric_steps.append(('num_missing_replacer', autoai_libs.transformers.exportable.NumpyReplaceMissingValues(filling_values=nan, missing_values=[])))\n",
        "    preprocessor_features_numeric_steps.append(('num_imputer', autoai_libs.transformers.exportable.NumImputer(activate_flag=True, missing_values=nan, strategy='median')))\n",
        "    preprocessor_features_numeric_steps.append(('num_scaler', autoai_libs.transformers.exportable.OptStandardScaler(num_scaler_copy=None, num_scaler_with_mean=None, num_scaler_with_std=None, use_scaler_flag=False)))\n",
        "    preprocessor_features_numeric_steps.append(('float32_transformer', autoai_libs.transformers.exportable.float32_transform(activate_flag=True)))\n",
        "    # assembling preprocessor_features_numeric_ Pipeline\n",
        "    preprocessor_features_numeric_pipeline = sklearn.pipeline.Pipeline(steps=preprocessor_features_numeric_steps)\n",
        "    preprocessor_features_transformer_list.append(('numeric', preprocessor_features_numeric_pipeline))\n",
        "    # assembling preprocessor_features_ FeatureUnion\n",
        "    preprocessor_features_pipeline = sklearn.pipeline.FeatureUnion(transformer_list=preprocessor_features_transformer_list)\n",
        "    preprocessor_steps.append(('features', preprocessor_features_pipeline))\n",
        "    preprocessor_steps.append(('permuter', autoai_libs.transformers.exportable.NumpyPermuteArray(axis=0, permutation_indices=[0, 1, 2, 3, 4, 8, 9, 10, 11, 5, 6, 7])))\n",
        "    # assembling preprocessor_ Pipeline\n",
        "    preprocessor_pipeline = sklearn.pipeline.Pipeline(steps=preprocessor_steps)\n",
        "    steps.append(('preprocessor', preprocessor_pipeline))\n",
        "    #\n",
        "    # composing steps for cognito Pipeline\n",
        "    #\n",
        "    cognito__input_metadata = None\n",
        "    cognito_steps = []\n",
        "    cognito_steps.append(('0', autoai_libs.cognito.transforms.transform_utils.TA1(fun=numpy.square, name='square', datatypes=['numeric'], feat_constraints=[autoai_libs.utils.fc_methods.is_not_categorical], tgraph=None, apply_all=True, col_names=['Gender', 'Married', 'Dependents', 'Education', 'Self_Employed', 'ApplicantIncome', 'CoapplicantIncome', 'LoanAmount', 'Loan_Term', 'Credit_History_Available', 'Housing', 'Locality'], col_dtypes=[dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32')], col_as_json_objects=None)))\n",
        "    cognito_steps.append(('1', autoai_libs.cognito.transforms.transform_utils.FS1(cols_ids_must_keep=range(0, 12), additional_col_count_to_keep=12, ptype='classification')))\n",
        "    cognito_steps.append(('2', autoai_libs.cognito.transforms.transform_utils.TAM(tans_class=sklearn.decomposition.pca.PCA(copy=True, iterated_power='auto', n_components=None, random_state=None, svd_solver='auto', tol=0.0, whiten=False), name='pca', tgraph=None, apply_all=True, col_names=['Gender', 'Married', 'Dependents', 'Education', 'Self_Employed', 'ApplicantIncome', 'CoapplicantIncome', 'LoanAmount', 'Loan_Term', 'Credit_History_Available', 'Housing', 'Locality', 'square(ApplicantIncome)', 'square(CoapplicantIncome)', 'square(LoanAmount)', 'square(Loan_Term)'], col_dtypes=[dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32')], col_as_json_objects=None)))\n",
        "    cognito_steps.append(('3', autoai_libs.cognito.transforms.transform_utils.FS1(cols_ids_must_keep=range(0, 12), additional_col_count_to_keep=12, ptype='classification')))\n",
        "    cognito_steps.append(('4', autoai_libs.cognito.transforms.transform_utils.TA1(fun=numpy.sin, name='sin', datatypes=['float'], feat_constraints=[autoai_libs.utils.fc_methods.is_not_categorical], tgraph=None, apply_all=True, col_names=['Gender', 'Married', 'Dependents', 'Education', 'Self_Employed', 'ApplicantIncome', 'CoapplicantIncome', 'LoanAmount', 'Loan_Term', 'Credit_History_Available', 'Housing', 'Locality', 'square(LoanAmount)', 'square(Loan_Term)', 'pca_2', 'pca_4', 'pca_5', 'pca_6', 'pca_7', 'pca_8', 'pca_12', 'pca_13', 'pca_14', 'pca_15'], col_dtypes=[dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32')], col_as_json_objects=None)))\n",
        "    cognito_steps.append(('5', autoai_libs.cognito.transforms.transform_utils.FS1(cols_ids_must_keep=range(0, 12), additional_col_count_to_keep=12, ptype='classification')))\n",
        "    # assembling cognito_ Pipeline\n",
        "    cognito_pipeline = sklearn.pipeline.Pipeline(steps=cognito_steps)\n",
        "    steps.append(('cognito', cognito_pipeline))\n",
        "    steps.append(('estimator', lightgbm.sklearn.LGBMClassifier(boosting_type='gbdt', class_weight='balanced', colsample_bytree=0.9775643042194456, importance_type='split', learning_rate=0.02168565444468173, max_depth=-1, min_child_samples=30, min_child_weight=0.0015226421960632711, min_split_gain=0.0, n_estimators=381, n_jobs=1, num_leaves=2, objective=None, random_state=33, reg_alpha=0.9934190174797073, reg_lambda=0.045733276980655324, silent=True, subsample=0.044454290568242016, subsample_for_bin=200000, subsample_freq=0)))\n",
        "    # assembling  Pipeline\n",
        "    pipeline = sklearn.pipeline.Pipeline(steps=steps)\n",
        "    return pipeline\n",
        "pipeline = compose_pipeline()\n"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "re-attempting import of pca from sklearn.decomposition\n",
            "Pipeline successfully instantiated\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.decomposition.pca module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.decomposition. Anything that cannot be imported from sklearn.decomposition is now part of the private API.\n",
            "  warnings.warn(message, FutureWarning)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jjTeoo7nDI2v",
        "colab_type": "text"
      },
      "source": [
        "### 3. Extract needed parameter values from AutoAI run metadata"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "21LpxPSsDI2w",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "# Metadata used in retrieving data and computing metrics.  Customize as necessary for your environment.\n",
        "#data_source='replace_with_path_and_csv_filename'\n",
        "target_label_name = _input_metadata['target_label_name']\n",
        "learning_type = _input_metadata['learning_type']\n",
        "optimization_metric = _input_metadata['optimization_metric']\n",
        "random_state = _input_metadata['random_state']\n",
        "cv_num_folds = _input_metadata['cv_num_folds']\n",
        "holdout_fraction = _input_metadata['holdout_fraction']\n",
        "if 'data_provenance' in _input_metadata:\n",
        "    data_provenance = _input_metadata['data_provenance']\n",
        "else:\n",
        "    data_provenance = None\n",
        "if 'pos_label' in _input_metadata and learning_type == 'classification':\n",
        "    pos_label = _input_metadata['pos_label']\n",
        "else:\n",
        "    pos_label = None\n"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lmvefzBlDI3D",
        "colab_type": "text"
      },
      "source": [
        "### 4. Create dataframe from dataset in Cloud Object Storage OR manually load csv if running notebook in a local machine"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O0Dq1xnaQWG7",
        "colab_type": "text"
      },
      "source": [
        "*****************For Cloud Storage Object"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eDsEUVL_DI3E",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "# @hidden_cell\n",
        "# The following code contains the credentials for a file in your IBM Cloud Object Storage.\n",
        "# You might want to remove those credentials before you share your notebook.\n",
        "#(removing credentials woth xx)\n",
        "credentials_0 = {\n",
        "    'ENDPOINT': 'xxx',\n",
        "    'IBM_AUTH_ENDPOINT': 'xx',\n",
        "    'APIKEY': 'xx',\n",
        "    'BUCKET': 'xx',\n",
        "    'FILE': 'fraud_dataset.csv',\n",
        "    'SERVICE_NAME': 's3',\n",
        "    'ASSET_ID': '1',\n",
        "    }\n"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "92LGrdIHDI3N",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        },
        "outputId": "6cb2485e-ff2d-4aa3-842f-874bce718079"
      },
      "source": [
        "#  Read the data as a dataframe\n",
        "import pandas as pd\n",
        "\n",
        "csv_encodings=['UTF-8','Latin-1'] # supplement list of encodings as necessary for your data\n",
        "df = None\n",
        "readable = None  # if automatic detection fails, you can supply a filename here\n",
        "\n",
        "# First, obtain a readable object\n",
        "# Cloud Object Storage data access\n",
        "# Assumes COS credentials are in a dictionary named 'credentials_0'\n",
        "   \n",
        "credentials = df = globals().get('credentials_0')       \n",
        "if readable is None and credentials is not None :\n",
        "    try:\n",
        "        import types\n",
        "        import pandas as pd\n",
        "        import io\n",
        "    except Exception as import_exception:\n",
        "        print('Error with importing packages - check if you installed them on your environment')\n",
        "    try:\n",
        "        if credentials['SERVICE_NAME'] == 's3':\n",
        "            try:\n",
        "                from botocore.client import Config\n",
        "                import ibm_boto3\n",
        "            except Exception as import_exception:\n",
        "                print('Installing required packages!')\n",
        "                !pip install ibm-cos-sdk\n",
        "                print('accessing data via Cloud Object Storage')\n",
        "            try:\n",
        "                client = ibm_boto3.client(service_name=credentials['SERVICE_NAME'],\n",
        "                                    ibm_api_key_id=credentials['APIKEY'],\n",
        "                                    ibm_auth_endpoint=credentials['IBM_AUTH_ENDPOINT'],\n",
        "                                    config=Config(signature_version='oauth'),\n",
        "                                    endpoint_url=credentials['ENDPOINT'])\n",
        "            except Exception as cos_exception:\n",
        "                print('unable to create client for cloud object storage')\n",
        "            try:\n",
        "                readable = client.get_object(Bucket=credentials['BUCKET'],Key=credentials['FILE'])['Body']\n",
        "                # add missing __iter__ method, so pandas accepts readable as file-like object\n",
        "                if not hasattr(readable, \"__iter__\"): readable.__iter__ = types.MethodType( __iter__, readable )\n",
        "            except Exception as cos_access_exception:\n",
        "                print('unable to access data object in cloud object storage with credentials supplied')\n",
        "        elif credentials['SERVICE_NAME'] == 'fs':\n",
        "            print('accessing data via File System')\n",
        "            try:\n",
        "                if credentials['FILE'].endswith('xlsx') or credentials['FILE'].endswith('xls'):\n",
        "                    df = pd.read_excel(credentials['FILE'])\n",
        "                else:\n",
        "                    df = pd.read_csv(credentials['FILE'], sep = _input_metadata['separator'])\n",
        "            except Exception as FS_access_exception:\n",
        "                print('unable to access data object in File System with path supplied') \n",
        "    except Exception as data_access_exception:\n",
        "        print('unable to access data object with credentials supplied') \n",
        "\n",
        "# IBM Cloud Pak for Data data access\n",
        "project_filename = globals().get('project_filename')       \n",
        "if readable is None and 'credentials_0' in globals() and 'ASSET_ID' in credentials_0:\n",
        "    project_filename = credentials_0['ASSET_ID']\n",
        "if project_filename != None and project_filename != '1':\n",
        "    print('attempting project_lib access to ' + str(project_filename))\n",
        "    try:\n",
        "        from project_lib import Project\n",
        "        project = Project.access()\n",
        "        storage_credentials = project.get_storage_metadata()\n",
        "        readable = project.get_file(project_filename)\n",
        "    except Exception as project_exception:\n",
        "        print('unable to access data using the project_lib interface and filename supplied')\n",
        "\n",
        "# Use data_provenance as filename if other access mechanisms are unsuccessful\n",
        "if readable is None and type(data_provenance) is str:\n",
        "    print('attempting to access local file using path and name ' + data_provenance)\n",
        "    readable = data_provenance\n",
        "\n",
        "# Second, use pd.read_csv to read object, iterating over list of csv_encodings until successful\n",
        "if readable is not None:\n",
        "    for encoding in csv_encodings:\n",
        "        try:\n",
        "            if credentials['FILE'].endswith('xlsx') or credentials['FILE'].endswith('xls'):\n",
        "                buffer = io.BytesIO(readable.read())\n",
        "                buffer.seek(0)\n",
        "                df = pd.read_excel(buffer, encoding=encoding,sheet_name=_input_metadata['excel_sheet'])\n",
        "            else:\n",
        "                df = pd.read_csv(readable, encoding = encoding, sep = _input_metadata['separator'])\n",
        "            print('successfully loaded dataframe using encoding = ' + str(encoding))\n",
        "            break\n",
        "        except Exception as exception_dataread:\n",
        "            print('unable to read csv using encoding ' + str(encoding))\n",
        "            print('handled error was ' + str(exception_dataread))\n",
        "    if df is None:\n",
        "        print('unable to read file/object as a dataframe using supplied csv_encodings ' + str(csv_encodings))\n",
        "        print(f'Please use \\'insert to code\\' on data panel to load dataframe.')\n",
        "        raise(ValueError('unable to read file/object as a dataframe using supplied csv_encodings ' + str(csv_encodings)))\n",
        "\n",
        "if isinstance(df,pd.DataFrame):\n",
        "    print('Data loaded succesfully')\n",
        "    if _input_metadata.get('subsampling') is not None:\n",
        "        df = df.sample(frac=_input_metadata['subsampling'], random_state=_input_metadata['random_state']) if _input_metadata['subsampling'] <= 1.0 else df.sample(n=_input_metadata['subsampling'], random_state=_input_metadata['random_state'])\n",
        "else:\n",
        "    print('Data cannot be loaded with credentials supplied, please provide DataFrame with training data.')"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "unable to read csv using encoding UTF-8\n",
            "handled error was 'StreamingBody' object has no attribute 'readable'\n",
            "unable to read csv using encoding Latin-1\n",
            "handled error was 'StreamingBody' object has no attribute 'readable'\n",
            "Data cannot be loaded with credentials supplied, please provide DataFrame with training data.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vtJyjy36QI6c",
        "colab_type": "text"
      },
      "source": [
        "*****************For locally importing csv"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2THTaH-DNc_s",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df=pd.read_csv(\"fraud_dataset.csv\")"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xuq8B29hPevR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "outputId": "9aaffddb-f2cb-44b6-af22-1923ecdd0a81"
      },
      "source": [
        "df.describe"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<bound method NDFrame.describe of      Gender  Married  Dependents  ...  Housing  Locality  Fraud_Risk\n",
              "0         1        0           0  ...        1         1           0\n",
              "1         1        1           1  ...        1         3           1\n",
              "2         1        1           0  ...        1         1           1\n",
              "3         1        1           0  ...        1         1           1\n",
              "4         1        0           0  ...        1         1           0\n",
              "..      ...      ...         ...  ...      ...       ...         ...\n",
              "822       0        0           1  ...        1         1           0\n",
              "823       0        0           1  ...        1         2           0\n",
              "824       1        0           1  ...        1         1           0\n",
              "825       1        0           2  ...        1         2           0\n",
              "826       1        0           0  ...        1         2           0\n",
              "\n",
              "[827 rows x 13 columns]>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r465fBVbDI3X",
        "colab_type": "text"
      },
      "source": [
        "### 5. Preprocess Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rw1cxSG8DI3a",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Drop rows whose target is not defined\n",
        "target = target_label_name # your target name here\n",
        "if learning_type == 'regression':\n",
        "    df[target] = pd.to_numeric(df[target], errors='coerce')\n",
        "df.dropna('rows', how='any', subset=[target], inplace=True)\n"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2CmPMqdPDI3j",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# extract X and y\n",
        "df_X = df.drop(columns=[target])\n",
        "df_y = df[target]\n"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jnJU_K60DI3o",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Detach preprocessing pipeline (which needs to see all training data)\n",
        "preprocessor_index = -1\n",
        "preprocessing_steps = [] \n",
        "for i, step in enumerate(pipeline.steps):\n",
        "    preprocessing_steps.append(step)\n",
        "    if step[0]=='preprocessor':\n",
        "        preprocessor_index = i\n",
        "        break\n",
        "#if len(pipeline.steps) > preprocessor_index+1 and pipeline.steps[preprocessor_index + 1][0] == 'cognito':\n",
        "    #preprocessor_index += 1\n",
        "    #preprocessing_steps.append(pipeline.steps[preprocessor_index])\n",
        "if preprocessor_index >= 0:\n",
        "    preprocessing_pipeline = Pipeline(memory=pipeline.memory, steps=preprocessing_steps)\n",
        "    pipeline = Pipeline(steps=pipeline.steps[preprocessor_index+1:])"
      ],
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Etlr4pXiDI3w",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Preprocess X\n",
        "# preprocessor should see all data for cross_validate on the remaining steps to match autoai scores\n",
        "known_values_list.clear()  #  known_values_list is filled in by the preprocessing_pipeline if needed\n",
        "preprocessing_pipeline.fit(df_X.values, df_y.values)\n",
        "X_prep = preprocessing_pipeline.transform(df_X.values)"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VnY5FLOADI34",
        "colab_type": "text"
      },
      "source": [
        "### 6. Split data into Training and Holdout sets"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lyuclksPDI35",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "77df22c6-6c3d-4faf-8ae8-b106e14c8fe6"
      },
      "source": [
        "# determine learning_type and perform holdout split (stratify conditionally)\n",
        "if learning_type is None:\n",
        "    # When the problem type is not available in the metadata, use the sklearn type_of_target to determine whether to stratify the holdout split\n",
        "    # Caution:  This can mis-classify regression targets that can be expressed as integers as multiclass, in which case manually override the learning_type\n",
        "    from sklearn.utils.multiclass import type_of_target\n",
        "    if type_of_target(df_y.values) in ['multiclass', 'binary']:\n",
        "        learning_type = 'classification'\n",
        "    else:\n",
        "        learning_type = 'regression'\n",
        "    print('learning_type determined by type_of_target as:',learning_type)\n",
        "else:\n",
        "    print('learning_type specified as:',learning_type)\n",
        "    \n",
        "from sklearn.model_selection import train_test_split\n",
        "if learning_type == 'classification':\n",
        "    X, X_holdout, y, y_holdout = train_test_split(X_prep, df_y.values, test_size=holdout_fraction, random_state=random_state, stratify=df_y.values)\n",
        "else:\n",
        "    X, X_holdout, y, y_holdout = train_test_split(X_prep, df_y.values, test_size=holdout_fraction, random_state=random_state)\n"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "learning_type specified as: classification\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QhpSoNiRDI4D",
        "colab_type": "text"
      },
      "source": [
        "#### 7. Generate features via Feature Engineering pipeline"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tfYVKAkwDI4F",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Detach Feature Engineering pipeline if next, fit it, and transform the training data\n",
        "fe_pipeline = None\n",
        "if pipeline.steps[0][0] == 'cognito':\n",
        "    try:\n",
        "        fe_pipeline = Pipeline(steps=[pipeline.steps[0]])\n",
        "        X = fe_pipeline.fit_transform(X, y)\n",
        "        X_holdout = fe_pipeline.transform(X_holdout)\n",
        "        pipeline.steps = pipeline.steps[1:]\n",
        "    except IndexError:\n",
        "        try:\n",
        "            print('Trying to compose pipeline with some of cognito steps')\n",
        "            fe_pipeline = Pipeline(steps = list([pipeline.steps[0][1].steps[0],pipeline.steps[0][1].steps[1]]))\n",
        "            X = fe_pipeline.fit_transform(X, y)\n",
        "            X_holdout = fe_pipeline.transform(X_holdout)\n",
        "            pipeline.steps = pipeline.steps[1:]\n",
        "        except IndexError:\n",
        "            print('Composing pipeline without cognito steps!')\n",
        "            pipeline.steps = pipeline.steps[1:]\n"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fIZoWQyMDI4J",
        "colab_type": "text"
      },
      "source": [
        " ### 8. Additional setup: Define a function that returns a scorer for the target's positive label"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "id": "SmFbUqDCDI4O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# create a function to produce a scorer for a given positive label\n",
        "def make_pos_label_scorer(scorer, pos_label):\n",
        "    kwargs = {'pos_label':pos_label}\n",
        "    for prop in ['needs_proba', 'needs_threshold']:\n",
        "        if prop+'=True' in scorer._factory_args():\n",
        "            kwargs[prop] = True\n",
        "    if scorer._sign == -1:\n",
        "        kwargs['greater_is_better'] = False\n",
        "    from sklearn.metrics import make_scorer\n",
        "    scorer=make_scorer(scorer._score_func, **kwargs)\n",
        "    return scorer"
      ],
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k_Us2_bqDI4h",
        "colab_type": "text"
      },
      "source": [
        "### 9. Fit pipeline, predict on Holdout set, calculate score, perform cross-validation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "pycharm": {
          "is_executing": false
        },
        "id": "xaq6ouMgDI4j",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 272
        },
        "outputId": "1ed64420-9c85-4c47-8c04-a6a80d1ee4eb"
      },
      "source": [
        "# fit the remainder of the pipeline on the training data\n",
        "pipeline.fit(X,y)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Pipeline(memory=None,\n",
              "         steps=[('estimator',\n",
              "                 LGBMClassifier(boosting_type='gbdt', class_weight='balanced',\n",
              "                                colsample_bytree=0.9775643042194456,\n",
              "                                importance_type='split',\n",
              "                                learning_rate=0.02168565444468173, max_depth=-1,\n",
              "                                min_child_samples=30,\n",
              "                                min_child_weight=0.0015226421960632711,\n",
              "                                min_split_gain=0.0, n_estimators=381, n_jobs=1,\n",
              "                                num_leaves=2, objective=None, random_state=33,\n",
              "                                reg_alpha=0.9934190174797073,\n",
              "                                reg_lambda=0.045733276980655324, silent=True,\n",
              "                                subsample=0.044454290568242016,\n",
              "                                subsample_for_bin=200000, subsample_freq=0))],\n",
              "         verbose=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D21zivFqDI5c",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# predict on the holdout data\n",
        "y_pred = pipeline.predict(X_holdout)"
      ],
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B5sv_9fJDI6g",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "66cfd350-21cb-4a6d-de9a-0542a11457ea"
      },
      "source": [
        "# compute score for the optimization metric\n",
        "# scorer may need pos_label, but not all scorers take pos_label parameter\n",
        "from sklearn.metrics import get_scorer\n",
        "scorer = get_scorer(optimization_metric)\n",
        "score = None\n",
        "#score = scorer(pipeline, X_holdout, y_holdout)  # this would suffice for simple cases\n",
        "pos_label = None  # if you want to supply the pos_label, specify it here\n",
        "if pos_label is None and 'pos_label' in _input_metadata:\n",
        "    pos_label=_input_metadata['pos_label']\n",
        "try:\n",
        "    score = scorer(pipeline, X_holdout, y_holdout)\n",
        "except Exception as e1:\n",
        "    if learning_type is \"classification\" and (pos_label is None or str(pos_label)==''):\n",
        "        print('You may have to provide a value for pos_label in order for a score to be calculated.')\n",
        "        raise(e1)\n",
        "    else:\n",
        "        exception_string=str(e1)\n",
        "        if 'pos_label' in exception_string:\n",
        "            try:\n",
        "                scorer = make_pos_label_scorer(scorer, pos_label=pos_label)\n",
        "                score = scorer(pipeline, X_holdout, y_holdout)\n",
        "                print('Retry was successful with pos_label supplied to scorer')\n",
        "            except Exception as e2:\n",
        "                print('Initial attempt to use scorer failed.  Exception was:')\n",
        "                print(e1)\n",
        "                print('')\n",
        "                print('Retry with pos_label failed.  Exception was:')\n",
        "                print(e2)\n",
        "        else:\n",
        "            raise(e1)\n",
        "\n",
        "if score is not None:\n",
        "    print(score)"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.963855421686747\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p6-4k4FuDI8O",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88
        },
        "outputId": "6d2140e9-672b-4158-9595-e12f49aaf716"
      },
      "source": [
        "# cross_validate pipeline using training data\n",
        "from sklearn.model_selection import cross_validate\n",
        "from sklearn.model_selection import StratifiedKFold, KFold\n",
        "if learning_type == 'classification':\n",
        "    fold_generator = StratifiedKFold(n_splits=cv_num_folds, random_state=random_state)\n",
        "else:\n",
        "    fold_generator = KFold(n_splits=cv_num_folds, random_state=random_state)\n",
        "cv_results = cross_validate(pipeline, X, y, cv=fold_generator, scoring={optimization_metric:scorer}, return_train_score=True)\n",
        "import numpy as np\n",
        "np.mean(cv_results['test_' + optimization_metric])"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_split.py:296: FutureWarning: Setting a random_state has no effect since shuffle is False. This will raise an error in 0.24. You should leave random_state to its default (None), or set shuffle=True.\n",
            "  FutureWarning\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9301075268817204"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_fMro8dADI81",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        },
        "outputId": "59d173c3-5b35-491b-f104-49bcae6472b1"
      },
      "source": [
        "cv_results"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'fit_time': array([0.05365086, 0.0550344 , 0.04959536]),\n",
              " 'score_time': array([0.00229263, 0.00204325, 0.00206661]),\n",
              " 'test_accuracy': array([0.92741935, 0.94758065, 0.91532258]),\n",
              " 'train_accuracy': array([0.93145161, 0.92137097, 0.9375    ])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    }
  ]
}