seldon_custom_server_README.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A practical guide to MLOps with Seldon Core and Jenkins X\n",
    "\n",
    "This tutorial provides an end-to-end tutorial that shows you how to build you MLOps pipeline with Seldon Core and Jenkins X:\n",
    "\n",
    "* Seldon Core is a machine learning deployment & orchestration engine in Kubernetes\n",
    "* Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Requests\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intuitive explanation\n",
    "\n",
    "Before we proceed, we want to understand what we will be trying to achieve. \n",
    "\n",
    "And what better way of doing this than by diving into an architectural diagram.\n",
    "\n",
    "[TODO ARCHITECTURE]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requirements\n",
    "\n",
    "* A Kubernetes cluster running v1.13+ (this was run using GKE)\n",
    "* The [jx CLI](https://github.com/jenkins-x/jx/) version 2.0.916\n",
    "* Jenkins-X installed in your cluster (you can set it up with the [jx boot tutorial](https://jenkins-x.io/docs/getting-started/setup/boot/))\n",
    "* Seldon Core [v0.5.0 installed]() in your cluster\n",
    "\n",
    "Once you set everything up, we'll be ready to kick off 🚀"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setting up repo\n",
    "\n",
    "Now we want to start setting up our repo. For this we will create the following structure:\n",
    "\n",
    "* `jenkins-x.yml` - File specifying the CI / CD steps \n",
    "* `Makefile` - Commands to build and test model\n",
    "* `README.(md|ipynb)` - This file!\n",
    "* `VERSION` - A file containing the version which is updated upon each release\n",
    "* `gitops/` - Folder containing the state of our production cluster\n",
    "* `assets/` - Folder containing other assets such as model binary, sample deployments, etc\n",
    "* `src`\n",
    "    * `ModelName.py` - Model server wrapper file\n",
    "    * `test_ModelName.py` - Unit test for model server\n",
    "    * `requirements-dev.txt` - Requirements for testing\n",
    "    * `requirements.txt` - Requiremnets for prod\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's train a model locally\n",
    "\n",
    "Let's have a look at the model we're using for text classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting requirements-dev.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile requirements-dev.txt\n",
    "scikit-learn==0.20.1\n",
    "pytest==5.1.1\n",
    "joblib==0.13.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!make install_dev"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "From: sd345@city.ac.uk (Michael Collier)\n",
      "Subject: Converting images to HP LaserJet III?\n",
      "Nntp-Posting-Host: hampton\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import fetch_20newsgroups\n",
    "categories = ['alt.atheism', 'soc.religion.christian',\n",
    "              'comp.graphics', 'sci.med']\n",
    "\n",
    "twenty_train = fetch_20newsgroups(\n",
    "    subset='train', categories=categories, shuffle=True, random_state=42)\n",
    "\n",
    "twenty_test = fetch_20newsgroups(\n",
    "    subset='test', categories=categories, shuffle=True, random_state=42)\n",
    "\n",
    "# Printing the top 3 newstories\n",
    "print(\"\\n\".join(twenty_train.data[0].split(\"\\n\")[:3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('vect',\n",
       "                 CountVectorizer(analyzer='word', binary=False,\n",
       "                                 decode_error='strict',\n",
       "                                 dtype=<class 'numpy.int64'>, encoding='utf-8',\n",
       "                                 input='content', lowercase=True, max_df=1.0,\n",
       "                                 max_features=None, min_df=1,\n",
       "                                 ngram_range=(1, 1), preprocessor=None,\n",
       "                                 stop_words=None, strip_accents=None,\n",
       "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
       "                                 tokenizer=None, vocabulary=None)),\n",
       "                ('tfidf',\n",
       "                 TfidfTransformer(norm='l2', smooth_idf=True,\n",
       "                                  sublinear_tf=False, use_idf=True)),\n",
       "                ('clf',\n",
       "                 MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.feature_extraction.text import TfidfTransformer, CountVectorizer\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "\n",
    "text_clf = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('clf', MultinomialNB()),\n",
    "])\n",
    "\n",
    "text_clf.fit(twenty_train.data, twenty_train.target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTENT:\n",
      "Subject: Re: HELP for Kidney Stones ..............\n",
      "Organization: The Avant-Garde of the Now, Ltd.\n",
      "Lines: 12\n",
      "NNTP-Posting-Host: ucsd.edu\n",
      "\n",
      "As I recall from my bout with kidney stones, there isn't \n",
      "\n",
      "-----------\n",
      "\n",
      "PREDICTED CLASS: comp.graphics\n"
     ]
    }
   ],
   "source": [
    "# Let's try one\n",
    "idx = 0\n",
    "print(f\"CONTENT:{twenty_test.data[idx][35:230]}\\n\\n-----------\\n\")\n",
    "print(f\"PREDICTED CLASS: {categories[twenty_test.target[idx]]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.83\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "predicted = text_clf.predict(twenty_test.data)\n",
    "print(f\"Accuracy: {np.mean(predicted == twenty_test.target):.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy the model\n",
    "\n",
    "Now we want to be able to deploy the model we just trained"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p assets/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['assets/model.joblib']"
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import joblib\n",
    "joblib.dump(text_clf, \"assets/model.joblib\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating gs://news_classifier/...\r\n"
     ]
    }
   ],
   "source": [
    "!gsutil mb gs://news_classifier/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying file://model.joblib [Content-Type=application/octet-stream]...\n",
      "/ [1 files][  4.4 MiB/  4.4 MiB]                                                \n",
      "Operation completed over 1 objects/4.4 MiB.                                      \n"
     ]
    }
   ],
   "source": [
    "!gsutil cp assets/model.joblib gs://news_classifier/model/model.joblib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Updated ACL on gs://news_classifier/model\n",
      "Updated ACL on gs://news_classifier/model.joblib\n",
      "Updated ACL on gs://news_classifier/model/model.joblib\n"
     ]
    }
   ],
   "source": [
    "!gsutil acl ch -r -u AllUsers:R gs://news_classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p src/\n",
    "!touch src/__init__.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting src/SklearnServer.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile src/SklearnServer.py\n",
    "\n",
    "import joblib, logging\n",
    "from seldon_core.storage import Storage\n",
    "\n",
    "class SklearnServer:\n",
    "    def __init__(self, model_uri):\n",
    "        self._model = joblib.load(f\"{output_dir}/model.joblib\")\n",
    "\n",
    "    def predict(self, data, feature_names=[], metadata={}):\n",
    "        logging.info(data)\n",
    "\n",
    "        prediction = self._model.predict(data)\n",
    "\n",
    "        logging.info(prediction)\n",
    "\n",
    "        return prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting src/test_SklearnServer.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile src/test_SklearnServer.py\n",
    "\n",
    "from .SklearnServer import SklearnServer\n",
    "import os\n",
    "\n",
    "def test_sklearn_server():\n",
    "    data = [\"From: brian@ucsd.edu (Brian Kantor)\\nSubject: Re: HELP for Kidney Stones ..............\\nOrganization: The Avant-Garde of the Now, Ltd.\\nLines: 12\\nNNTP-Posting-Host: ucsd.edu\\n\\nAs I recall from my bout with kidney stones, there isn't any\\nmedication that can do anything about them except relieve the pain.\\n\\nEither they pass, or they have to be broken up with sound, or they have\\nto be extracted surgically.\\n\\nWhen I was in, the X-ray tech happened to mention that she'd had kidney\\nstones and children, and the childbirth hurt less.\\n\\nDemerol worked, although I nearly got arrested on my way home when I barfed\\nall over the police car parked just outside the ER.\\n\\t- Brian\\n\",\n",
    "            'From: rind@enterprise.bih.harvard.edu (David Rind)\\nSubject: Re: Candida(yeast) Bloom, Fact or Fiction\\nOrganization: Beth Israel Hospital, Harvard Medical School, Boston Mass., USA\\nLines: 37\\nNNTP-Posting-Host: enterprise.bih.harvard.edu\\n\\nIn article <1993Apr26.103242.1@vms.ocom.okstate.edu>\\n banschbach@vms.ocom.okstate.edu writes:\\n>are in a different class.  The big question seems to be is it reasonable to \\n>use them in patients with GI distress or sinus problems that *could* be due \\n>to candida blooms following the use of broad-spectrum antibiotics?\\n\\nI guess I\\'m still not clear on what the term \"candida bloom\" means,\\nbut certainly it is well known that thrush (superficial candidal\\ninfections on mucous membranes) can occur after antibiotic use.\\nThis has nothing to do with systemic yeast syndrome, the \"quack\"\\ndiagnosis that has been being discussed.\\n\\n\\n>found in the sinus mucus membranes than is candida.  Women have been known \\n>for a very long time to suffer from candida blooms in the vagina and a \\n>women is lucky to find a physician who is willing to treat the cause and \\n>not give give her advise to use the OTC anti-fungal creams.\\n\\nLucky how?  Since a recent article (randomized controlled trial) of\\noral yogurt on reducing vaginal candidiasis, I\\'ve mentioned to a \\nnumber of patients with frequent vaginal yeast infections that they\\ncould try eating 6 ounces of yogurt daily.  It turns out most would\\nrather just use anti-fungal creams when they get yeast infections.\\n\\n>yogurt dangerous).  If this were a standard part of medical practice, as \\n>Gordon R. says it is, then the incidence of GI distress and vaginal yeast \\n>infections should decline.\\n\\nAgain, this just isn\\'t what the systemic yeast syndrome is about, and\\nhas nothing to do with the quack therapies that were being discussed.\\nThere is some evidence that attempts to reinoculate the GI tract with\\nbacteria after antibiotic therapy don\\'t seem to be very helpful in\\nreducing diarrhea, but I don\\'t think anyone would view this as a\\nquack therapy.\\n-- \\nDavid Rind\\nrind@enterprise.bih.harvard.edu\\n']\n",
    "    labels = [2, 2]\n",
    "\n",
    "    s = SklearnServer(f\"file://{os.getcwd()}/assets/\")\n",
    "    result = s.predict(data)\n",
    "    assert all(result == labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cat: VERSION: No such file or directory\n",
      "Makefile:25: warning: overriding recipe for target 'make'\n",
      "Makefile:22: warning: ignoring old recipe for target 'make'\n",
      "pytest -s --verbose -W ignore 2>&1\n",
      "\u001b[1m============================= test session starts ==============================\u001b[0m\n",
      "platform linux -- Python 3.7.3, pytest-5.1.1, py-1.8.0, pluggy-0.12.0 -- /home/alejandro/miniconda3/envs/reddit-classification/bin/python\n",
      "cachedir: .pytest_cache\n",
      "rootdir: /home/alejandro/Programming/kubernetes/seldon/sig-mlops-example\n",
      "plugins: cov-2.7.1, forked-1.0.2, localserver-0.5.0\n",
      "collected 1 item                                                               \u001b[0m\u001b[1m\n",
      "\n",
      "src/test_SklearnServer.py::test_sklearn_server \u001b[32mPASSED\u001b[0m\n",
      "\n",
      "\u001b[32m\u001b[1m============================== 1 passed in 1.70s ===============================\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!make test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting src/requirements.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile src/requirements.txt\n",
    "scikit-learn==0.20.1\n",
    "joblib==0.13.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting src/seldon_model.conf\n"
     ]
    }
   ],
   "source": [
    "%%writefile src/seldon_model.conf\n",
    "MODEL_NAME=SklearnServer\n",
    "API_TYPE=REST\n",
    "SERVICE_TYPE=MODEL\n",
    "PERSISTENCE=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "---> Installing application source...\n",
      "---> Installing dependencies ...\n",
      "Looking in links: /whl\n",
      "Collecting scikit-learn==0.20.1 (from -r requirements.txt (line 1))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/10/26/d04320c3edf2d59b1fcd0720b46753d4d603a76e68d8ad10a9b92ab06db2/scikit_learn-0.20.1-cp36-cp36m-manylinux1_x86_64.whl (5.4MB)\n",
      "Collecting joblib==0.13.2 (from -r requirements.txt (line 2))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/cd/c1/50a758e8247561e58cb87305b1e90b171b8c767b15b12a1734001f41d356/joblib-0.13.2-py2.py3-none-any.whl (278kB)\n",
      "Requirement already satisfied: numpy>=1.8.2 in /usr/local/lib/python3.6/site-packages (from scikit-learn==0.20.1->-r requirements.txt (line 1)) (1.17.2)\n",
      "Collecting scipy>=0.13.3 (from scikit-learn==0.20.1->-r requirements.txt (line 1))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/29/50/a552a5aff252ae915f522e44642bb49a7b7b31677f9580cfd11bcc869976/scipy-1.3.1-cp36-cp36m-manylinux1_x86_64.whl (25.2MB)\n",
      "Installing collected packages: scipy, scikit-learn, joblib\n",
      "Successfully installed joblib-0.13.2 scikit-learn-0.20.1 scipy-1.3.1\n",
      "WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "WARNING: You are using pip version 19.1, however version 19.3.1 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\n",
      "Build completed successfully\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "SELDON_BASE_WRAPPER=\"seldonio/seldon-core-s2i-python36:0.12\"\n",
    "s2i build src/. $SELDON_BASE_WRAPPER sklearn-server:0.1 \\\n",
    "    --environment-file src/seldon_model.conf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Process is interrupted.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "YOUR_DOCKER_USERNAME=\"axsauze\"\n",
    "\n",
    "docker tag sklearn-server:0.1 $YOUR_DOCKER_USERNAME/sklearn-server:0.1\n",
    "docker push $YOUR_DOCKER_USERNAME/sklearn-server:0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p gitops"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing gitops/test_deployment.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile gitops/test_deployment.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1alpha2\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: news-classifier-server\n",
    "  namespace: default\n",
    "  creationTimestamp: \n",
    "spec:\n",
    "  name: news-classifier-server\n",
    "  predictors:\n",
    "  - name: default\n",
    "    graph:\n",
    "      name: news-classifier-server-processor\n",
    "      endpoint:\n",
    "        type: REST\n",
    "      type: MODEL\n",
    "      children: []\n",
    "      parameters:\n",
    "      - name: model_uri\n",
    "        type: STRING\n",
    "        value: \"gs://news_classifier/model/\"\n",
    "    componentSpecs:\n",
    "    - metadata:\n",
    "        creationTimestamp: '2019-10-12T16:00:00Z'\n",
    "      spec:\n",
    "        containers:\n",
    "        - image: axsauze/sklearn-server:0.1\n",
    "          name: news-classifier-server-processor\n",
    "          env:\n",
    "          - name: SELDON_LOG_LEVEL\n",
    "            value: DEBUG\n",
    "        terminationGracePeriodSeconds: 1\n",
    "    replicas: 1\n",
    "    engineResources: {}\n",
    "    svcOrchSpec: {}\n",
    "    traffic: 100\n",
    "    explainer:\n",
    "      containerSpec:\n",
    "        name: ''\n",
    "        resources: {}\n",
    "  annotations:\n",
    "    seldon.io/engine-seldon-log-messages-externally: 'true'\n",
    "status: {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/news-classifier-server created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f gitops/test_deployment.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ndarray {\n",
       "  values {\n",
       "    number_value: 2.0\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "import numpy as np\n",
    "\n",
    "url = !kubectl get svc ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'\n",
    "\n",
    "sc = SeldonClient(\n",
    "    gateway=\"ambassador\", \n",
    "    gateway_endpoint=\"localhost:80\",\n",
    "    deployment_name=\"news-classifier-server\",\n",
    "    payload_type=\"ndarray\",\n",
    "    namespace=\"default\",\n",
    "    transport=\"rest\")\n",
    "\n",
    "response = sc.predict(data=np.array([twenty_test.data[0]]))\n",
    "\n",
    "response.response.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"so6n21pkf70fm66eka28lc63cr\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"news-classifier-server-processor\": \"axsauze/sklearn-server:0.1\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [],\n",
      "    \"ndarray\": [2.0]\n",
      "  }\n",
      "}"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "\r",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
      "100   350  100   278  100    72   7942   2057 --:--:-- --:--:-- --:--:-- 10294\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "curl -X POST -H 'Content-Type: application/json' \\\n",
    "     -d \"{'data': {'names': ['text'], 'ndarray': ['Hello world this is a test']}}\" \\\n",
    "    http://localhost/seldon/default/news-classifier-server/api/v0.1/predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"news-classifier-server\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f gitops/test_deployment.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setting up CI before CD\n",
    "\n",
    "We have now separated our model development into two chunks: \n",
    "\n",
    "* The first one involves the creation of a model serve, and the second one involves the CI of the model server, and the second involves the deployment of models that create the model.\n",
    "\n",
    "\n",
    "## Using the Jenkins X pipeline\n",
    "\n",
    "In order to do this we will be able to first run some tests and the push to the docker repo.\n",
    "\n",
    "For this we will be leveraging the Jenkins X file, we'll first start with a simple file that just runs the tests:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting jenkins-x.yml\n"
     ]
    }
   ],
   "source": [
    "%%writefile jenkins-x.yml\n",
    "buildPack: none\n",
    "pipelineConfig:\n",
    "  pipelines:\n",
    "    release:\n",
    "      pipeline:\n",
    "        agent:\n",
    "          image: seldonio/core-builder:0.4\n",
    "        stages:\n",
    "          - name: test-sklearn-server\n",
    "            steps:\n",
    "            - name: run-tests\n",
    "              command: make\n",
    "              args:\n",
    "              - install_dev\n",
    "              - test\n",
    "    pullRequest:\n",
    "      pipeline:\n",
    "        agent:\n",
    "          image: seldonio/core-builder:0.4\n",
    "        stages:\n",
    "          - name: test-sklearn-server\n",
    "            steps:\n",
    "            - name: run-tests\n",
    "              command: make\n",
    "              args:\n",
    "              - install_dev\n",
    "              - test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `jenkins-x.yml` file is pretty easy to understand if we read through the different steps.\n",
    "\n",
    "Basically we can define the steps of what happens upon `release` - i.e. when a PR / Commit is added to master - and what happens upon `pullRequest` - whenever someone opens a pull request.\n",
    "\n",
    "You can see that the steps are exactly the same for both release and PR for now - namely, we run `make install_dev test` which basically installs all the dependencies and runs all the tests.\n",
    "\n",
    "### Setting up the repo with the pipeline\n",
    "\n",
    "In order for the Pipeline to be executed on PR and release, we must import it into our Jenkins X cluster. \n",
    "\n",
    "We can do this by running this command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jx import --no-draft=true"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As soon as we import the repository into Jenkins X, the release path gets triggered.\n",
    "\n",
    "We can see the activities that have been triggered by running:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jx get activities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "And we can actually see the logs of what is happening at every step by running:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jx get build logs \"$GIT_USERNAME/seldon-jx-mlops/master #1 release\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "As we can see, the `release` trigger is working as expected. We can now trigger the PR by opening a PR.\n",
    "\n",
    "For this, let's add a small change and push a PR:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash \n",
    "\n",
    "# Create new branch and move into it\n",
    "git checkout -b feature-1\n",
    "\n",
    "# Add an extra space at the end\n",
    "echo \" \" >> jenkins-x.yml\n",
    "git add jenkins-x\n",
    "git commit -m \"Added extra space to trigger master\"\n",
    "git push origin feature-1\n",
    "\n",
    "# Now create pull request\n",
    "git request-pull -p origin/master ./"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Once we create the pull request we can visualise that the PR has been created and the bot has commented.\n",
    "\n",
    "We would now also be able to see that the tests are now running, and similar to above we can see the logs with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl get build logs \"$GIT_USERNAME/seldon-jx-mlops/pr-1 #1 pr-build\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pushing images automatically\n",
    "Now that we're able to build some tests, we want to update the images so we can have the latest on each release.\n",
    "\n",
    "For this, we will have to add a couple of things, including:\n",
    "\n",
    "1. The task in the `jenkins-x.yml` file that would allow us to build and push the image\n",
    "2. The config in the `jenkins-x.yml` to provide docker authentications (to push images)\n",
    "3. A script that starts a docker daemon and then builds+psuhes the images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### JX Task to Build and Push image\n",
    "\n",
    "For this, we would just have to append the following task in our jenkins file:\n",
    "    \n",
    "```\n",
    "    - name: build-and-push-images\n",
    "      command: bash\n",
    "      args:\n",
    "      - assets/scripts/build_and_push_docker_daemon.sh\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Config to provide docker authentication\n",
    "\n",
    "This piece is slightly more extensive, as we will need to use Docker to build out containers due to the dependency on `s2i` to build the model wrappers.\n",
    "\n",
    "First we need to define the volumes that we'll be mounting to the container.\n",
    "\n",
    "The first few volumes before basically consist of the core components that docker will need to be able to run.\n",
    "```\n",
    "          volumes:\n",
    "            - name: modules\n",
    "              hostPath:\n",
    "                path: /lib/modules\n",
    "                type: Directory\n",
    "            - name: cgroup\n",
    "              hostPath:\n",
    "                path: /sys/fs/cgroup\n",
    "                type: Directory\n",
    "            - name: dind-storage\n",
    "              emptyDir: {}\n",
    "```\n",
    "We also want to mount the docker credentials which we will generate in the next step.\n",
    "```\n",
    "            - name: jenkins-docker-config-volume\n",
    "              secret:\n",
    "                items:\n",
    "                - key: config.json\n",
    "                  path: config.json\n",
    "                secretName: jenkins-docker-cfg\n",
    "```\n",
    "Once we've created the volumes, now we just need to mount them. This can be done as follows:\n",
    "```\n",
    "        options:\n",
    "          containerOptions:\n",
    "            volumeMounts:\n",
    "              - mountPath: /lib/modules\n",
    "                name: modules\n",
    "                readOnly: true\n",
    "              - mountPath: /sys/fs/cgroup\n",
    "                name: cgroup\n",
    "              - name: dind-storage\n",
    "                mountPath: /var/lib/docker                 \n",
    "```\n",
    "And finally we also mount the docker auth configuration so we don't have to run `docker login`:\n",
    "```\n",
    "              - mountPath: /builder/home/.docker\n",
    "                name: jenkins-docker-config-volume\n",
    "```\n",
    "\n",
    "And to finalise, we need to make sure that the pod can run with privileged context.\n",
    "\n",
    "The reason why this is required is in order to be able to run the docker daemon:\n",
    "```\n",
    "            securityContext:\n",
    "              privileged: true\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Updating Jenkins X file and testing\n",
    "\n",
    "Now that we've gotten a breakdown of the different additions for the `jenkins-x.yml` file, we can update it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting jenkins-x.yml\n"
     ]
    }
   ],
   "source": [
    "%%writefile jenkins-x.yml\n",
    "buildPack: none\n",
    "pipelineConfig:\n",
    "  pipelines:\n",
    "    release:\n",
    "      pipeline:\n",
    "        agent:\n",
    "          image: seldonio/core-builder:0.4\n",
    "        stages:\n",
    "        - name: build-and-test\n",
    "          parallel:\n",
    "          - name: test-and-deploy-sklearn-server\n",
    "            steps:\n",
    "            - name: test-sklearn-server\n",
    "              steps:\n",
    "              - name: run-tests\n",
    "                command: make\n",
    "                args:\n",
    "                - install_dev\n",
    "                - test\n",
    "            - name: build-and-push-images\n",
    "              command: bash\n",
    "              args:\n",
    "              - assets/scripts/build_and_push_docker_daemon.sh\n",
    "        options:\n",
    "          containerOptions:\n",
    "            volumeMounts:\n",
    "              - mountPath: /lib/modules\n",
    "                name: modules\n",
    "                readOnly: true\n",
    "              - mountPath: /sys/fs/cgroup\n",
    "                name: cgroup\n",
    "              - name: dind-storage\n",
    "                mountPath: /var/lib/docker\n",
    "              - mountPath: /builder/home/.docker\n",
    "                name: jenkins-docker-config-volume\n",
    "            securityContext:\n",
    "              privileged: true\n",
    "          volumes:\n",
    "            - name: modules\n",
    "              hostPath:\n",
    "                path: /lib/modules\n",
    "                type: Directory\n",
    "            - name: cgroup\n",
    "              hostPath:\n",
    "                path: /sys/fs/cgroup\n",
    "                type: Directory\n",
    "            - name: dind-storage\n",
    "              emptyDir: {}\n",
    "            - name: jenkins-docker-config-volume\n",
    "              secret:\n",
    "                items:\n",
    "                - key: config.json\n",
    "                  path: config.json\n",
    "                secretName: jenkins-docker-cfg\n",
    "    pullRequest:\n",
    "      pipeline:\n",
    "        agent:\n",
    "          image: seldonio/core-builder:0.4\n",
    "        stages:\n",
    "        - name: build-and-test\n",
    "          parallel:\n",
    "          - name: test-and-deploy-sklearn-server\n",
    "            steps:\n",
    "            - name: test-sklearn-server\n",
    "              steps:\n",
    "              - name: run-tests\n",
    "                command: make\n",
    "                args:\n",
    "                - install_dev\n",
    "                - test\n",
    "            - name: build-and-push-images\n",
    "              command: bash\n",
    "              args:\n",
    "              - assets/scripts/build_and_push_docker_daemon.sh\n",
    "        options:\n",
    "          containerOptions:\n",
    "            volumeMounts:\n",
    "              - mountPath: /lib/modules\n",
    "                name: modules\n",
    "                readOnly: true\n",
    "              - mountPath: /sys/fs/cgroup\n",
    "                name: cgroup\n",
    "              - name: dind-storage\n",
    "                mountPath: /var/lib/docker\n",
    "              - mountPath: /builder/home/.docker\n",
    "                name: jenkins-docker-config-volume\n",
    "            securityContext:\n",
    "              privileged: true\n",
    "          volumes:\n",
    "            - name: modules\n",
    "              hostPath:\n",
    "                path: /lib/modules\n",
    "                type: Directory\n",
    "            - name: cgroup\n",
    "              hostPath:\n",
    "                path: /sys/fs/cgroup\n",
    "                type: Directory\n",
    "            - name: dind-storage\n",
    "              emptyDir: {}\n",
    "            - name: jenkins-docker-config-volume\n",
    "              secret:\n",
    "                items:\n",
    "                - key: config.json\n",
    "                  path: config.json\n",
    "                secretName: jenkins-docker-cfg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}