COVID_19_BERT_ResearchPapers_Semantic_Search.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "COVID-19-BERT-ResearchPapers-Semantic-Search.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "Uidivjf6fRdW",
        "sFVjsMb8xPkD",
        "_p2-bJT_EyZL"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nMKZM7dK80C7",
        "colab_type": "text"
      },
      "source": [
        "# COVID-19-BERT-ResearchPapers-Semantic-Search"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jve55iUI9Cyr",
        "colab_type": "text"
      },
      "source": [
        "This work builds a **semantic search engine using BERT**, to search a query through the dataset of research papers provided as part of [Kaggle's competion CORD-19-research-challenge](https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge), we like to thank kaggle and all of the competion sponsers for this competion in bringing up efforts for fighting this virus.\n",
        "\n",
        "This work, \n",
        "1.   first divides the dataset to paragraphs\n",
        "2.   then uses BERT to embedded paragraphs of papers using bert-base-nli-mean-tokens pretrained model\n",
        "3.   finally runs a query and returns the top 5 paragraphs and their papers' titles,abstract,abstract_summary\n",
        "\n",
        "We have built this notebook to run seamlesly on google colab, connect with google drive, and downalod the data using [kaggle api](https://github.com/Kaggle/kaggle-api), so no data is downloaded to your device, and no need to have a powerful GPU, as all is done freely through google colab, we like to thank google for providing the research community with google colab\n",
        "\n",
        "**Code** is found here [on github ](https://github.com/theamrzaki/COVID-19-BERT-ResearchPapers-Semantic-Search), we truly hope that this work has a postive impact in the fight aganist this evil virus, we truly pray for all people to be able to win this fight.\n",
        "\n",
        "\n",
        "**References** :\n",
        "\n",
        "*   We use the library provided by [UKPLab](https://github.com/UKPLab) called [sentence-transformers](https://github.com/UKPLab/sentence-transformers), this library makes it truly easy to use BERT and other architectures like ALBERT,XLNet for sentence embedding, they also provide simple interface to query and cluster data.\n",
        "*   We have used the code from [maksimeren](https://www.kaggle.com/maksimeren/covid-19-literature-clustering) for data processing, we truly like to thank him.\n",
        "*   We used the concept of drawing BERT, disccussed here [Jay Alammar](http://jalammar.github.io/) in illustrating how our architecture works, his blogs are extremly informative and easily understood.\n",
        "*   We used the pre-trained models disccess in Conneau et al., 2017, show in the InferSent-Paper (Supervised Learning of Universal Sentence Representations from Natural Language Inference Data) that training on Natural Language Inference (NLI) data can produce universal sentence embeddings.\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s7hJ6rfxA5Fr",
        "colab_type": "text"
      },
      "source": [
        "## Architecture"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8p4ZotIXCuU9",
        "colab_type": "text"
      },
      "source": [
        "The paper is found in the json from the dataset in paragraphs, we use this division in the papers, then we pass these paragraphs to a pre-trained BERT model [bert-base-nli-mean-tokens](https://github.com/UKPLab/sentence-transformers/blob/master/docs/pretrained-models/nli-models.md) to be embedded\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ERBx-rtG78zN",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://github.com/theamrzaki/COVID-19-BERT-ResearchPapers-Semantic-Search/blob/master/assets/Bert%20Information%20Retrival_Train.jpg?raw=true)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wlMOexswDST2",
        "colab_type": "text"
      },
      "source": [
        "After embedding is done, we pass the embedded the query using the same bert model\n",
        "\n",
        "Then we compare the both embedding represnetations (paragraphs and query) using cosine similarity, we then return the most similar paragraphs with their paper details (title,abstract,abstract_summary)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1qSWg9ja8LbD",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://github.com/theamrzaki/COVID-19-BERT-ResearchPapers-Semantic-Search/blob/master/assets/Bert%20Information%20Retrival_Test.jpg?raw=true)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6PK6rRi4P7ON",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#first install the library that would help us use BERT in an easy to use interface\n",
        "#https://github.com/UKPLab/sentence-transformers/tree/master/sentence_transformers\n",
        "!pip install -U sentence-transformers"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y_qHBXK0YBqR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#install the kaggle data to google colab\n",
        "#https://github.com/Kaggle/kaggle-api#api-credentials\n",
        "!pip install kaggle\n",
        "import os\n",
        "!cp \"/content/kaggle.json\" /root/.kaggle\n",
        "!kaggle datasets download -d allen-institute-for-ai/CORD-19-research-challenge\n",
        "!unzip  CORD-19-research-challenge.zip -d /content/CORD-19-research-challenge"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Uidivjf6fRdW",
        "colab_type": "text"
      },
      "source": [
        "## Data Processing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C9wyrWwff2Eg",
        "colab_type": "text"
      },
      "source": [
        "built using\n",
        "https://www.kaggle.com/maksimeren/covid-19-literature-clustering"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aUsuL1pYfTzl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import glob\n",
        "import json\n",
        "import pandas as pd\n",
        "from tqdm import tqdm\n",
        "root_path = '/content/CORD-19-research-challenge/'\n",
        "all_json = glob.glob(f'{root_path}/**/*.json', recursive=True)\n",
        "len(all_json)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9ioYpWc-gI11",
        "colab_type": "code",
        "outputId": "35ef7f9b-7387-4fdd-e0e4-7b82e0d0fdd3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 649
        }
      },
      "source": [
        "metadata_path = f'{root_path}/metadata.csv'\n",
        "meta_df = pd.read_csv(metadata_path, dtype={\n",
        "    'pubmed_id': str,\n",
        "    'Microsoft Academic Paper ID': str, \n",
        "    'doi': str\n",
        "})\n",
        "meta_df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sha</th>\n",
              "      <th>source_x</th>\n",
              "      <th>title</th>\n",
              "      <th>doi</th>\n",
              "      <th>pmcid</th>\n",
              "      <th>pubmed_id</th>\n",
              "      <th>license</th>\n",
              "      <th>abstract</th>\n",
              "      <th>publish_time</th>\n",
              "      <th>authors</th>\n",
              "      <th>journal</th>\n",
              "      <th>Microsoft Academic Paper ID</th>\n",
              "      <th>WHO #Covidence</th>\n",
              "      <th>has_full_text</th>\n",
              "      <th>full_text_file</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>NaN</td>\n",
              "      <td>Elsevier</td>\n",
              "      <td>Intrauterine virus infections and congenital h...</td>\n",
              "      <td>10.1016/0002-8703(72)90077-4</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4361535</td>\n",
              "      <td>els-covid</td>\n",
              "      <td>Abstract The etiologic basis for the vast majo...</td>\n",
              "      <td>1972-12-31</td>\n",
              "      <td>Overall, James C.</td>\n",
              "      <td>American Heart Journal</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>custom_license</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>NaN</td>\n",
              "      <td>Elsevier</td>\n",
              "      <td>Coronaviruses in Balkan nephritis</td>\n",
              "      <td>10.1016/0002-8703(80)90355-5</td>\n",
              "      <td>NaN</td>\n",
              "      <td>6243850</td>\n",
              "      <td>els-covid</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1980-03-31</td>\n",
              "      <td>Georgescu, Leonida; Diosi, Peter; Buţiu, Ioan;...</td>\n",
              "      <td>American Heart Journal</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>custom_license</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>NaN</td>\n",
              "      <td>Elsevier</td>\n",
              "      <td>Cigarette smoking and coronary heart disease: ...</td>\n",
              "      <td>10.1016/0002-8703(80)90356-7</td>\n",
              "      <td>NaN</td>\n",
              "      <td>7355701</td>\n",
              "      <td>els-covid</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1980-03-31</td>\n",
              "      <td>Friedman, Gary D</td>\n",
              "      <td>American Heart Journal</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>custom_license</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>aecbc613ebdab36753235197ffb4f35734b5ca63</td>\n",
              "      <td>Elsevier</td>\n",
              "      <td>Clinical and immunologic studies in identical ...</td>\n",
              "      <td>10.1016/0002-9343(73)90176-9</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4579077</td>\n",
              "      <td>els-covid</td>\n",
              "      <td>Abstract Middle-aged female identical twins, o...</td>\n",
              "      <td>1973-08-31</td>\n",
              "      <td>Brunner, Carolyn M.; Horwitz, David A.; Shann,...</td>\n",
              "      <td>The American Journal of Medicine</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>True</td>\n",
              "      <td>custom_license</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>NaN</td>\n",
              "      <td>Elsevier</td>\n",
              "      <td>Epidemiology of community-acquired respiratory...</td>\n",
              "      <td>10.1016/0002-9343(85)90361-4</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4014285</td>\n",
              "      <td>els-covid</td>\n",
              "      <td>Abstract Upper respiratory tract infections ar...</td>\n",
              "      <td>1985-06-28</td>\n",
              "      <td>Garibaldi, Richard A.</td>\n",
              "      <td>The American Journal of Medicine</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "      <td>custom_license</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                        sha  ...  full_text_file\n",
              "0                                       NaN  ...  custom_license\n",
              "1                                       NaN  ...  custom_license\n",
              "2                                       NaN  ...  custom_license\n",
              "3  aecbc613ebdab36753235197ffb4f35734b5ca63  ...  custom_license\n",
              "4                                       NaN  ...  custom_license\n",
              "\n",
              "[5 rows x 15 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1a2kHu4jfySx",
        "colab_type": "text"
      },
      "source": [
        "### Read Data (Helpers)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jaqXqTfLfVDD",
        "colab_type": "code",
        "outputId": "bc568ecf-7f68-4811-9771-aaf0c0d3a117",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "class FileReader:\n",
        "    def __init__(self, file_path):\n",
        "        with open(file_path) as file:\n",
        "            content = json.load(file)\n",
        "            self.paper_id = content['paper_id']\n",
        "            self.abstract = []\n",
        "            self.body_text = []\n",
        "            # Abstract\n",
        "            for entry in content['abstract']:\n",
        "                self.abstract.append(entry['text'])\n",
        "            # Body text\n",
        "            for entry in content['body_text']:\n",
        "                self.body_text.append(entry['text'])\n",
        "            self.abstract = '\\n'.join(self.abstract)\n",
        "            self.body_text = '\\n'.join(self.body_text)\n",
        "    def __repr__(self):\n",
        "        return f'{self.paper_id}: {self.abstract[:200]}... {self.body_text[:200]}...'\n",
        "first_row = FileReader(all_json[0])\n",
        "print(first_row)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "dcd7a1235ea74e3ef71d051103bf8a64c3c8f457: 12 Background 13 After the outbreak of novel coronavirus (2019-nCoV) starting in late 2019, a number 14 of researchers have reported the predicted the virus transmission dynamics. However, 15 under th... A novel coronavirus (2019-nCoV) appeared in December 2019 in Wuhan, Hubei 33 Province in central China had triggered city closure on Jan. 23, 2020, and lockdown 34 of all major cities in the province ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ukE_CxmKf4b8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def get_breaks(content, length):\n",
        "    data = \"\"\n",
        "    words = content.split(' ')\n",
        "    total_chars = 0\n",
        "\n",
        "    # add break every length characters\n",
        "    for i in range(len(words)):\n",
        "        total_chars += len(words[i])\n",
        "        if total_chars > length:\n",
        "            data = data + \"<br>\" + words[i]\n",
        "            total_chars = 0\n",
        "        else:\n",
        "            data = data + \" \" + words[i]\n",
        "    return data"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gtDN1zQ8f63B",
        "colab_type": "code",
        "outputId": "bf3c962b-06b6-4c23-8052-03c04fb68e10",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 544
        }
      },
      "source": [
        "dict_ = {'paper_id': [], 'abstract': [], 'body_text': [], 'authors': [], 'title': [], 'journal': [], 'abstract_summary': []}\n",
        "for idx, entry in enumerate(all_json):\n",
        "    if idx % (len(all_json) // 10) == 0:\n",
        "        print(f'Processing index: {idx} of {len(all_json)}')\n",
        "    content = FileReader(entry)\n",
        "    \n",
        "    # get metadata information\n",
        "    meta_data = meta_df.loc[meta_df['sha'] == content.paper_id]\n",
        "    # no metadata, skip this paper\n",
        "    if len(meta_data) == 0:\n",
        "        continue\n",
        "    \n",
        "    dict_['paper_id'].append(content.paper_id)\n",
        "    dict_['abstract'].append(content.abstract)\n",
        "    dict_['body_text'].append(content.body_text)\n",
        "    \n",
        "    # also create a column for the summary of abstract to be used in a plot\n",
        "    if len(content.abstract) == 0: \n",
        "        # no abstract provided\n",
        "        dict_['abstract_summary'].append(\"Not provided.\")\n",
        "    elif len(content.abstract.split(' ')) > 100:\n",
        "        # abstract provided is too long for plot, take first 300 words append with ...\n",
        "        info = content.abstract.split(' ')[:100]\n",
        "        summary = get_breaks(' '.join(info), 40)\n",
        "        dict_['abstract_summary'].append(summary + \"...\")\n",
        "    else:\n",
        "        # abstract is short enough\n",
        "        summary = get_breaks(content.abstract, 40)\n",
        "        dict_['abstract_summary'].append(summary)\n",
        "        \n",
        "    # get metadata information\n",
        "    meta_data = meta_df.loc[meta_df['sha'] == content.paper_id]\n",
        "    \n",
        "    try:\n",
        "        # if more than one author\n",
        "        authors = meta_data['authors'].values[0].split(';')\n",
        "        if len(authors) > 2:\n",
        "            # more than 2 authors, may be problem when plotting, so take first 2 append with ...\n",
        "            dict_['authors'].append(\". \".join(authors[:2]) + \"...\")\n",
        "        else:\n",
        "            # authors will fit in plot\n",
        "            dict_['authors'].append(\". \".join(authors))\n",
        "    except Exception as e:\n",
        "        # if only one author - or Null valie\n",
        "        dict_['authors'].append(meta_data['authors'].values[0])\n",
        "    \n",
        "    # add the title information, add breaks when needed\n",
        "    try:\n",
        "        title = get_breaks(meta_data['title'].values[0], 40)\n",
        "        dict_['title'].append(title)\n",
        "    # if title was not provided\n",
        "    except Exception as e:\n",
        "        dict_['title'].append(meta_data['title'].values[0])\n",
        "    \n",
        "    # add the journal information\n",
        "    dict_['journal'].append(meta_data['journal'].values[0])\n",
        "    \n",
        "df_covid = pd.DataFrame(dict_, columns=['paper_id', 'abstract', 'body_text', 'authors', 'title', 'journal', 'abstract_summary'])\n",
        "df_covid.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Processing index: 0 of 29315\n",
            "Processing index: 2931 of 29315\n",
            "Processing index: 5862 of 29315\n",
            "Processing index: 8793 of 29315\n",
            "Processing index: 11724 of 29315\n",
            "Processing index: 14655 of 29315\n",
            "Processing index: 17586 of 29315\n",
            "Processing index: 20517 of 29315\n",
            "Processing index: 23448 of 29315\n",
            "Processing index: 26379 of 29315\n",
            "Processing index: 29310 of 29315\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>paper_id</th>\n",
              "      <th>abstract</th>\n",
              "      <th>body_text</th>\n",
              "      <th>authors</th>\n",
              "      <th>title</th>\n",
              "      <th>journal</th>\n",
              "      <th>abstract_summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>dcd7a1235ea74e3ef71d051103bf8a64c3c8f457</td>\n",
              "      <td>12 Background 13 After the outbreak of novel c...</td>\n",
              "      <td>A novel coronavirus (2019-nCoV) appeared in De...</td>\n",
              "      <td>Xinhai Li.  Xumao Zhao...</td>\n",
              "      <td>The lockdown of Hubei Province causing&lt;br&gt;dif...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>12 Background 13 After the outbreak of novel&lt;...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>86b6b0c1b2777541feb83116bcb7a5cb12a52310</td>\n",
              "      <td></td>\n",
              "      <td>Firstly informed to World Health Organization ...</td>\n",
              "      <td>Jung, Y. J..  Park, G.-S....</td>\n",
              "      <td>Comparative analysis of primer-probe sets for...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Not provided.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>73d80c8f5780d70bd8d343188c56e898e91557b6</td>\n",
              "      <td>Middle East respiratory syndrome coronavirus (...</td>\n",
              "      <td>Coronaviruses (CoVs) comprise a family of enve...</td>\n",
              "      <td>Straus, M. R..  Tang, T....</td>\n",
              "      <td>Ca2+ ions promote fusion of Middle East&lt;br&gt;Re...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Middle East respiratory syndrome coronavirus&lt;...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>70cc2e5152d3dc4d44494124ff556c9bbe9e6f41</td>\n",
              "      <td>1 Background: A new virus broke out in Wuhan, ...</td>\n",
              "      <td>In December 2019, a new type of unexplained pn...</td>\n",
              "      <td>Yafei Wang.  Ying Zhou...</td>\n",
              "      <td>Clinical Characteristics of Patients with&lt;br&gt;...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1 Background: A new virus broke out in Wuhan,...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>3b22eecad8a582436c52284a4db2198a98a94e18</td>\n",
              "      <td>The host antiviral response involves the induc...</td>\n",
              "      <td>Respiratory syncytial virus (RSV) belongs to t...</td>\n",
              "      <td>Robitaille, A. C..  Caron, E....</td>\n",
              "      <td>DUSP1 regulates apoptosis and cell migration,...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>The host antiviral response involves the&lt;br&gt;i...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                   paper_id  ...                                   abstract_summary\n",
              "0  dcd7a1235ea74e3ef71d051103bf8a64c3c8f457  ...   12 Background 13 After the outbreak of novel<...\n",
              "1  86b6b0c1b2777541feb83116bcb7a5cb12a52310  ...                                      Not provided.\n",
              "2  73d80c8f5780d70bd8d343188c56e898e91557b6  ...   Middle East respiratory syndrome coronavirus<...\n",
              "3  70cc2e5152d3dc4d44494124ff556c9bbe9e6f41  ...   1 Background: A new virus broke out in Wuhan,...\n",
              "4  3b22eecad8a582436c52284a4db2198a98a94e18  ...   The host antiviral response involves the<br>i...\n",
              "\n",
              "[5 rows x 7 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dS_Tcti1g00Y",
        "colab_type": "text"
      },
      "source": [
        "### Handle Possible Duplicates"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pt8YNzlEgD0P",
        "colab_type": "code",
        "outputId": "d09fd6c7-af78-400f-bf4b-d383f668749a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "df_covid.drop_duplicates(['abstract', 'body_text'], inplace=True)\n",
        "df_covid['abstract'].describe(include='all')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count     27663\n",
              "unique    20191\n",
              "top            \n",
              "freq       7444\n",
              "Name: abstract, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yJsPOSg0g6YQ",
        "colab_type": "code",
        "outputId": "8d46fe8f-f34c-4bcd-fdf2-41a671ec823d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "df_covid['body_text'].describe(include='all')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count                                                 27663\n",
              "unique                                                27662\n",
              "top       In a global world, knowledge of imported infec...\n",
              "freq                                                      2\n",
              "Name: body_text, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HOxMWZJtg_yB",
        "colab_type": "text"
      },
      "source": [
        "### Take a Look at the Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SEcHXnMbhAhH",
        "colab_type": "code",
        "outputId": "42c2e0cd-d3c1-42e5-f678-a6d2b6662fe5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "source": [
        "df_covid.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>paper_id</th>\n",
              "      <th>abstract</th>\n",
              "      <th>body_text</th>\n",
              "      <th>authors</th>\n",
              "      <th>title</th>\n",
              "      <th>journal</th>\n",
              "      <th>abstract_summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>dcd7a1235ea74e3ef71d051103bf8a64c3c8f457</td>\n",
              "      <td>12 Background 13 After the outbreak of novel c...</td>\n",
              "      <td>A novel coronavirus (2019-nCoV) appeared in De...</td>\n",
              "      <td>Xinhai Li.  Xumao Zhao...</td>\n",
              "      <td>The lockdown of Hubei Province causing&lt;br&gt;dif...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>12 Background 13 After the outbreak of novel&lt;...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>86b6b0c1b2777541feb83116bcb7a5cb12a52310</td>\n",
              "      <td></td>\n",
              "      <td>Firstly informed to World Health Organization ...</td>\n",
              "      <td>Jung, Y. J..  Park, G.-S....</td>\n",
              "      <td>Comparative analysis of primer-probe sets for...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Not provided.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>73d80c8f5780d70bd8d343188c56e898e91557b6</td>\n",
              "      <td>Middle East respiratory syndrome coronavirus (...</td>\n",
              "      <td>Coronaviruses (CoVs) comprise a family of enve...</td>\n",
              "      <td>Straus, M. R..  Tang, T....</td>\n",
              "      <td>Ca2+ ions promote fusion of Middle East&lt;br&gt;Re...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Middle East respiratory syndrome coronavirus&lt;...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>70cc2e5152d3dc4d44494124ff556c9bbe9e6f41</td>\n",
              "      <td>1 Background: A new virus broke out in Wuhan, ...</td>\n",
              "      <td>In December 2019, a new type of unexplained pn...</td>\n",
              "      <td>Yafei Wang.  Ying Zhou...</td>\n",
              "      <td>Clinical Characteristics of Patients with&lt;br&gt;...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1 Background: A new virus broke out in Wuhan,...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>3b22eecad8a582436c52284a4db2198a98a94e18</td>\n",
              "      <td>The host antiviral response involves the induc...</td>\n",
              "      <td>Respiratory syncytial virus (RSV) belongs to t...</td>\n",
              "      <td>Robitaille, A. C..  Caron, E....</td>\n",
              "      <td>DUSP1 regulates apoptosis and cell migration,...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>The host antiviral response involves the&lt;br&gt;i...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                   paper_id  ...                                   abstract_summary\n",
              "0  dcd7a1235ea74e3ef71d051103bf8a64c3c8f457  ...   12 Background 13 After the outbreak of novel<...\n",
              "1  86b6b0c1b2777541feb83116bcb7a5cb12a52310  ...                                      Not provided.\n",
              "2  73d80c8f5780d70bd8d343188c56e898e91557b6  ...   Middle East respiratory syndrome coronavirus<...\n",
              "3  70cc2e5152d3dc4d44494124ff556c9bbe9e6f41  ...   1 Background: A new virus broke out in Wuhan,...\n",
              "4  3b22eecad8a582436c52284a4db2198a98a94e18  ...   The host antiviral response involves the<br>i...\n",
              "\n",
              "[5 rows x 7 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Kt2icxzXhDJx",
        "colab_type": "code",
        "outputId": "32456b81-f926-4cbc-b882-95b78d90ff63",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 173
        }
      },
      "source": [
        "df_covid.describe()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>paper_id</th>\n",
              "      <th>abstract</th>\n",
              "      <th>body_text</th>\n",
              "      <th>authors</th>\n",
              "      <th>title</th>\n",
              "      <th>journal</th>\n",
              "      <th>abstract_summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>27663</td>\n",
              "      <td>27663</td>\n",
              "      <td>27663</td>\n",
              "      <td>26917</td>\n",
              "      <td>27619</td>\n",
              "      <td>26769</td>\n",
              "      <td>27663</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>unique</th>\n",
              "      <td>27663</td>\n",
              "      <td>20191</td>\n",
              "      <td>27662</td>\n",
              "      <td>25568</td>\n",
              "      <td>27240</td>\n",
              "      <td>3323</td>\n",
              "      <td>20184</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>top</th>\n",
              "      <td>ce248b901191d45f3e56f1e6664a0239738aa148</td>\n",
              "      <td></td>\n",
              "      <td>In a global world, knowledge of imported infec...</td>\n",
              "      <td>Domingo, Esteban</td>\n",
              "      <td>Index</td>\n",
              "      <td>PLoS One</td>\n",
              "      <td>Not provided.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>freq</th>\n",
              "      <td>1</td>\n",
              "      <td>7444</td>\n",
              "      <td>2</td>\n",
              "      <td>14</td>\n",
              "      <td>68</td>\n",
              "      <td>1511</td>\n",
              "      <td>7444</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                        paper_id  ... abstract_summary\n",
              "count                                      27663  ...            27663\n",
              "unique                                     27663  ...            20184\n",
              "top     ce248b901191d45f3e56f1e6664a0239738aa148  ...    Not provided.\n",
              "freq                                           1  ...             7444\n",
              "\n",
              "[4 rows x 7 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eOFo1hGZhWnj",
        "colab_type": "text"
      },
      "source": [
        "### Data Pre-Process"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ypxaLErKhGYQ",
        "colab_type": "code",
        "outputId": "cf65ec2f-f0b0-42fb-d93f-2ee3d552c4d9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        }
      },
      "source": [
        "df_covid.dropna(inplace=True)\n",
        "df_covid.info()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "Int64Index: 26043 entries, 885 to 27677\n",
            "Data columns (total 7 columns):\n",
            "paper_id            26043 non-null object\n",
            "abstract            26043 non-null object\n",
            "body_text           26043 non-null object\n",
            "authors             26043 non-null object\n",
            "title               26043 non-null object\n",
            "journal             26043 non-null object\n",
            "abstract_summary    26043 non-null object\n",
            "dtypes: object(7)\n",
            "memory usage: 1.6+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qDmPRIUchbj5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df_covid = df_covid.head(12500)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FH_LxRnxhfzY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import re\n",
        "\n",
        "df_covid['body_text'] = df_covid['body_text'].apply(lambda x: re.sub('[^a-zA-z0-9\\s]','',x))\n",
        "df_covid['abstract'] = df_covid['abstract'].apply(lambda x: re.sub('[^a-zA-z0-9\\s]','',x))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YwR2gI9fhiQP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def lower_case(input_str):\n",
        "    input_str = input_str.lower()\n",
        "    return input_str\n",
        "\n",
        "df_covid['body_text'] = df_covid['body_text'].apply(lambda x: lower_case(x))\n",
        "df_covid['abstract'] = df_covid['abstract'].apply(lambda x: lower_case(x))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HbSvi-5GhmBh",
        "colab_type": "code",
        "outputId": "f14fb98f-c29e-4775-b53e-dafe59d5c88a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 326
        }
      },
      "source": [
        "df_covid.head(4)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>paper_id</th>\n",
              "      <th>abstract</th>\n",
              "      <th>body_text</th>\n",
              "      <th>authors</th>\n",
              "      <th>title</th>\n",
              "      <th>journal</th>\n",
              "      <th>abstract_summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>885</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>footandmouth disease fmd an acute highly conta...</td>\n",
              "      <td>Gao, Yuan.  Sun, Shi-Qi...</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>Virol J</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>886</th>\n",
              "      <td>335b0a3f21f764adcbe20ff71e422d823c410098</td>\n",
              "      <td>background gray wolves canis lupus were reintr...</td>\n",
              "      <td>several highmortality disease outbreaks among ...</td>\n",
              "      <td>Almberg, Emily S..  Mech, L. David...</td>\n",
              "      <td>A Serological Survey of Infectious Disease in...</td>\n",
              "      <td>PLoS One</td>\n",
              "      <td>Background: Gray wolves (Canis lupus) were&lt;br...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>887</th>\n",
              "      <td>bad0e9f737316570c33138d5cc95cc233cd937ab</td>\n",
              "      <td>in niger acute respiratory infections aris are...</td>\n",
              "      <td>acute respiratory infections aris are responsi...</td>\n",
              "      <td>Lagare, Adamou.  Ousmane, Sani...</td>\n",
              "      <td>Molecular detection of respiratory pathogens&lt;...</td>\n",
              "      <td>Health Sci Rep</td>\n",
              "      <td>In Niger, acute respiratory infections (ARIs)...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>888</th>\n",
              "      <td>007bf75961da42a7e0cc8e2855e5c208a5ec65c1</td>\n",
              "      <td>the hemagglutininesterases hes envelope glycop...</td>\n",
              "      <td>to initiate infection viruses must bind to an ...</td>\n",
              "      <td>Langereis, Martijn A..  Zeng, Qinghong...</td>\n",
              "      <td>The Murine Coronavirus&lt;br&gt;Hemagglutinin-ester...</td>\n",
              "      <td>PLoS Pathog</td>\n",
              "      <td>The hemagglutinin-esterases (HEs), envelope&lt;b...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                     paper_id  ...                                   abstract_summary\n",
              "885  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "886  335b0a3f21f764adcbe20ff71e422d823c410098  ...   Background: Gray wolves (Canis lupus) were<br...\n",
              "887  bad0e9f737316570c33138d5cc95cc233cd937ab  ...   In Niger, acute respiratory infections (ARIs)...\n",
              "888  007bf75961da42a7e0cc8e2855e5c208a5ec65c1  ...   The hemagglutinin-esterases (HEs), envelope<b...\n",
              "\n",
              "[4 rows x 7 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7u1jrmcNcjeK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df_covid.to_csv(\"/content/drive/My Drive/BertSentenceSimilarity/Data/covid.csv\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "btwZT7iZdjlp",
        "colab_type": "code",
        "outputId": "0e09cb6c-c217-4ab1-f1c7-526c76c3d660",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 306
        }
      },
      "source": [
        "df_covid_test = pd.read_csv(\"/content/drive/My Drive/BertSentenceSimilarity/Data/covid.csv\")\n",
        "text = df_covid_test.drop([\"authors\", \"journal\", \"Unnamed: 0\"], axis=1)\n",
        "text.head(5)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>paper_id</th>\n",
              "      <th>abstract</th>\n",
              "      <th>body_text</th>\n",
              "      <th>title</th>\n",
              "      <th>abstract_summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>footandmouth disease fmd an acute highly conta...</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>335b0a3f21f764adcbe20ff71e422d823c410098</td>\n",
              "      <td>background gray wolves canis lupus were reintr...</td>\n",
              "      <td>several highmortality disease outbreaks among ...</td>\n",
              "      <td>A Serological Survey of Infectious Disease in...</td>\n",
              "      <td>Background: Gray wolves (Canis lupus) were&lt;br...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>bad0e9f737316570c33138d5cc95cc233cd937ab</td>\n",
              "      <td>in niger acute respiratory infections aris are...</td>\n",
              "      <td>acute respiratory infections aris are responsi...</td>\n",
              "      <td>Molecular detection of respiratory pathogens&lt;...</td>\n",
              "      <td>In Niger, acute respiratory infections (ARIs)...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>007bf75961da42a7e0cc8e2855e5c208a5ec65c1</td>\n",
              "      <td>the hemagglutininesterases hes envelope glycop...</td>\n",
              "      <td>to initiate infection viruses must bind to an ...</td>\n",
              "      <td>The Murine Coronavirus&lt;br&gt;Hemagglutinin-ester...</td>\n",
              "      <td>The hemagglutinin-esterases (HEs), envelope&lt;b...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>d6a325260dac29bfe718f1e57160583cb23b5908</td>\n",
              "      <td>emerging evidence suggests that dipeptidyl pep...</td>\n",
              "      <td>the global burden of diabetes is escalating at...</td>\n",
              "      <td>The role of renal dipeptidyl peptidase-4 in&lt;b...</td>\n",
              "      <td>Emerging evidence suggests that dipeptidyl&lt;br...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                   paper_id  ...                                   abstract_summary\n",
              "0  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "1  335b0a3f21f764adcbe20ff71e422d823c410098  ...   Background: Gray wolves (Canis lupus) were<br...\n",
              "2  bad0e9f737316570c33138d5cc95cc233cd937ab  ...   In Niger, acute respiratory infections (ARIs)...\n",
              "3  007bf75961da42a7e0cc8e2855e5c208a5ec65c1  ...   The hemagglutinin-esterases (HEs), envelope<b...\n",
              "4  d6a325260dac29bfe718f1e57160583cb23b5908  ...   Emerging evidence suggests that dipeptidyl<br...\n",
              "\n",
              "[5 rows x 5 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TPmt9OWTehJY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "text_dict = text.to_dict()\n",
        "len_text = len(text_dict[\"paper_id\"])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aELXYwyAht4a",
        "colab_type": "code",
        "outputId": "db20f74c-9deb-4b1a-82b6-c494a706e8f8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "paper_id_list  = []\n",
        "body_text_list = []\n",
        "\n",
        "title_list = []\n",
        "abstract_list = []\n",
        "abstract_summary_list = []\n",
        "for i in tqdm(range(0,len_text)):\n",
        "  paper_id = text_dict[\"paper_id\"][i]\n",
        "  body_text = text_dict[\"body_text\"][i].split(\"\\n\")\n",
        "  title = text_dict[\"title\"][i]\n",
        "  abstract = text_dict[\"abstract\"][i]\n",
        "  abstract_summary = text_dict[\"abstract_summary\"][i]\n",
        "  for b in body_text:\n",
        "    paper_id_list.append(paper_id)\n",
        "    body_text_list.append(b)\n",
        "    title_list.append(title)\n",
        "    abstract_list.append(abstract)\n",
        "    abstract_summary_list.append(abstract_summary)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 12500/12500 [00:00<00:00, 23067.45it/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VCgvs7VjWlGo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df_sentences = pd.DataFrame({\"paper_id\":paper_id_list},index=body_text_list)\n",
        "df_sentences.to_csv(\"/content/drive/My Drive/BertSentenceSimilarity/Data/covid_sentences.csv\")\n",
        "df_sentences.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vgi6-K52eSCq",
        "colab_type": "code",
        "outputId": "f394a193-dd80-46ba-a567-b09e55045c5c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "df_sentences = pd.DataFrame({\"paper_id\":paper_id_list,\"title\":title_list,\"abstract\":abstract_list,\"abstract_summary\":abstract_summary_list},index=body_text_list)\n",
        "df_sentences.to_csv(\"/content/drive/My Drive/BertSentenceSimilarity/Data/covid_sentences_Full.csv\")\n",
        "df_sentences.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>paper_id</th>\n",
              "      <th>title</th>\n",
              "      <th>abstract</th>\n",
              "      <th>abstract_summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>footandmouth disease fmd an acute highly contagious viral disease in susceptible clovenhoofed animals was described 100 years ago the etiologic agent fmd virus fmdv is a positivesense singlestranded rna virus that belongs to the aphthovirus genus picornaviridae family fmdv is one of the most contagious viruses in clovenhoofed animals and can cause both acute and prolonged asymptomatic but persistent infection [1]  upon infection of susceptible species fmdv proliferates rapidly and causes vesicular disease in feet and mouth</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the rna virus genome of fmdv displays a very high mutation rate because the virusencoded rna polymerase lacks a proofreading mechanism [2 3]  the high mutation rate of fmdv coupled with its rapid proliferation and extensive population result in the rapid evolution of this virus [4]  which contributes to the existence of seven main serotypes a o c asia1 south african territories sat 1 sat2 and sat3 in addition numerous variants and subtypes have been further evolved from each serotype [1]  given that crossreactivity varies antigenic diversity among these serotypes have to be considered during vaccine development [5]</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>fmdv virion has a symmetric protein shell or capsid enclosing the genomic rna genome rna contains a positive singlestrand chain approximately 83 kb long and encodes a single long open reading frame orf of about 7 kb with two alternative initiation sites the orf is flanked by a long 5untranslated region 5utr and a short 3utr and ends with a genetically encoded polya tail [6]  a genomelinked viral nonstructural protein nsp 3b also known as vpg containing 2324 amino acid aa residues is covalently bound to its 5 end although this protein is rapidly released into an infected cell and is deemed to play no part in translation initiation [7]  the viral orf can be translated into a polyprotein of about 250 kda which is subsequently cleaved by two virusencoded proteinases leader l pro  and 3c pro  to yield structural and nsps [8 9] fig 1</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the fmdv genome was completely sequenced and all cleavage sites involved in the processing of polypeptides were also identified in the past two decades generally the orf region in fmdv genome is artificially divided into four functional areas due to the different functions of mature polypeptides [10]  which are shown as follows  fig 1  l region which is located at 5 end to the capsid component and codes for l pro  p1 region encoding a precursor for capsid polypeptide which can generate four mature capsid proteins vp4 vp2 vp3 and vp1 upon cleavage by viral protease p2 region encodes three viral proteins 2a 2b and 2c in the middle region of the genome and p3 region which encodes four viral proteins 3a 3b 3c pro and 3d pol  in which 3c is a viral protease and 3d an rnadependent rna polymerase [11]  actually primary polyprotein is not strictly processed into four products as the functional regions by initial protease but l pro  p12a 2bc and p3 by l pro  2a and 3c pro  the precursors p12a 2bc and p3 are further processed into mature viral proteins and some cleavage intermediates with relative stability such as vp0 or 1ab 3abc 3bcd 3ab and 3cd by 3c pro fig 1  usually the intermediates may perform functions other than those of their individual constituents with two alternative initiation sites the orf is flanked by a long 5untranslated region 5utr and a short 3utr 3b vpg is covalently bound to its 5 end the orf region is generally divided into four functional areas l p1 p2 and p3 due to the different functions of mature polypeptides orfencoded polyprotein is processed into four products l pro  p12a 2bc and p3 by l pro  2a and 3c pro  the precursors p12a 2bc and p3 are further processed into mature viral proteins and some cleavage intermediates with relative stability such as vp0 or 1ab 3abc 3bcd 3ab and 3cd by 3c pro  structural proteins form the biological protomer and viral capsid</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the virus capsid consists of 60 copies of each of the four structural polypeptides vp1 to vp4 which are selfassembled into an icosahedral structure with a diameter of 30 nm [12 13] fig 1  studies on structural information and protein interaction have shown that the structural protein or the precursor products vp0 vp24 or 1ab vp1 1d and vp3 1c which are encoded by p1 region form immature protomers through weak chemical bond interaction then pentamers are assembled by five protomers [14]  after selfassembly of pentamers to generate an empty capsid the viral genomic rna covalently linked to vpg at the 5 end enters the capsid to produce provirion then the provirion is eventually processed into a mature virion following the rnatriggered autocleavage of vp0 [15]  finally the virion particles with complete assembly are released from the infected host cells fig 2</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                                                    paper_id  ...                                   abstract_summary\n",
              "footandmouth disease fmd an acute highly contag...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "the rna virus genome of fmdv displays a very hi...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "fmdv virion has a symmetric protein shell or ca...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "the fmdv genome was completely sequenced and al...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "the virus capsid consists of 60 copies of each ...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "\n",
              "[5 rows x 4 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sFVjsMb8xPkD",
        "colab_type": "text"
      },
      "source": [
        "## Preparing Data for Embedding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "71E8Y5XJpZ2S",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "from tqdm import tqdm\n",
        "\n",
        "df_sentences = pd.read_csv(\"/content/drive/My Drive/BertSentenceSimilarity/Data/covid_sentences.csv\")\n",
        "df_sentences = df_sentences.set_index(\"Unnamed: 0\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "brsFIaJ2y6RI",
        "colab_type": "code",
        "outputId": "6f6841ef-5374-4916-f8d1-a2f3ea296dd0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 779
        }
      },
      "source": [
        "df_sentences.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>paper_id</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>footandmouth disease fmd an acute highly contagious viral disease in susceptible clovenhoofed animals was described 100 years ago the etiologic agent fmd virus fmdv is a positivesense singlestranded rna virus that belongs to the aphthovirus genus picornaviridae family fmdv is one of the most contagious viruses in clovenhoofed animals and can cause both acute and prolonged asymptomatic but persistent infection [1]  upon infection of susceptible species fmdv proliferates rapidly and causes vesicular disease in feet and mouth</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the rna virus genome of fmdv displays a very high mutation rate because the virusencoded rna polymerase lacks a proofreading mechanism [2 3]  the high mutation rate of fmdv coupled with its rapid proliferation and extensive population result in the rapid evolution of this virus [4]  which contributes to the existence of seven main serotypes a o c asia1 south african territories sat 1 sat2 and sat3 in addition numerous variants and subtypes have been further evolved from each serotype [1]  given that crossreactivity varies antigenic diversity among these serotypes have to be considered during vaccine development [5]</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>fmdv virion has a symmetric protein shell or capsid enclosing the genomic rna genome rna contains a positive singlestrand chain approximately 83 kb long and encodes a single long open reading frame orf of about 7 kb with two alternative initiation sites the orf is flanked by a long 5untranslated region 5utr and a short 3utr and ends with a genetically encoded polya tail [6]  a genomelinked viral nonstructural protein nsp 3b also known as vpg containing 2324 amino acid aa residues is covalently bound to its 5 end although this protein is rapidly released into an infected cell and is deemed to play no part in translation initiation [7]  the viral orf can be translated into a polyprotein of about 250 kda which is subsequently cleaved by two virusencoded proteinases leader l pro  and 3c pro  to yield structural and nsps [8 9] fig 1</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the fmdv genome was completely sequenced and all cleavage sites involved in the processing of polypeptides were also identified in the past two decades generally the orf region in fmdv genome is artificially divided into four functional areas due to the different functions of mature polypeptides [10]  which are shown as follows  fig 1  l region which is located at 5 end to the capsid component and codes for l pro  p1 region encoding a precursor for capsid polypeptide which can generate four mature capsid proteins vp4 vp2 vp3 and vp1 upon cleavage by viral protease p2 region encodes three viral proteins 2a 2b and 2c in the middle region of the genome and p3 region which encodes four viral proteins 3a 3b 3c pro and 3d pol  in which 3c is a viral protease and 3d an rnadependent rna polymerase [11]  actually primary polyprotein is not strictly processed into four products as the functional regions by initial protease but l pro  p12a 2bc and p3 by l pro  2a and 3c pro  the precursors p12a 2bc and p3 are further processed into mature viral proteins and some cleavage intermediates with relative stability such as vp0 or 1ab 3abc 3bcd 3ab and 3cd by 3c pro fig 1  usually the intermediates may perform functions other than those of their individual constituents with two alternative initiation sites the orf is flanked by a long 5untranslated region 5utr and a short 3utr 3b vpg is covalently bound to its 5 end the orf region is generally divided into four functional areas l p1 p2 and p3 due to the different functions of mature polypeptides orfencoded polyprotein is processed into four products l pro  p12a 2bc and p3 by l pro  2a and 3c pro  the precursors p12a 2bc and p3 are further processed into mature viral proteins and some cleavage intermediates with relative stability such as vp0 or 1ab 3abc 3bcd 3ab and 3cd by 3c pro  structural proteins form the biological protomer and viral capsid</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the virus capsid consists of 60 copies of each of the four structural polypeptides vp1 to vp4 which are selfassembled into an icosahedral structure with a diameter of 30 nm [12 13] fig 1  studies on structural information and protein interaction have shown that the structural protein or the precursor products vp0 vp24 or 1ab vp1 1d and vp3 1c which are encoded by p1 region form immature protomers through weak chemical bond interaction then pentamers are assembled by five protomers [14]  after selfassembly of pentamers to generate an empty capsid the viral genomic rna covalently linked to vpg at the 5 end enters the capsid to produce provirion then the provirion is eventually processed into a mature virion following the rnatriggered autocleavage of vp0 [15]  finally the virion particles with complete assembly are released from the infected host cells fig 2</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                                                    paper_id\n",
              "Unnamed: 0                                                                                  \n",
              "footandmouth disease fmd an acute highly contag...  db5333b01a10f165ae516d30f9d1fbf96ab4b841\n",
              "the rna virus genome of fmdv displays a very hi...  db5333b01a10f165ae516d30f9d1fbf96ab4b841\n",
              "fmdv virion has a symmetric protein shell or ca...  db5333b01a10f165ae516d30f9d1fbf96ab4b841\n",
              "the fmdv genome was completely sequenced and al...  db5333b01a10f165ae516d30f9d1fbf96ab4b841\n",
              "the virus capsid consists of 60 copies of each ...  db5333b01a10f165ae516d30f9d1fbf96ab4b841"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZsvaWGEdonX5",
        "colab_type": "code",
        "outputId": "4c9cddc0-20e1-4fe0-a80a-c0f52b46d384",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "df_sentences = df_sentences[\"paper_id\"].to_dict()\n",
        "df_sentences_list = list(df_sentences.keys())\n",
        "len(df_sentences_list)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "403341"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dArptx27xrlg",
        "colab_type": "code",
        "outputId": "e5196015-0fd7-4264-eb07-4fa7a49e4f88",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        }
      },
      "source": [
        "list(df_sentences.keys())[:5]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['footandmouth disease fmd an acute highly contagious viral disease in susceptible clovenhoofed animals was described 100 years ago the etiologic agent fmd virus fmdv is a positivesense singlestranded rna virus that belongs to the aphthovirus genus picornaviridae family fmdv is one of the most contagious viruses in clovenhoofed animals and can cause both acute and prolonged asymptomatic but persistent infection [1]  upon infection of susceptible species fmdv proliferates rapidly and causes vesicular disease in feet and mouth',\n",
              " 'the rna virus genome of fmdv displays a very high mutation rate because the virusencoded rna polymerase lacks a proofreading mechanism [2 3]  the high mutation rate of fmdv coupled with its rapid proliferation and extensive population result in the rapid evolution of this virus [4]  which contributes to the existence of seven main serotypes a o c asia1 south african territories sat 1 sat2 and sat3 in addition numerous variants and subtypes have been further evolved from each serotype [1]  given that crossreactivity varies antigenic diversity among these serotypes have to be considered during vaccine development [5] ',\n",
              " 'fmdv virion has a symmetric protein shell or capsid enclosing the genomic rna genome rna contains a positive singlestrand chain approximately 83 kb long and encodes a single long open reading frame orf of about 7 kb with two alternative initiation sites the orf is flanked by a long 5untranslated region 5utr and a short 3utr and ends with a genetically encoded polya tail [6]  a genomelinked viral nonstructural protein nsp 3b also known as vpg containing 2324 amino acid aa residues is covalently bound to its 5 end although this protein is rapidly released into an infected cell and is deemed to play no part in translation initiation [7]  the viral orf can be translated into a polyprotein of about 250 kda which is subsequently cleaved by two virusencoded proteinases leader l pro  and 3c pro  to yield structural and nsps [8 9] fig 1 ',\n",
              " 'the fmdv genome was completely sequenced and all cleavage sites involved in the processing of polypeptides were also identified in the past two decades generally the orf region in fmdv genome is artificially divided into four functional areas due to the different functions of mature polypeptides [10]  which are shown as follows  fig 1  l region which is located at 5 end to the capsid component and codes for l pro  p1 region encoding a precursor for capsid polypeptide which can generate four mature capsid proteins vp4 vp2 vp3 and vp1 upon cleavage by viral protease p2 region encodes three viral proteins 2a 2b and 2c in the middle region of the genome and p3 region which encodes four viral proteins 3a 3b 3c pro and 3d pol  in which 3c is a viral protease and 3d an rnadependent rna polymerase [11]  actually primary polyprotein is not strictly processed into four products as the functional regions by initial protease but l pro  p12a 2bc and p3 by l pro  2a and 3c pro  the precursors p12a 2bc and p3 are further processed into mature viral proteins and some cleavage intermediates with relative stability such as vp0 or 1ab 3abc 3bcd 3ab and 3cd by 3c pro fig 1  usually the intermediates may perform functions other than those of their individual constituents with two alternative initiation sites the orf is flanked by a long 5untranslated region 5utr and a short 3utr 3b vpg is covalently bound to its 5 end the orf region is generally divided into four functional areas l p1 p2 and p3 due to the different functions of mature polypeptides orfencoded polyprotein is processed into four products l pro  p12a 2bc and p3 by l pro  2a and 3c pro  the precursors p12a 2bc and p3 are further processed into mature viral proteins and some cleavage intermediates with relative stability such as vp0 or 1ab 3abc 3bcd 3ab and 3cd by 3c pro  structural proteins form the biological protomer and viral capsid',\n",
              " 'the virus capsid consists of 60 copies of each of the four structural polypeptides vp1 to vp4 which are selfassembled into an icosahedral structure with a diameter of 30 nm [12 13] fig 1  studies on structural information and protein interaction have shown that the structural protein or the precursor products vp0 vp24 or 1ab vp1 1d and vp3 1c which are encoded by p1 region form immature protomers through weak chemical bond interaction then pentamers are assembled by five protomers [14]  after selfassembly of pentamers to generate an empty capsid the viral genomic rna covalently linked to vpg at the 5 end enters the capsid to produce provirion then the provirion is eventually processed into a mature virion following the rnatriggered autocleavage of vp0 [15]  finally the virion particles with complete assembly are released from the infected host cells fig 2 ']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CoNUPZ740Gd4",
        "colab_type": "code",
        "outputId": "fb73eda8-289f-4632-b7f7-2e5b4ae31109",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "df_sentences_list = [str(d) for d in tqdm(df_sentences_list)]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 403341/403341 [00:00<00:00, 1913170.91it/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rQGN0-qbU09N",
        "colab_type": "code",
        "outputId": "46874a6c-868b-422c-eed5-a4f11dceef36",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv(\"/content/drive/My Drive/BertSentenceSimilarity/Data/covid_sentences_Full.csv\", index_col=0)\n",
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>paper_id</th>\n",
              "      <th>title</th>\n",
              "      <th>abstract</th>\n",
              "      <th>abstract_summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>footandmouth disease fmd an acute highly contagious viral disease in susceptible clovenhoofed animals was described 100 years ago the etiologic agent fmd virus fmdv is a positivesense singlestranded rna virus that belongs to the aphthovirus genus picornaviridae family fmdv is one of the most contagious viruses in clovenhoofed animals and can cause both acute and prolonged asymptomatic but persistent infection [1]  upon infection of susceptible species fmdv proliferates rapidly and causes vesicular disease in feet and mouth</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the rna virus genome of fmdv displays a very high mutation rate because the virusencoded rna polymerase lacks a proofreading mechanism [2 3]  the high mutation rate of fmdv coupled with its rapid proliferation and extensive population result in the rapid evolution of this virus [4]  which contributes to the existence of seven main serotypes a o c asia1 south african territories sat 1 sat2 and sat3 in addition numerous variants and subtypes have been further evolved from each serotype [1]  given that crossreactivity varies antigenic diversity among these serotypes have to be considered during vaccine development [5]</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>fmdv virion has a symmetric protein shell or capsid enclosing the genomic rna genome rna contains a positive singlestrand chain approximately 83 kb long and encodes a single long open reading frame orf of about 7 kb with two alternative initiation sites the orf is flanked by a long 5untranslated region 5utr and a short 3utr and ends with a genetically encoded polya tail [6]  a genomelinked viral nonstructural protein nsp 3b also known as vpg containing 2324 amino acid aa residues is covalently bound to its 5 end although this protein is rapidly released into an infected cell and is deemed to play no part in translation initiation [7]  the viral orf can be translated into a polyprotein of about 250 kda which is subsequently cleaved by two virusencoded proteinases leader l pro  and 3c pro  to yield structural and nsps [8 9] fig 1</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the fmdv genome was completely sequenced and all cleavage sites involved in the processing of polypeptides were also identified in the past two decades generally the orf region in fmdv genome is artificially divided into four functional areas due to the different functions of mature polypeptides [10]  which are shown as follows  fig 1  l region which is located at 5 end to the capsid component and codes for l pro  p1 region encoding a precursor for capsid polypeptide which can generate four mature capsid proteins vp4 vp2 vp3 and vp1 upon cleavage by viral protease p2 region encodes three viral proteins 2a 2b and 2c in the middle region of the genome and p3 region which encodes four viral proteins 3a 3b 3c pro and 3d pol  in which 3c is a viral protease and 3d an rnadependent rna polymerase [11]  actually primary polyprotein is not strictly processed into four products as the functional regions by initial protease but l pro  p12a 2bc and p3 by l pro  2a and 3c pro  the precursors p12a 2bc and p3 are further processed into mature viral proteins and some cleavage intermediates with relative stability such as vp0 or 1ab 3abc 3bcd 3ab and 3cd by 3c pro fig 1  usually the intermediates may perform functions other than those of their individual constituents with two alternative initiation sites the orf is flanked by a long 5untranslated region 5utr and a short 3utr 3b vpg is covalently bound to its 5 end the orf region is generally divided into four functional areas l p1 p2 and p3 due to the different functions of mature polypeptides orfencoded polyprotein is processed into four products l pro  p12a 2bc and p3 by l pro  2a and 3c pro  the precursors p12a 2bc and p3 are further processed into mature viral proteins and some cleavage intermediates with relative stability such as vp0 or 1ab 3abc 3bcd 3ab and 3cd by 3c pro  structural proteins form the biological protomer and viral capsid</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>the virus capsid consists of 60 copies of each of the four structural polypeptides vp1 to vp4 which are selfassembled into an icosahedral structure with a diameter of 30 nm [12 13] fig 1  studies on structural information and protein interaction have shown that the structural protein or the precursor products vp0 vp24 or 1ab vp1 1d and vp3 1c which are encoded by p1 region form immature protomers through weak chemical bond interaction then pentamers are assembled by five protomers [14]  after selfassembly of pentamers to generate an empty capsid the viral genomic rna covalently linked to vpg at the 5 end enters the capsid to produce provirion then the provirion is eventually processed into a mature virion following the rnatriggered autocleavage of vp0 [15]  finally the virion particles with complete assembly are released from the infected host cells fig 2</th>\n",
              "      <td>db5333b01a10f165ae516d30f9d1fbf96ab4b841</td>\n",
              "      <td>Biological function of Foot-and-mouth&lt;br&gt;dise...</td>\n",
              "      <td>footandmouth disease virus fmdv represses host...</td>\n",
              "      <td>Foot-and-mouth disease virus (FMDV)&lt;br&gt;repres...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                                                    paper_id  ...                                   abstract_summary\n",
              "footandmouth disease fmd an acute highly contag...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "the rna virus genome of fmdv displays a very hi...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "fmdv virion has a symmetric protein shell or ca...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "the fmdv genome was completely sequenced and al...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "the virus capsid consists of 60 copies of each ...  db5333b01a10f165ae516d30f9d1fbf96ab4b841  ...   Foot-and-mouth disease virus (FMDV)<br>repres...\n",
              "\n",
              "[5 rows x 4 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_p2-bJT_EyZL",
        "colab_type": "text"
      },
      "source": [
        "## BERT"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F_Zmp_o7k5FK",
        "colab_type": "code",
        "outputId": "ce5219ca-f451-4b86-c9be-17632013e564",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "#https://github.com/UKPLab/sentence-transformers/blob/master/examples/application_semantic_search.py\n",
        "\"\"\"\n",
        "This is a simple application for sentence embeddings: semantic search\n",
        "We have a corpus with various sentences. Then, for a given query sentence,\n",
        "we want to find the most similar sentence in this corpus.\n",
        "This script outputs for various queries the top 5 most similar sentences in the corpus.\n",
        "\"\"\"\n",
        "\n",
        "from sentence_transformers import SentenceTransformer\n",
        "import scipy.spatial\n",
        "import pickle as pkl\n",
        "embedder = SentenceTransformer('bert-base-nli-mean-tokens')\n",
        "\n",
        "# Corpus with example sentences\n",
        "corpus = df_sentences_list\n",
        "#corpus_embeddings = embedder.encode(corpus,show_progress_bar=True)\n",
        "with open(\"/content/drive/My Drive/BertSentenceSimilarity/Pickles/corpus_embeddings.pkl\" , \"rb\") as file_:\n",
        "  corpus_embeddings = pkl.load(file_)\n",
        "\n",
        "# Query sentences:\n",
        "queries = ['What has been published about medical care?',\n",
        "           'Knowledge of the frequency, manifestations, and course of extrapulmonary manifestations of COVID-19, including, but not limited to, possible cardiomyopathy and cardiac arrest',\n",
        "           'Use of AI in real-time health care delivery to evaluate interventions, risk factors, and outcomes in a way that could not be done manually',\n",
        "           'Resources to support skilled nursing facilities and long term care facilities.',\n",
        "           'Mobilization of surge medical staff to address shortages in overwhelmed communities .',\n",
        "           'Age-adjusted mortality data for Acute Respiratory Distress Syndrome (ARDS) with/without other organ failure – particularly for viral etiologies .']\n",
        "query_embeddings = embedder.encode(queries,show_progress_bar=True)\n",
        "\n",
        "# Find the closest 5 sentences of the corpus for each query sentence based on cosine similarity\n",
        "closest_n = 5\n",
        "print(\"\\nTop 5 most similar sentences in corpus:\")\n",
        "for query, query_embedding in zip(queries, query_embeddings):\n",
        "    distances = scipy.spatial.distance.cdist([query_embedding], corpus_embeddings, \"cosine\")[0]\n",
        "\n",
        "    results = zip(range(len(distances)), distances)\n",
        "    results = sorted(results, key=lambda x: x[1])\n",
        "\n",
        "    print(\"\\n\\n=========================================================\")\n",
        "    print(\"==========================Query==============================\")\n",
        "    print(\"===\",query,\"=====\")\n",
        "    print(\"=========================================================\")\n",
        "\n",
        "\n",
        "    for idx, distance in results[0:closest_n]:\n",
        "        print(\"Score:   \", \"(Score: %.4f)\" % (1-distance) , \"\\n\" )\n",
        "        print(\"Paragraph:   \", corpus[idx].strip(), \"\\n\" )\n",
        "        row_dict = df.loc[df.index== corpus[idx]].to_dict()\n",
        "        print(\"paper_id:  \" , row_dict[\"paper_id\"][corpus[idx]] , \"\\n\")\n",
        "        print(\"Title:  \" , row_dict[\"title\"][corpus[idx]] , \"\\n\")\n",
        "        print(\"Abstract:  \" , row_dict[\"abstract\"][corpus[idx]] , \"\\n\")\n",
        "        print(\"Abstract_Summary:  \" , row_dict[\"abstract_summary\"][corpus[idx]] , \"\\n\")\n",
        "        print(\"-------------------------------------------\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Batches: 100%|██████████| 1/1 [00:00<00:00, 27.80it/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Top 5 most similar sentences in corpus:\n",
            "\n",
            "\n",
            "=========================================================\n",
            "==========================Query==============================\n",
            "=== What has been published about medical care? =====\n",
            "=========================================================\n",
            "Score:    (Score: 0.8296) \n",
            "\n",
            "Paragraph:    how may state authorities require persons to undergo medical treatment \n",
            "\n",
            "paper_id:   1950c30fea7ef227129d94831df3fd0c57b9802c \n",
            "\n",
            "Title:    Chapter 10 Legal Aspects of Biosecurity \n",
            "\n",
            "Abstract:   when bad men combine the good must associate else they will fall one by one an unpitied sacrifice in a contemptible struggle\n",
            "the study of this chapter will enable you to\n",
            "1 discuss the definitions of terrorism and weapons of mass destruction and their relation to the illicit use of biological agents\n",
            "2 list all legislative and administrative documents that address the legal aspects of the unlawful use of biological agents\n",
            "4 discuss the prohibited uses of biological agents under international law 5 list and briefly discuss the homeland security presidential directives that apply to biosecurity and biodefense \n",
            "\n",
            "Abstract_Summary:    When bad men combine, the good must associate;<br>else they will fall one by one, an unpitied sacrifice<br>in a contemptible struggle.\n",
            "The study of this<br>chapter will enable you to:\n",
            "1. Discuss the definitions<br>of terrorism and weapons of mass destruction and<br>their relation to the illicit use of biological<br>agents.\n",
            "2. List all legislative and administrative<br>documents that address the legal aspects of the unlawful<br>use of biological agents.\n",
            "4. Discuss the<br>prohibited uses of biological agents under international<br>law. 5. List and briefly discuss the Homeland<br>Security Presidential Directives that apply to<br>biosecurity and biodefense. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.8220) \n",
            "\n",
            "Paragraph:    to identify how one health has been used recently in the medical literature \n",
            "\n",
            "paper_id:   f703b510de361a759c9fe3419fa3aeb092c95512 \n",
            "\n",
            "Title:    One Health and Zoonoses: The Evolution of One<br>Health and Incorporation of Zoonoses \n",
            "\n",
            "Abstract:   nan \n",
            "\n",
            "Abstract_Summary:   Not provided. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.8187) \n",
            "\n",
            "Paragraph:    medical questions in the personal questionnaire included the following \n",
            "\n",
            "paper_id:   2afa3da371e5495dd55f0b4dde4ae7700eaf99d6 \n",
            "\n",
            "Title:    Rhinitis, Asthma and Respiratory Infections<br>among Adults in Relation to the Home Environment in<br>Multi-Family Buildings in Sweden \n",
            "\n",
            "Abstract:   risk factors for rhinitis asthma and respiratory infections in the home environment were studied by a questionnaire survey totally 5775 occupants 18 years old from a stratified random sample of multifamily buildings in sweden participated 46 510 had rhinitis in the last 3 months current rhinitis 115 doctor diagnosed asthma 464 respiratory infections in the last 3 months and 119 antibiotic medication for respiratory infections in the last 12 months associations between home environment and health were analyzed by multiple logistic regression controlling for gender age and smoking and mutual adjustment buildings constructed during 19601975 were risk factors for day time breathlessness or  153 95ci 103229 and those constructed during 19761985 had more current rhinitis or  143 95ci 112184 and respiratory infections or  146 95ci 121178 cities with higher population density had more current rhinitis p  0008 and respiratory infections p0001 rented apartments had more current rhinitis or  123 95ci 107140 wheeze or  120 95ci 102141 day time breathlessness or  131 95ci 104166 and respiratory infections or  113 95ci 101126 living in colder parts of the country was a risk factor for wheeze p  003 and night time breathlessness p  0002 building dampness was a risk factor for wheeze or  142 95ci 108186 and day time breathlessness or  157 95ci 109227 building dampness was a risk factor for health among those below 66 years old odor at home was a risk factor for doctor diagnosed asthma or  149 95ci 108206 and current asthma or  152 95ci 103224 environmental tobacco smoke ets was a risk factor for current asthma or  153 95ci 109216 window pane condensation was a risk factor for antibiotic medication for respiratory infections or  141 95ci 110182 in conclusion rhinitis asthma and respiratory infections were related to a number of factors in the home environment certain building years  building dampness window pane condensation and odor in the dwelling may be risk factors \n",
            "\n",
            "Abstract_Summary:    Risk factors for rhinitis, asthma and<br>respiratory infections in the home environment were<br>studied by a questionnaire survey. Totally 5775<br>occupants ($18 years old) from a stratified random sample<br>of multi-family buildings in Sweden<br>participated (46%). 51.0% had rhinitis in the last 3 months<br>(current rhinitis); 11.5% doctor diagnosed asthma;<br>46.4% respiratory infections in the last 3 months and<br>11.9% antibiotic medication for respiratory<br>infections in the last 12 months. Associations between<br>home environment and health were analyzed by<br>multiple logistic regression, controlling for gender,<br>age and smoking and mutual adjustment. Buildings<br>constructed during 1960-1975 were risk factors for day... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7816) \n",
            "\n",
            "Paragraph:    when you spoke to nhs direct what did they tell you \n",
            "\n",
            "paper_id:   6fc8c1b4cfbd790cbe02c825fd6f9181b1fc6645 \n",
            "\n",
            "Title:    Reassuring and managing patients with<br>concerns about swine flu: Qualitative interviews with<br>callers to NHS Direct \n",
            "\n",
            "Abstract:   background during the early stages of the 2009 swine flu influenza h1n1 outbreak the large majority of patients who contacted the health services about the illness did not have it in the uk the nhs direct telephone service was used by many of these patients we used qualitative interviews to identify the main reasons why people approached nhs direct with concerns about swine flu and to identify aspects of their contact which were reassuring using a framework approach methods 33 patients participated in semistructured interviews all patients had telephoned nhs direct between 11 and 14 may with concerns about swine flu and had been assessed as being unlikely to have the illness results reasons for seeking advice about swine flu included the presence of unexpectedly severe flulike symptoms uncertainties about how one can catch swine flu concern about giving it to others pressure from friends or employers and seeking peace of mind most participants found speaking to nhs direct reassuring or useful helpful aspects included having swine flu ruled out receiving an alternative explanation for symptoms clarification on how swine flu is transmitted and the perceived credibility of nhs direct noone reported anything that had increased their anxiety and only one participant subsequently sought additional advice about swine flu from elsewhere conclusions future major incidents involving other forms of chemical biological or radiological hazards may also cause large numbers of unexposed people to seek health advice our data suggest that providing telephone triage and information is helpful in such instances particularly where advice can be given via a trusted preexisting service \n",
            "\n",
            "Abstract_Summary:    Background: During the early stages of the 2009<br>swine flu (influenza H1N1) outbreak, the large<br>majority of patients who contacted the health services<br>about the illness did not have it. In the UK, the NHS<br>Direct telephone service was used by many of these<br>patients. We used qualitative interviews to identify the<br>main reasons why people approached NHS Direct with<br>concerns about swine flu and to identify aspects of their<br>contact which were reassuring, using a framework<br>approach. Methods: 33 patients participated in<br>semi-structured interviews. All patients had telephoned NHS<br>Direct between 11 and 14 May with concerns about swine... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7758) \n",
            "\n",
            "Paragraph:    how should this patient be evaluated and treated \n",
            "\n",
            "paper_id:   31881cc2530107849da340e6a16c9618b7e7bb8d \n",
            "\n",
            "Title:    Hemophagocytic lymphohistiocytosis<br>associated with viral infections: Diagnostic challenge<br>and therapeutic dilemma \n",
            "\n",
            "Abstract:   hemophagocytic lymphohistiocytosis is a frequently fatal clinicopathologic syndrome in which an uncontrolled and ineffective immune response leads to severe hyperinflammation it may occur as either a familial disorder or a sporadic condition in association with a variety of triggers infections malignancies autoimmune diseases and acquired immune deficiencies however the most consistent association is with viral infections especially epsteinbarr virus the main clinical features are fever liver dysfunction coagulation abnormalities and pancytopenia early diagnosis and treatment are important to reducing mortality but the diagnosis is difficult because of the rarity of the syndrome and the lack of specificity of the clinical findings treatment should be directed toward treating the underlying disease and to suppressing the exaggerated inflammatory response through the use of immunosuppressive agents\n",
            "resumen la linfohistiocitosis hemofagoctica es un sndrome clinicopatolgico de evolucin potencialmente fatal en el que una respuesta inmune no controlada e ineficaz conduce a hiperinflamacin puede aparecer como una enfermedad familiar o espordica asociado a diferentes factores desencadenantes infecciones neoplasias enfermedades autoinmunes o inmunodeficiencias adquiridas pero la asociacin ms consistente es con infecciones virales  please cite this article as mostazafernndez jl guerra laso j carriedo ule d ruiz de morales jmg linfohistiocitosis hemofagoctica asociada a infecciones virales reto diagnstico y dilema teraputico rev clin esp 2014214320327 \n",
            "\n",
            "Abstract_Summary:    Hemophagocytic lymphohistiocytosis is a<br>frequently fatal clinicopathologic syndrome in which an<br>uncontrolled and ineffective immune response leads to<br>severe hyperinflammation. It may occur as either a<br>familial disorder or a sporadic condition in<br>association with a variety of triggers: infections,<br>malignancies, autoimmune diseases, and acquired immune<br>deficiencies. However, the most consistent association is<br>with viral infections, especially Epstein---Barr<br>virus. The main clinical features are fever, liver<br>dysfunction, coagulation abnormalities and pancytopenia.<br>Early diagnosis and treatment are important to<br>reducing mortality, but the diagnosis is difficult<br>because of the rarity of the syndrome and the lack of<br>specificity of the clinical... \n",
            "\n",
            "-------------------------------------------\n",
            "\n",
            "\n",
            "=========================================================\n",
            "==========================Query==============================\n",
            "=== Knowledge of the frequency, manifestations, and course of extrapulmonary manifestations of COVID-19, including, but not limited to, possible cardiomyopathy and cardiac arrest =====\n",
            "=========================================================\n",
            "Score:    (Score: 0.8139) \n",
            "\n",
            "Paragraph:    clinical signs in hcm are explained by leftsided chf complications of arterial thromboembolism ate lv outflow tract obstruction or arrhythmias capable of causing syncope or sudden cardiac death \n",
            "\n",
            "paper_id:   1fdc1a513291d7613c95bbefae5b7ea0e811d5a6 \n",
            "\n",
            "Title:    Chapter 150 Cardiomyopathy \n",
            "\n",
            "Abstract:   1527\n",
            "1528 section 11  cardiopulmonary system this chapter will next describe the clinical features of feline cardiomyopathies and the therapy of related complications following this is a consideration of canine dcm and arrhythmogenic cardiomyopathy\n",
            "overview and pathophysiology of feline hcm  feline idiopathic hcm is characterized by hypertrophy and thickening of the left ventricle unexplained by congenital heart disease systemic hypertension or an endocrinopathy\n",
            " the condition is genetic in a number of feline breeds including the maine coon cat persian cat and the ragdoll thus far one sarcomeric mutation has been identified \n",
            "\n",
            "Abstract_Summary:    1527\n",
            "1528 Section 11 / Cardiopulmonary System<br>This chapter will next describe the clinical<br>features of feline cardiomyopathies and the therapy of<br>related complications. Following this is a<br>consideration of canine DCM and arrhythmogenic<br>cardiomyopathy.\n",
            "Overview and Pathophysiology of Feline HCM • Feline<br>idiopathic HCM is characterized by hypertrophy and<br>thickening of the left ventricle unexplained by<br>congenital heart disease, systemic hypertension, or an<br>endocrinopathy.\n",
            "• The condition is genetic in a number of feline<br>breeds, including the Maine coon cat, Persian cat, and<br>the Ragdoll. Thus far, one sarcomeric mutation has<br>been identified. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7966) \n",
            "\n",
            "Paragraph:    the term arrhythmogenic cardiomyopathy is a useful expression that refers to recurrent or persistent ventricular or atrial arrhythmias in the setting of a normal echocardiogram the most commonly observed rhythm disturbances are pvcs and ventricular tachycardia vt however atrial rhythm disturbances may be recognized including atrial fibrillation paroxysmal or sustained atrial tachycardia and atrial flutter \n",
            "\n",
            "paper_id:   1fdc1a513291d7613c95bbefae5b7ea0e811d5a6 \n",
            "\n",
            "Title:    Chapter 150 Cardiomyopathy \n",
            "\n",
            "Abstract:   1527\n",
            "1528 section 11  cardiopulmonary system this chapter will next describe the clinical features of feline cardiomyopathies and the therapy of related complications following this is a consideration of canine dcm and arrhythmogenic cardiomyopathy\n",
            "overview and pathophysiology of feline hcm  feline idiopathic hcm is characterized by hypertrophy and thickening of the left ventricle unexplained by congenital heart disease systemic hypertension or an endocrinopathy\n",
            " the condition is genetic in a number of feline breeds including the maine coon cat persian cat and the ragdoll thus far one sarcomeric mutation has been identified \n",
            "\n",
            "Abstract_Summary:    1527\n",
            "1528 Section 11 / Cardiopulmonary System<br>This chapter will next describe the clinical<br>features of feline cardiomyopathies and the therapy of<br>related complications. Following this is a<br>consideration of canine DCM and arrhythmogenic<br>cardiomyopathy.\n",
            "Overview and Pathophysiology of Feline HCM • Feline<br>idiopathic HCM is characterized by hypertrophy and<br>thickening of the left ventricle unexplained by<br>congenital heart disease, systemic hypertension, or an<br>endocrinopathy.\n",
            "• The condition is genetic in a number of feline<br>breeds, including the Maine coon cat, Persian cat, and<br>the Ragdoll. Thus far, one sarcomeric mutation has<br>been identified. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7873) \n",
            "\n",
            "Paragraph:    many cv diseases are tightly associated with inflammatory immune responses in their pathogenesis this include a wide range of vascular diseases endotoxin shock acute vasculitis atherosclerosis postoperative stenosis allograft vasculopathy cardiac diseases cardiac hypertrophy dilated cardiomyopathy myocardial infarction or ischemiareperfusion injury and myocarditis [133] [134] [135] [136] [137] [138] [139] [140] [141] [142]  and several autoimmune diseases in the cv system [8]  in the final section of this review we will briefly overview what is presently known about some of these cv diseases in terms of trp channel physiology and pathophysiology see also table 1 \n",
            "\n",
            "paper_id:   9221c3ed344b506c1208f8c2c4f9bf31f60b89da \n",
            "\n",
            "Title:    “TRP inflammation” relationship in<br>cardiovascular system \n",
            "\n",
            "Abstract:   despite considerable advances in the research and treatment the precise relationship between inflammation and cardiovascular cv disease remains incompletely understood therefore understanding the immunoinflammatory processes underlying the initiation progression and exacerbation of many cardiovascular diseases is of prime importance the innate immune system has an ancient origin and is well conserved across species its activation occurs in response to pathogens or tissue injury recent studies suggest that altered ionic balance and production of noxious gaseous mediators link to immune and inflammatory responses with altered ion channel expression and function among plausible candidates for this are transient receptor potential trp channels that function as polymodal sensors and scaffolding proteins involved in many physiological and pathological processes in this review we will first focus on the relevance of trp channel to both exogenous and endogenous factors related to innate immune response and transcription factors related to sustained inflammatory status the emerging role of inflammasome to regulate innate immunity and its possible connection to trp channels will also be discussed secondly we will discuss about the linkage of trp channels to inflammatory cv diseases from a viewpoint of inflammation in a general sense which is not restricted to the innate immunity these knowledge may serve to provide new insights into the pathogenesis of various inflammatory cv diseases and their novel therapeutic strategies \n",
            "\n",
            "Abstract_Summary:    Despite considerable advances in the research<br>and treatment, the precise relationship between<br>inflammation and cardiovascular (CV) disease remains<br>incompletely understood. Therefore, understanding the<br>immunoinflammatory processes underlying the initiation,<br>progression, and exacerbation of many cardiovascular<br>diseases is of prime importance. The innate immune<br>system has an ancient origin and is well conserved<br>across species. Its activation occurs in response to<br>pathogens or tissue injury. Recent studies suggest that<br>altered ionic balance, and production of noxious<br>gaseous mediators link to immune and inflammatory<br>responses with altered ion channel expression and<br>function. Among plausible candidates for this are<br>transient receptor potential (TRP) channels... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7791) \n",
            "\n",
            "Paragraph:    cardiac ppar ppar and ppar1 have pivotal roles in the pathophysiology of diabetic cardiomyopathy [44] \n",
            "\n",
            "paper_id:   f29df5233e688dddd0286374e0993e43459fe04b \n",
            "\n",
            "Title:    MuRF2 regulates PPARγ1 activity to protect<br>against diabetic cardiomyopathy and enhance weight<br>gain induced by a high fat diet \n",
            "\n",
            "Abstract:   background in diabetes mellitus the morbidity and mortality of cardiovascular disease is increased and represents an important independent mechanism by which heart disease is exacerbated the pathogenesis of diabetic cardiomyopathy involves the enhanced activation of ppar transcription factors including ppar and to a lesser degree ppar and ppar1 how these transcription factors are regulated in the heart is largely unknown recent studies have described posttranslational ubiquitination of ppars as ways in which ppar activity is inhibited in cancer however specific mechanisms in the heart have not previously been described recent studies have implicated the musclespecific ubiquitin ligase muscle ring finger2 murf2 in inhibiting the nuclear transcription factor srf initial studies of murf2 hearts revealed enhanced ppar activity leading to the hypothesis that murf2 regulates ppar activity by posttranslational ubiquitination\n",
            "methods murf2 mice were challenged with a 26week 60 fat diet designed to simulate obesitymediated insulin resistance and diabetic cardiomyopathy mice were followed by conscious echocardiography blood glucose tissue triglyceride glycogen levels immunoblot analysis of intracellular signaling heart and skeletal muscle morphometrics and ppar ppar and ppar1regulated mrna expression \n",
            "\n",
            "Abstract_Summary:    Background: In diabetes mellitus the<br>morbidity and mortality of cardiovascular disease is<br>increased and represents an important independent<br>mechanism by which heart disease is exacerbated. The<br>pathogenesis of diabetic cardiomyopathy involves the<br>enhanced activation of PPAR transcription factors,<br>including PPARα, and to a lesser degree PPARβ and PPARγ1.<br>How these transcription factors are regulated in<br>the heart is largely unknown. Recent studies have<br>described post-translational ubiquitination of PPARs<br>as ways in which PPAR activity is inhibited in<br>cancer. However, specific mechanisms in the heart have<br>not previously been described. Recent studies<br>have implicated the musclespecific ubiquitin<br>ligase muscle ring finger-2... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7782) \n",
            "\n",
            "Paragraph:    currently symmetrical peripheral gangrene in sepsis is suspected to occur due to vasopressor agents used in septic shock however a better explanation rests on combined hemostasis of microthrombogenesis and macrothrombogenesis although the latter is secondary event occurring as a complication of eavmtddit the pathophysiological mechanism of combined micromacrothrombotic syndrome can be easily supported by twopath unifying theory typically first setup is the hospital admission of a patient with sepsis that provokes eavmtddit the patient would need multiple venous and arterial accesses andor perhaps surgery during hospitalization [91]  this vascular intervention causes intravascular injury leading to damage of setevt and release of tf activated tf path leads to fibrinogenesis abbreviations apl acute promyelocytic leukemia dic disseminated intravascular coagulation dic illfounded dic dit disseminated intravascular microthrombosis fhf fulminant hepatic failure maha microangiopathic hemolytic anemia mods multiorgan dysfunction syndrome pt prothrombin time aptt activated partial thromboplastin time ttp thrombotic thrombocytopenic purpura ulvwf unusually large von willebrand factor multimers eavmtd endotheliopathyassociated vascular microthrombotic disease a please note that dic without hepatic coagulopathy is exactly the same to eavmtddit without hepatic coagulopathy which is also consistent with the thesis that dic is dit and forms fibrin clots in circulation since ongoing ulvwf path is already active from endotheliopathy and microthrombi strings are being formed fibrin clots and some of microthrombi would be unified and produce intravascular macrothrombi via macrothrombogenesis [20]  macrothrombi could travel to the peripheral arterial vasculatures in small and large arteries of the digits and extremities this combined micromacrothrombotic syndrome could lead to multiple welldemarcated peripheral symmetrical gangrene at distal vascular trees \n",
            "\n",
            "paper_id:   879b563cbe739c7ec914ce9eef39d9a85d6d3801 \n",
            "\n",
            "Title:    Sepsis and septic shock: endothelial<br>molecular pathogenesis associated with vascular<br>microthrombotic disease \n",
            "\n",
            "Abstract:   in addition to protective  \n",
            "\n",
            "Abstract_Summary:    In addition to protective \" \n",
            "\n",
            "-------------------------------------------\n",
            "\n",
            "\n",
            "=========================================================\n",
            "==========================Query==============================\n",
            "=== Use of AI in real-time health care delivery to evaluate interventions, risk factors, and outcomes in a way that could not be done manually =====\n",
            "=========================================================\n",
            "Score:    (Score: 0.8002) \n",
            "\n",
            "Paragraph:    conclusion several methods and approaches could be used in the healthcare arena time series is an analytical tool to study diseases and resources management at healthcare institutions the flexibility to follow up and recognize data patterns and provide explanations must not be neglected in studies of healthcare interventions in this study the arima model was introduced without the use of mathematical details or other extensions to the model the investigator or the healthcare organization involved in disease management programs could have great advantages when using analytical methodology in several areas with the ability to perform provisions in many cases despite the analytical possibility by statistical means this approach does not replace investigators common sense and experience in disease interventions \n",
            "\n",
            "paper_id:   1b8a085682d10dad4baf90417a8cbcde144cc8c9 \n",
            "\n",
            "Title:    Disease management with ARIMA model in time<br>series \n",
            "\n",
            "Abstract:   the evaluation of infectious and noninfectious disease management can be done through the use of a time series analysis in this study we expect to measure the results and prevent intervention effects on the disease clinical studies have benefited from the use of these techniques particularly for the wide applicability of the arima model this study briefly presents the process of using the arima model this analytical tool offers a great contribution for researchers and healthcare managers in the evaluation of healthcare interventions in specific populations \n",
            "\n",
            "Abstract_Summary:    The evaluation of infectious and<br>noninfectious disease management can be done through the use<br>of a time series analysis. In this study, we expect<br>to measure the results and prevent intervention<br>effects on the disease. Clinical studies have<br>benefited from the use of these techniques, particularly<br>for the wide applicability of the ARIMA model. This<br>study briefly presents the process of using the ARIMA<br>model. This analytical tool offers a great<br>contribution for researchers and healthcare managers in the<br>evaluation of healthcare interventions in specific<br>populations. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7745) \n",
            "\n",
            "Paragraph:    whether the health sector is in fact more skillintensive than all other sectors is an empirical question as is that of whether the incidence of illness and the provision and effectiveness of health care are independent of labour type in a multisectoral model with more than two factors possibly health carespecific and other reallife complexities the foregoing predictions are unlikely to be wholly true nevertheless these effects will still operate in the background and thus give a useful guide to the interpretation of the outcomes of such a model \n",
            "\n",
            "paper_id:   c33e8d913713308e554815bda48d2bfc618f6943 \n",
            "\n",
            "Title:    A comparative analysis of some policy options<br>to reduce rationing in the UK's NHS: Lessons from a<br>general equilibrium model incorporating positive<br>health effects \n",
            "\n",
            "Abstract:   this paper seeks to determine the macroeconomic impacts of changes in health care provision the resource allocation issues have been explored in theory by applying the rybczynski theorem and empirically using a computable general equilibrium cge model for the uk with a detailed health component from the theory changes in nonhealth outputs are shown to depend on factorbias and scale effects the net effects generally being indeterminate from the applied model a rise in the national health service nhs budget is shown to yield overall welfare gains which fall by twothirds assuming health carespecific factors a nominally equivalent migration policy yields even higher welfare gains \n",
            "\n",
            "Abstract_Summary:    This paper seeks to determine the<br>macro-economic impacts of changes in health care provision.<br>The resource allocation issues have been explored<br>in theory, by applying the Rybczynski theorem,<br>and empirically, using a computable general<br>equilibrium (CGE) model for the UK with a detailed health<br>component. From the theory, changes in non-health outputs<br>are shown to depend on factor-bias and scale<br>effects, the net effects generally being<br>indeterminate. From the applied model, a rise in the National<br>Health Service (NHS) budget is shown to yield overall<br>welfare gains, which fall by two-thirds assuming<br>health care-specific factors. A nominally<br>equivalent migration policy... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7576) \n",
            "\n",
            "Paragraph:    the central question is the following how can triage systems in emergency care be ethically assessed so as to realize optimal use of scarce resources in an ethically just way without remaining on the abstract level that is by taking the effect of triage on the individual patients and caregivers into account \n",
            "\n",
            "paper_id:   34190b8df3ed825e3c49e41a3782d04acf1e21e9 \n",
            "\n",
            "Title:    Emergency department triage: an ethical<br>analysis \n",
            "\n",
            "Abstract:   background emergency departments across the globe follow a triage system in order to cope with overcrowding the intention behind triage is to improve the emergency care and to prioritize cases in terms of clinical urgency\n",
            "discussion in emergency department triage medical care might lead to adverse consequences like delay in providing care compromise in privacy and confidentiality poor physicianpatient communication failing to provide the necessary care altogether or even having to decide whose life to save when not everyone can be saved these consequences challenge the ethical quality of emergency care this article provides an ethical analysis of routine emergency department triage the four principles of biomedical ethics viz respect for autonomy beneficence nonmaleficence and justice provide the starting point and help us to identify the ethical challenges of emergency department triage however they do not offer a comprehensive ethical view to address the ethical issues of emergency department triage from a more comprehensive ethical view the care ethics perspective offers additional insights summary we integrate the results from the analysis using four principles of biomedical ethics into care ethics perspective on triage and propose an integrated clinically and ethically based framework of emergency department triage planning as seen from a comprehensive ethics perspective that incorporates both the principlesbased and careoriented approach \n",
            "\n",
            "Abstract_Summary:    Background: Emergency departments across the<br>globe follow a triage system in order to cope with<br>overcrowding. The intention behind triage is to improve the<br>emergency care and to prioritize cases in terms of<br>clinical urgency.\n",
            "Discussion: In emergency<br>department triage, medical care might lead to adverse<br>consequences like delay in providing care, compromise in<br>privacy and confidentiality, poor physician-patient<br>communication, failing to provide the necessary care<br>altogether, or even having to decide whose life to save when<br>not everyone can be saved. These consequences<br>challenge the ethical quality of emergency care. This<br>article provides an ethical analysis of \"routine\"<br>emergency department... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7564) \n",
            "\n",
            "Paragraph:    limitations of the study our findings are exploratory rather than definitive and indicate directions for further planning and research like any new tool the framework and its application in a given context needs testing and refinement through simulation exercises targeting ambulatory care services as well as the broader health system \n",
            "\n",
            "paper_id:   646484c8081b3f1bf336bae89411c2aa86885788 \n",
            "\n",
            "Title:    General Practice and Pandemic Influenza: A<br>Framework for Planning and Comparison of Plans in Five<br>Countries \n",
            "\n",
            "Abstract:   background although primary health care and in particular general practice will be at the frontline in the response to pandemic influenza there are no frameworks to guide systematic planning for this task or to appraise available plans for their relevance to general practice we aimed to develop a framework that will facilitate planning for general practice and used it to appraise pandemic plans from australia england usa new zealand and canada\n",
            "methodologyprincipal findings we adapted the haddon matrix to develop the framework populating its cells through a multimethod study that incorporated the peerreviewed and grey literature interviews with general practitioners practice nurses and senior decisionmakers and desktop simulation exercises we used the framework to analyse 89 publiclyavailable jurisdictional plans at similar managerial levels in the five countries the framework identifies four functional domains clinical care for influenza and other needs public health responsibilities the internal environment and the macroenvironment of general practice no plan addressed all four domains most plans either ignored or were sketchy about noninfluenza clinical needs and about the contribution of general practice to public health beyond surveillance collaborations between general practices were addressed in few plans and interrelationships with the broader health system even less frequently\n",
            "conclusions this is the first study to provide a framework to guide general practice planning for pandemic influenza the framework helped identify critical shortcomings in available plans engaging general practice effectively in planning is challenging particularly where governance structures for primary health care are weak we identify implications for practice and for research \n",
            "\n",
            "Abstract_Summary:    Background: Although primary health care, and<br>in particular, general practice will be at the<br>frontline in the response to pandemic influenza, there<br>are no frameworks to guide systematic planning for<br>this task or to appraise available plans for their<br>relevance to general practice. We aimed to develop a<br>framework that will facilitate planning for general<br>practice, and used it to appraise pandemic plans from<br>Australia, England, USA, New Zealand and<br>Canada.\n",
            "Methodology/Principal Findings: We adapted the Haddon matrix to<br>develop the framework, populating its cells through a<br>multi-method study that incorporated the peer-reviewed and<br>grey literature, interviews with general<br>practitioners, practice... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7471) \n",
            "\n",
            "Paragraph:    cea measures have and can be used to help resource allocation decisions without formally using a specific cutoff the common use of league tables to help people understand where a medical intervention or technology fits in the general realm of common use and practice can inform some decisions about health technology use in these settings an implicit cea cutoff is used in lieu of an explicit measure this approach however tacitly assumes that existing interventions in common use have passed an implicit or explicit costeffectiveness test which is not necessarily true \n",
            "\n",
            "paper_id:   5719f6793a83d07462521f608b01fe66efeb6500 \n",
            "\n",
            "Title:    Approaches to Aggregation and Decision<br>Making—A Health Economics Approach: An ISPOR Special<br>Task Force Report [5] \n",
            "\n",
            "Abstract:   the fifth section of our special task force report identifies and discusses two aggregation issues 1 aggregation of cost and benefit information across individuals to a population level for benefit plan decision making and 2 combining multiple elements of value into a single value metric for individuals first we argue that additional elements could be included in measures of value but such elements have not generally been included in measures of qualityadjusted lifeyears for example we describe a recently developed extended costeffectiveness analysis ecea that provides a good example of how to use a broader concept of utility ecea adds two featuresmeasures of financial risk protection and income distributional consequences we then discuss a further option for expanding this approachaugmented cea which can introduce many value measures neither of these approaches however provide a comprehensive measure of value to resolve this issue we review a technique called multicriteria decision analysis that can provide a comprehensive measure of value we then discuss budgetsetting and prioritization using multicriteria decision analysis issues not yet fully resolved next we discuss deliberative processes which represent another important approach for populationor planlevel decisions used by many health technology assessment bodies these use quantitative information on cea and other elements but the group decisions are reached by a deliberative voting process finally we briefly discuss the use of stated preference methods for developing hedonic value frameworks and conclude with some recommendations in this area \n",
            "\n",
            "Abstract_Summary:    The fifth section of our Special Task Force<br>report identifies and discusses two aggregation<br>issues: 1) aggregation of cost and benefit information<br>across individuals to a population level for benefit<br>plan decision making and 2) combining multiple<br>elements of value into a single value metric for<br>individuals. First, we argue that additional elements could<br>be included in measures of value, but such<br>elements have not generally been included in measures of<br>quality-adjusted lifeyears. For example, we describe a recently<br>developed extended costeffectiveness analysis (ECEA)<br>that provides a good example of how to use a broader<br>concept of utility. ECEA adds... \n",
            "\n",
            "-------------------------------------------\n",
            "\n",
            "\n",
            "=========================================================\n",
            "==========================Query==============================\n",
            "=== Resources to support skilled nursing facilities and long term care facilities. =====\n",
            "=========================================================\n",
            "Score:    (Score: 0.7985) \n",
            "\n",
            "Paragraph:    health care workforce capacity building adequacy of health care workforce \n",
            "\n",
            "paper_id:   2e87670d524b1b76e2af6090d5cba472a6448f85 \n",
            "\n",
            "Title:    Capacity building in health care professions<br>within the Gulf cooperation council countries:<br>paving the way forward \n",
            "\n",
            "Abstract:   background there is a worldwide shortage of health care workers this problem is particularly severe in the gulf cooperation council gcc countries because of shortages in certain medical disciplines due to a lack of nationallytrained professionals and a less developed educational system compared to other high income countries consequently gcc countries are heavily dependent on an expatriate health care workforce a problem exacerbated by high turnover we discuss challenges and potential strategies for improving and strengthening capacity building efforts in health care professions in the gcc main text in the gcc there are 139 schools providing professional health education in medicine dentistry pharmacy nursing midwifery and other specialties health education school density reported for the gcc countries ranges between 22 and 28 schools per one million inhabitants except in oman where it is 40 per one million inhabitants the gcc countries rely heavily on expatriate health professionals the number of physicians and nurses in the gcc countries are 21 and 45 per 1000 respectively compared to 28 and 79 among member countries of the organisation for economic cooperation and development oecd interestingly the number of dentists and pharmacists is higher in the gcc countries compared to oecd countries a nationally trained health care workforce is essential for the gcc countries physiotherapy and occupational therapy are two identified areas where growth and development are recommended customtailored continuing medical education and continuing professional development cpd programs can augment the skills of health practitioners and allow for the expansion of their scope of practice when warranted conclusion capacity building can play an essential role in addressing the major health challenges and improving the overall quality of health care in the region efforts aimed at increasing the number of locallytrained graduates and developing and implementing needbased cpd programs are vital for capacity building and lifelong learning in health care professions \n",
            "\n",
            "Abstract_Summary:    Background: There is a worldwide shortage of<br>health care workers. This problem is particularly<br>severe in the Gulf Cooperation Council (GCC)<br>countries because of shortages in certain medical<br>disciplines, due to a lack of nationallytrained<br>professionals and a less developed educational system<br>compared to other high income countries. Consequently,<br>GCC countries are heavily dependent on an<br>expatriate health care workforce; a problem exacerbated<br>by high turnover. We discuss challenges and<br>potential strategies for improving and strengthening<br>capacity building efforts in health care professions in<br>the GCC. Main text: In the GCC, there are 139 schools<br>providing professional health education in... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7676) \n",
            "\n",
            "Paragraph:    with ili to an acute care facility \n",
            "\n",
            "paper_id:   67c79979c388cfcad87bb12d0ea644699500d736 \n",
            "\n",
            "Title:    Guidance on the use of antiviral drugs for<br>influenza in acute care facilities in Canada, 2014–2015 \n",
            "\n",
            "Abstract:   nan \n",
            "\n",
            "Abstract_Summary:   Not provided. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7558) \n",
            "\n",
            "Paragraph:    key points that contribute to caregiver success are summarized in table 3  regular prescheduled followup the ability for vai initiated medical support and respite care for the ventilatorassisted individual or caregiver are especially important \n",
            "\n",
            "paper_id:   e20c3efee1c9653c22821eb34138d6138daefe40 \n",
            "\n",
            "Title:    8(th) International conference on management<br>and rehabilitation of chronic respiratory<br>failure: the long summaries – Part 3 \n",
            "\n",
            "Abstract:   nan \n",
            "\n",
            "Abstract_Summary:   Not provided. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7511) \n",
            "\n",
            "Paragraph:    level of healthcare resources \n",
            "\n",
            "paper_id:   30fc9806c9b1ba0d786a31cad4a1d98b6642462d \n",
            "\n",
            "Title:    Pandemic influenza preparedness and health<br>systems challenges in Asia: results from rapid<br>analyses in 6 Asian countries \n",
            "\n",
            "Abstract:   background since 2003 asiapacific particularly southeast asia has received substantial attention because of the anticipation that it could be the epicentre of the next pandemic there has been active investment but earlier review of pandemic preparedness plans in the region reveals that the translation of these strategic plans into operational plans is still lacking in some countries particularly those with low resources the objective of this study is to understand the pandemic preparedness programmes the health systems context and challenges and constraints specific to the six asian countries namely cambodia indonesia lao pdr taiwan thailand and viet nam in the prepandemic phase before the start of h1n12009\n",
            "the study relied on the systemic rapid assessment sysra toolkit which evaluates priority disease programmes by taking into account the programmes the general health system and the wider sociocultural and political context the components under review were external context stewardship and organisational arrangements financing resource generation and allocation healthcare provision and information systems qualitative and quantitative data were collected in the second half of 2008 based on a review of published data and interviews with key informants exploring past and current patterns of health programme and pandemic response\n",
            "the study shows that health systems in the six countries varied in regard to the epidemiological context health care financing and health service provision patterns for pandemic preparation all six countries have developed national governance on pandemic preparedness as well as national pandemic influenza preparedness plans and avian and human influenza ahi response plans however the governance arrangements and the nature of the plans differed in the five developing countries the focus was on surveillance and rapid containment of poultry related transmission while preparation for later pandemic stages was limited the interfaces and linkages between health system contexts and pandemic preparedness programmes in these countries were explored\n",
            "health system context influences how the six countries have been preparing themselves for a pandemic at the same time investment in pandemic preparation in the six asian countries has contributed to improvement in health system surveillance laboratory capacity monitoring and evaluation and public communications a number of suggestions for improvement were presented to strengthen the pandemic preparation and mitigation as well as to overcome some of the underlying health system constraints \n",
            "\n",
            "Abstract_Summary:    Background: Since 2003, Asia-Pacific,<br>particularly Southeast Asia, has received substantial<br>attention because of the anticipation that it could be the<br>epicentre of the next pandemic. There has been active<br>investment but earlier review of pandemic preparedness<br>plans in the region reveals that the translation of<br>these strategic plans into operational plans is<br>still lacking in some countries particularly those<br>with low resources. The objective of this study is to<br>understand the pandemic preparedness programmes, the<br>health systems context, and challenges and<br>constraints specific to the six Asian countries namely<br>Cambodia, Indonesia, Lao PDR, Taiwan, Thailand, and Viet<br>Nam in the... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7450) \n",
            "\n",
            "Paragraph:    life support elements within the integrated management of pregnancy and childbirth eg enc \n",
            "\n",
            "paper_id:   737650f6ab6664da7582430d4b45ff9dd2e76866 \n",
            "\n",
            "Title:    In-service training for health professionals<br>to improve care of seriously ill newborns and<br>children in low-income countries \n",
            "\n",
            "Abstract:   inservice training for health professionals to improve care of seriously ill newborns and children in lowincome countries review \n",
            "\n",
            "Abstract_Summary:    In-service training for health professionals<br>to improve care of seriously ill newborns and<br>children in low-income countries (Review) \n",
            "\n",
            "-------------------------------------------\n",
            "\n",
            "\n",
            "=========================================================\n",
            "==========================Query==============================\n",
            "=== Mobilization of surge medical staff to address shortages in overwhelmed communities . =====\n",
            "=========================================================\n",
            "Score:    (Score: 0.8239) \n",
            "\n",
            "Paragraph:    immediate public health and medical support challenges included the identification triage and treatment of acutely sick and injured patients the management of chronic medical conditions in large number of evacuees with special health care needs the assessment communication and mitigation of public health risks mortuary support and the provision of assistance to state and local health officials to quickly reestablish health care deliver systems and public health infrastructures \n",
            "\n",
            "paper_id:   26e62203e2aecbd14310f48420b0f5b368453628 \n",
            "\n",
            "Title:    Directions for Disaster Nursing Education in<br>the United States \n",
            "\n",
            "Abstract:   nan \n",
            "\n",
            "Abstract_Summary:   Not provided. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7772) \n",
            "\n",
            "Paragraph:    many poor countries already have a health crisis and need massive international investments including mobilisation and strengthening of human resources to build sustainable health systems strong leadership and political commitment [73]  in the face of the pandemic threat primary health care in developing countries will need resources to develop a suite of policies including clarification of what essential primary health care will continue through a pandemic developing health workforce plans that may entail diverting clinicians from other areas of the health workforce establishing nonhierarchical links between primary health care hospitals and public health and injecting funds into hospital and primary care preparedness simultaneously \n",
            "\n",
            "paper_id:   646484c8081b3f1bf336bae89411c2aa86885788 \n",
            "\n",
            "Title:    General Practice and Pandemic Influenza: A<br>Framework for Planning and Comparison of Plans in Five<br>Countries \n",
            "\n",
            "Abstract:   background although primary health care and in particular general practice will be at the frontline in the response to pandemic influenza there are no frameworks to guide systematic planning for this task or to appraise available plans for their relevance to general practice we aimed to develop a framework that will facilitate planning for general practice and used it to appraise pandemic plans from australia england usa new zealand and canada\n",
            "methodologyprincipal findings we adapted the haddon matrix to develop the framework populating its cells through a multimethod study that incorporated the peerreviewed and grey literature interviews with general practitioners practice nurses and senior decisionmakers and desktop simulation exercises we used the framework to analyse 89 publiclyavailable jurisdictional plans at similar managerial levels in the five countries the framework identifies four functional domains clinical care for influenza and other needs public health responsibilities the internal environment and the macroenvironment of general practice no plan addressed all four domains most plans either ignored or were sketchy about noninfluenza clinical needs and about the contribution of general practice to public health beyond surveillance collaborations between general practices were addressed in few plans and interrelationships with the broader health system even less frequently\n",
            "conclusions this is the first study to provide a framework to guide general practice planning for pandemic influenza the framework helped identify critical shortcomings in available plans engaging general practice effectively in planning is challenging particularly where governance structures for primary health care are weak we identify implications for practice and for research \n",
            "\n",
            "Abstract_Summary:    Background: Although primary health care, and<br>in particular, general practice will be at the<br>frontline in the response to pandemic influenza, there<br>are no frameworks to guide systematic planning for<br>this task or to appraise available plans for their<br>relevance to general practice. We aimed to develop a<br>framework that will facilitate planning for general<br>practice, and used it to appraise pandemic plans from<br>Australia, England, USA, New Zealand and<br>Canada.\n",
            "Methodology/Principal Findings: We adapted the Haddon matrix to<br>develop the framework, populating its cells through a<br>multi-method study that incorporated the peer-reviewed and<br>grey literature, interviews with general<br>practitioners, practice... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7747) \n",
            "\n",
            "Paragraph:    finally in some countries particularly the smaller ones substantial shortfalls in preparedness were revealed concerning accommodating a surge of cases in healthcare facilities testing for multiple cases and contacts and mobilizing staff for contact tracing countries need to be prepared for a scenario that rapidly overwhelms the capacity of health authorities they should therefore consider detailed surge capacity planning that includes standby arrangements with other ministries eg defence or interior and civil society or international partners \n",
            "\n",
            "paper_id:   814ecf9043112382eaae1310c6359ddc70557bba \n",
            "\n",
            "Title:    Assessment of Ebola virus disease<br>preparedness in the WHO South-East Asia Region \n",
            "\n",
            "Abstract:   nan \n",
            "\n",
            "Abstract_Summary:   Not provided. \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7723) \n",
            "\n",
            "Paragraph:    in contemporary emergency care triage is regarded as an essential function not only during massive influx of patients as in disasters epidemics and pandemics but also in regular emergency care departments the burden in emergency care is increasing and so are the expectations of patients [1]  in hospitals that apply triage for regular emergency care triage is the first point of contact with the ed assessment by the triage officers involves a combination of the chief complaint of the patient general appearance and at times recording of vital signs [25] \n",
            "\n",
            "paper_id:   34190b8df3ed825e3c49e41a3782d04acf1e21e9 \n",
            "\n",
            "Title:    Emergency department triage: an ethical<br>analysis \n",
            "\n",
            "Abstract:   background emergency departments across the globe follow a triage system in order to cope with overcrowding the intention behind triage is to improve the emergency care and to prioritize cases in terms of clinical urgency\n",
            "discussion in emergency department triage medical care might lead to adverse consequences like delay in providing care compromise in privacy and confidentiality poor physicianpatient communication failing to provide the necessary care altogether or even having to decide whose life to save when not everyone can be saved these consequences challenge the ethical quality of emergency care this article provides an ethical analysis of routine emergency department triage the four principles of biomedical ethics viz respect for autonomy beneficence nonmaleficence and justice provide the starting point and help us to identify the ethical challenges of emergency department triage however they do not offer a comprehensive ethical view to address the ethical issues of emergency department triage from a more comprehensive ethical view the care ethics perspective offers additional insights summary we integrate the results from the analysis using four principles of biomedical ethics into care ethics perspective on triage and propose an integrated clinically and ethically based framework of emergency department triage planning as seen from a comprehensive ethics perspective that incorporates both the principlesbased and careoriented approach \n",
            "\n",
            "Abstract_Summary:    Background: Emergency departments across the<br>globe follow a triage system in order to cope with<br>overcrowding. The intention behind triage is to improve the<br>emergency care and to prioritize cases in terms of<br>clinical urgency.\n",
            "Discussion: In emergency<br>department triage, medical care might lead to adverse<br>consequences like delay in providing care, compromise in<br>privacy and confidentiality, poor physician-patient<br>communication, failing to provide the necessary care<br>altogether, or even having to decide whose life to save when<br>not everyone can be saved. These consequences<br>challenge the ethical quality of emergency care. This<br>article provides an ethical analysis of \"routine\"<br>emergency department... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.7684) \n",
            "\n",
            "Paragraph:    the fi fth priority is emergency preparedness and response protracted emergencies seem almost to have become a way of life in some parts of the region and more than half of the countries are currently facing either acute or chronic crises the major source of emergencies is civil unrest and violent confl ict the consequences are clear in the expanding humanitarian crisis in syria and its neighbours with rising numbers of people displaced health systems in all countries aff ected are facing major diffi culties in coping with the demands collective action and solidarity are needed to deliver health services to refugees and host communities and to increase the resilience of countries to emergencies and ensure eff ective public health responses during crises \n",
            "\n",
            "paper_id:   6c7de5a96730bd3962f86713db5aebdb4724f14a \n",
            "\n",
            "Title:    Responding to priority health challenges in<br>the Arab world \n",
            "\n",
            "Abstract:   nan \n",
            "\n",
            "Abstract_Summary:   Not provided. \n",
            "\n",
            "-------------------------------------------\n",
            "\n",
            "\n",
            "=========================================================\n",
            "==========================Query==============================\n",
            "=== Age-adjusted mortality data for Acute Respiratory Distress Syndrome (ARDS) with/without other organ failure – particularly for viral etiologies . =====\n",
            "=========================================================\n",
            "Score:    (Score: 0.8562) \n",
            "\n",
            "Paragraph:    patients were categorized according to their comorbidity status ie patients without comorbidity with chronic respiratory disease or with other comorbidities and compared using chisquared tests viral and bacterial infection status were compared by age group and comorbidity status using chisquared tests fishers exact tests and chisquared tests for trend inhospital mortality rates were calculated by viral and bacterial infection status and compared with those of the virusnegative group the effects of viral infection on inhospital mortality were expressed as risk ratios with 95 confidence intervals ci and estimated using poisson regression models with robust standard errors age study site comorbidity status duration of symptoms month of diagnosis antibiotic use and presence of bacteria were considered potential confounders based on prior knowledge and were included in the multiple regression models for patients whose onset of symptoms were unknown 5 we coded those missing values as unknown and included all patients in our analysis the data were analysed using stata version 13 stata corp college station tx usa \n",
            "\n",
            "paper_id:   6302e80258b85ac4f27c6db06bcf9e245e981ba9 \n",
            "\n",
            "Title:    The impact of virus infections on pneumonia<br>mortality is complex in adults: a prospective<br>multicentre observational study \n",
            "\n",
            "Abstract:   background various viruses are known to be associated with pneumonia however the impact of viral infections on adult pneumonia mortality remains unclear this study aimed to clarify the effect of virus infection on pneumonia mortality among adults stratified by virus type and patient comorbidities\n",
            "methods this multicentre prospective study enrolled pneumonia patients aged 15 years from september 2011 to august 2014 sputum samples were tested by inhouse multiplex polymerase chain reaction assays to identify 13 respiratory viruses viral infection status and its effect on inhospital mortality were examined by age group and comorbidity status results a total of 2617 patients were enrolled in the study and 778 was aged 65 years 574 219 did not have comorbidities 790 302 had chronic respiratory disease and 1253 479 had other comorbidities viruses were detected in 605 231 patients human rhinovirus 98 was the most frequently identified virus followed by influenza a 39 and respiratory syncytial virus 39 respiratory syncytial virus was more frequently identified in patients with chronic respiratory disease 47 than those with other comorbidities 42 and without comorbidities 21 p  0037 the frequencies of other viruses were almost identical between the three groups virus detection overall was not associated with increased mortality adjusted risk ratio arr 076 95 ci 053109 however influenza virus a and b were associated with threefold higher mortality in patients with chronic respiratory disease but not with other comorbidities arr 338 95 ci 154742 intriguingly paramyxoviruses were associated with dramatically lower mortality in patients with other comorbidities arr 010 95 ci 001070 but not with chronic respiratory disease these effects were not affected by age group conclusions the impact of virus infections on pneumonia mortality varies by virus type and comorbidity status in adults \n",
            "\n",
            "Abstract_Summary:    Background: Various viruses are known to be<br>associated with pneumonia. However, the impact of viral<br>infections on adult pneumonia mortality remains unclear.<br>This study aimed to clarify the effect of virus<br>infection on pneumonia mortality among adults<br>stratified by virus type and patient<br>comorbidities.\n",
            "Methods: This multicentre prospective study enrolled<br>pneumonia patients aged ≥15 years from September 2011 to<br>August 2014. Sputum samples were tested by in-house<br>multiplex polymerase chain reaction assays to identify<br>13 respiratory viruses. Viral infection status<br>and its effect on in-hospital mortality were<br>examined by age group and comorbidity status. Results: A<br>total of 2617 patients were... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.8340) \n",
            "\n",
            "Paragraph:    statistical analysis associations between categorical and continuous variables were analyzed using the mannwhitneyu test or kruskalwallis test for continuous variables and the fisher exact test for dichotomous variables spearmans rank correlation coefficient was used to assess a general monotonic trend between two continuous variables a simple linear regression model was used to measure linear associations between rsv viral load and age or day of illness or number of leucocytes in blood multivariable logistic regression analysis was performed to assess risk factors for severe disease or long hospitalization the following variables were considered for the models age sex premature birth previous hospitalization for respiratory illness other household members sick at home living with smokers number of members at home fever rsv infection rsv positivity viral load and subgroup and rv infection rv positivity as single infection or coinfection with rsv the models predictive ability was investigated by calculating the area under the receiver operating characteristic roc curve of the model auc ie the higher the auc the better prediction power the model has the hosmerlemeshow goodnessoffit test was done to assess model adequacy the hosmerlemeshow test of goodnessoffit tests the null hypothesis that there is no difference between the observed and predicted values of the response variables therefore when the test is not significant p005 the null hypothesis cannot be rejected and this means that the model fits the data well risk factors for disease severity or long hospitalization were further assessed using odds ratios or and 95 confidence intervals 95 cis all statistical tests were performed as twotailed tests at 5 significance in either r version 2131 r foundation for statistical computing vienna austria or intercooled stata version 92 college station tx usa \n",
            "\n",
            "paper_id:   ebf7e1110aa56a3066f9590befe475eb12b3eaa1 \n",
            "\n",
            "Title:    Respiratory Syncytial Virus and Other Viral<br>Infections among Children under Two Years Old in Southern<br>Vietnam 2009-2010: Clinical Characteristics and<br>Disease Severity \n",
            "\n",
            "Abstract:   despite a high burden of respiratory syncytial virus rsv infections among children data on demographic and clinical characteristics of rsv are scarce in low and middle income countries this study aims to describe the viral etiologies the demographic epidemiological and clinical characteristics of children under two years of age who were hospitalized with a lower respiratory tract infections lrti focusing on rsv prevalence seasonality subgroups viral load and its association with disease severity\n",
            "a prospective study among children under two years of age hospitalized with lrti was conducted in two referral pediatric hospitals in ho chi minh city vietnam from may 2009 to december 2010 sociodemographic clinical data and nasopharyngeal swabs were collected on enrolment and discharge multiplex realtime rtpcr 13 viruses and quantitative rsv rtpcr were used to identify viral pathogens rsv load and subgroups\n",
            "among 632 cases 48 were rsv positive rsv infections occurred at younger age than three other leading viral infections ie rhinovirus rv metapneumovirus mpv parainfluenza virus piv3 and were significantly more frequent in the first 6 months of life clinical severity score of rsv infection was significantly higher than piv3 but not for rv or mpv in multivariate analysis rv infection was significantly associated with severity while rsv infection was not among rsv infections neither viral load nor viral coinfections were significantly associated with severity young age and having fever at admission were significantly associated with both rsv and lrti severity a shift in rsv subgroup predominance was observed during two consecutive rainy seasons but was not associated with severity\n",
            "we report etiologies the epidemiological and clinical characteristics of lrti among hospitalized children under two years of age and risk factors of rsv and lrti severity\n",
            "sociodemographic data medical history clinical data from enrolment to discharge were recorded in standardized case report forms crfs nasopharyngeal swabs and edta blood respiratory syncytial virus and other viral respiratory infections in children under two years old plos one  \n",
            "\n",
            "Abstract_Summary:    Despite a high burden of respiratory syncytial<br>virus (RSV) infections among children, data on<br>demographic and clinical characteristics of RSV are scarce<br>in low and middle income countries. This study<br>aims to describe the viral etiologies, the<br>demographic, epidemiological, and clinical<br>characteristics of children under two years of age who were<br>hospitalized with a lower respiratory tract infections<br>(LRTI), focusing on RSV (prevalence, seasonality,<br>subgroups, viral load) and its association with disease<br>severity.\n",
            "A prospective study among children under two<br>years of age, hospitalized with LRTI was conducted in<br>two referral pediatric hospitals in Ho Chi Minh<br>City, Vietnam, from May... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.8151) \n",
            "\n",
            "Paragraph:    stochastic and deterministic sirvtype epidemic models in which individuals are susceptible s infected and infectious i 1 or i 2  recovered and immune r or partially protected against infection by antiviral prophylactic treatment v form the basis of the analyses figure 1 shows a schematic of the model details are given in text s1 throughout susceptible individuals enter the population by birth the natural death rate of susceptible and recovered individuals is denoted by m and the excess mortality while on antiviral treatment is given by c hence life expectancy is m 21 in the absence of infection and antiviral control and mc 21 while on antiviral drugs in the infected classes i 1 and i 2  the excess death rates resulting from infection are given by n and n12ave i  where ave i is the antiviral efficacy for infectiousness and virulence in the following c and n will be referred to succinctly as the cost of antiviral prophylaxis and infection from the susceptible class individuals move to the protected and infected classes at rates s and l the parameter l is colloquially called the force of infection and it depends on the prevalence of infection text s1 individuals in class v are infected at a reduced rate l12ave s  where ave s denotes the antiviral efficacy for susceptibility this implies that individuals in class v cannot be infected at all if ave s  1 while the antiviral drug provides no protection against infection if ave s  0 finally the rates of recovery and noncompliance are given by a and r respectively an overview of the model parameters and their default values is given in table 1  details of the model analyses are provided in text s1 \n",
            "\n",
            "paper_id:   26670350d15f4a1f0a7072ad985d98b21c28fd5a \n",
            "\n",
            "Title:    Self-Interest versus Group-Interest in<br>Antiviral Control \n",
            "\n",
            "Abstract:   antiviral agents have been hailed to hold considerable promise for the treatment and prevention of emerging viral diseases like h5n1 avian influenza and sars however antiviral drugs are not completely harmless and the conditions under which individuals are willing to participate in a largescale antiviral drug treatment program are as yet unknown we provide population dynamical and game theoretical analyses of largescale prophylactic antiviral treatment programs throughout we compare the antiviral control strategy that is optimal from the public health perspective with the control strategy that would evolve if individuals make their own rational decisions to this end we investigate the conditions under which a largescale antiviral control program can prevent an epidemic and we analyze at what point in an unfolding epidemic the risk of infection starts to outweigh the cost of antiviral treatment this enables investigation of how the optimal control strategy is moulded by the efficacy of antiviral drugs the risk of mortality by antiviral prophylaxis and the transmissibility of the pathogen our analyses show that there can be a strong incentive for an individual to take less antiviral drugs than is optimal from the public health perspective in particular when public health asks for early and aggressive control to prevent or curb an emerging pathogen for the individual antiviral drug treatment is attractive only when the risk of infection has become nonnegligible it is even possible that from a public health perspective a situation in which everybody takes antiviral drugs is optimal while the process of individual choice leads to a situation where nobody is willing to take antiviral drugs \n",
            "\n",
            "Abstract_Summary:    Antiviral agents have been hailed to hold<br>considerable promise for the treatment and prevention of<br>emerging viral diseases like H5N1 avian influenza and<br>SARS. However, antiviral drugs are not completely<br>harmless, and the conditions under which individuals are<br>willing to participate in a large-scale antiviral drug<br>treatment program are as yet unknown. We provide<br>population dynamical and game theoretical analyses of<br>large-scale prophylactic antiviral treatment programs.<br>Throughout we compare the antiviral control strategy that<br>is optimal from the public health perspective<br>with the control strategy that would evolve if<br>individuals make their own, rational decisions. To this end<br>we investigate... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.8138) \n",
            "\n",
            "Paragraph:    when substantial heterogeneity was found i 2 40 a priori defined subgroup analyses were performed which included age  18 years versus 18 years the nature of the underlying respiratory illness aris bronchiolitis radiologically confirmed community acquired pneumonia cap respectively the presence of underlying comorbidities the impact of overall viral infections versus specific viral pathogens and high versus low risk of bias in the included studies as defined above interaction tests for subgroup differences were conducted using the chisquare and i 2 statistic subgroup credibility was examined using the criteria described by sun and colleagues [13]  we made a posthoc stratified analysis of children by age subgroups based on data availability in the abstracted studies infants 023 months of age preschool children 059 months of age and children 017 years old \n",
            "\n",
            "paper_id:   a65d8f4193a3b03c5d20fac1972e0756ecf2d328 \n",
            "\n",
            "Title:    Clinical Disease Severity of Respiratory<br>Viral Co-Infection versus Single Viral Infection: A<br>Systematic Review and Meta-Analysis \n",
            "\n",
            "Abstract:   results from cohort studies evaluating the severity of respiratory viral coinfections are conflicting we conducted a systematic review and metaanalysis to assess the clinical severity of viral coinfections as compared to single viral respiratory infections\n",
            "we searched electronic databases and other sources for studies published up to january 28 2013 we included observational studies on inpatients with respiratory illnesses comparing the clinical severity of viral coinfections to single viral infections as detected by molecular assays the primary outcome reflecting clinical disease severity was length of hospital stay los a randomeffects model was used to conduct the metaanalyses\n",
            "results twentyone studies involving 4280 patients were included the overall quality of evidence applying the grade approach ranged from moderate for oxygen requirements to low for all other outcomes no significant differences in length of hospital stay los mean difference md 2020 days 95 ci 2094 053 p  059 or mortality rr 244 95 ci 086 691 p  009 were documented in subjects with viral coinfections compared to those with a single viral infection there was no evidence for differences in effects across age subgroups in post hoc analyses with the exception of the higher mortality in preschool children rr 982 95 ci 309 3120 p0001 with viral coinfection as compared to other age groups i 2 for subgroup analysis 64 p  004\n",
            "conclusions no differences in clinical disease severity between viral coinfections and single respiratory infections were documented the suggested increased risk of mortality observed amongst children with viral coinfections requires further investigation \n",
            "\n",
            "Abstract_Summary:    Results from cohort studies evaluating the<br>severity of respiratory viral co-infections are<br>conflicting. We conducted a systematic review and<br>meta-analysis to assess the clinical severity of viral<br>co-infections as compared to single viral respiratory<br>infections.\n",
            "We searched electronic databases and other<br>sources for studies published up to January 28, 2013. We<br>included observational studies on inpatients with<br>respiratory illnesses comparing the clinical severity of<br>viral co-infections to single viral infections as<br>detected by molecular assays. The primary outcome<br>reflecting clinical disease severity was length of<br>hospital stay (LOS). A random-effects model was used to<br>conduct the meta-analyses.\n",
            "Results: Twenty-one<br>studies involving 4,280... \n",
            "\n",
            "-------------------------------------------\n",
            "Score:    (Score: 0.8112) \n",
            "\n",
            "Paragraph:    multivariable logistic regression was used to estimate the odds ratio or and 95 confidence interval ci associated with hospitalization outcomes between those infected with rsv and those infected with influenza after adjustment for potential differences between cohorts including age sex raceethnicity history of influenza and pneumococcal vaccination prior chronic obstructive pulmonary disease copdchronic bronchitisemphysema charlson comorbidity index [17]  history of healthcare utilization and recent use of antivirals antibiotics or steroids oneyear survival after admission was estimated using the kaplanmeier method and survival rates between hospitalized persons infected with rsv or influenza were compared using the logrank test the analyses were performed using sas enterprise guide 51 sas institute inc cary nc \n",
            "\n",
            "paper_id:   4261cfd373572b30e170171afce095e6eb5d806a \n",
            "\n",
            "Title:    Severe Morbidity and Mortality Associated<br>With Respiratory Syncytial Virus Versus Influenza<br>Infection in Hospitalized Older Adults \n",
            "\n",
            "Abstract:   background respiratory syncytial virus rsv is an important cause of serious respiratory illness in older adults comparison of rsv and influenza infection in hospitalized older adults may increase awareness of adult rsv disease burden\n",
            "methods hospitalized adults aged 60 years who tested positive for rsv or influenza between 1 january 2011 and 30 june 2015 were identified from kaiser permanente southern california electronic medical records baseline characteristics comorbidities utilization and outcomes were compared\n",
            "results the study included 645 rsvand 1878 influenzainfected hospitalized adults patients with rsv were older than those with influenza mean 785 vs 774 years p  035 and more likely to have congestive heart failure 353 vs 245 p  001 and chronic obstructive pulmonary disease copd 298 vs 243 p  006 at baseline in adjusted analyses rsv infection was associated with greater odds of length of stay 7 days odds ratio [or]  15 95 confidence interval [ci] 1218 p  001 pneumonia or  27 95 ci 2232 p  001 intensive care unit admission or  13 95 ci 1017 p  023 exacerbation of copd or  17 95 ci 1324 p  001 and greater mortality within 1 year of admission or  13 95 ci 1016 p  019\n",
            "conclusions rsv infection may result in greater morbidity and mortality among older hospitalized adults than influenza increased recognition of adult rsv disease burden will be important in the evaluation and use of new rsv vaccines and antivirals \n",
            "\n",
            "Abstract_Summary:    Background. Respiratory syncytial virus<br>(RSV) is an important cause of serious respiratory<br>illness in older adults. Comparison of RSV and<br>influenza infection in hospitalized older adults may<br>increase awareness of adult RSV disease<br>burden.\n",
            "Methods. Hospitalized adults aged ≥60 years who tested<br>positive for RSV or influenza between 1 January 2011 and<br>30 June 2015 were identified from Kaiser<br>Permanente Southern California electronic medical<br>records. Baseline characteristics, comorbidities,<br>utilization, and outcomes were compared.\n",
            "Results. The<br>study included 645 RSV-and 1878 influenza-infected<br>hospitalized adults. Patients with RSV were older than those<br>with influenza (mean, 78.5 vs 77.4 years; P = .035)<br>and more... \n",
            "\n",
            "-------------------------------------------\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hzmpe7Af17VK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#import pickle as pkl\n",
        "#with open(\"/content/drive/My Drive/BertSentenceSimilarity/Pickles/corpus_embeddings.pkl\" , \"wb\") as file_:\n",
        "#  pkl.dump(corpus_embeddings,file_)"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}