"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "top_probs = [sum(topic_model.get_topics(topic_ids=[i], num_words=10)['score']) for i in range(10)]\n",
+ "\n",
+ "ind = np.arange(10)\n",
+ "width = 0.5\n",
+ "\n",
+ "fig, ax = plt.subplots()\n",
+ "\n",
+ "ax.bar(ind-(width/2),top_probs,width)\n",
+ "ax.set_xticks(ind)\n",
+ "\n",
+ "plt.xlabel('Topic')\n",
+ "plt.ylabel('Probability')\n",
+ "plt.title('Total Probability of Top 10 Words in each Topic')\n",
+ "plt.xlim(-0.5,9.5)\n",
+ "plt.ylim(0,0.15)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here we see that, for our topic model, the top 10 words only account for a small fraction (in this case, between 5% and 13%) of their topic's total probability mass. So while we can use the top words to identify broad themes for each topic, we should keep in mind that in reality these topics are more complex than a simple 10-word summary.\n",
+ "\n",
+ "Finally, we observe that some 'junk' words appear highly rated in some topics despite our efforts to remove unhelpful words before fitting the model; for example, the word 'born' appears as a top 10 word in three different topics, but it doesn't help us describe these topics at all."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Topic distributions for some example documents\n",
+ "\n",
+ "As we noted in the introduction to this assignment, LDA allows for mixed membership, which means that each document can partially belong to several different topics. For each document, topic membership is expressed as a vector of weights that sum to one; the magnitude of each weight indicates the degree to which the document represents that particular topic.\n",
+ "\n",
+ "We'll explore this in our fitted model by looking at the topic distributions for a few example Wikipedia articles from our data set. We should find that these articles have the highest weights on the topics whose themes are most relevant to the subject of the article - for example, we'd expect an article on a politician to place relatively high weight on topics related to government, while an article about an athlete should place higher weight on topics related to sports or competition."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Topic distributions for documents can be obtained using GraphLab Create's predict() function. GraphLab Create uses a collapsed Gibbs sampler similar to the one described in the video lectures, where only the word assignments variables are sampled. To get a document-specific topic proportion vector post-facto, predict() draws this vector from the conditional distribution given the sampled word assignments in the document. Notice that, since these are draws from a _distribution_ over topics that the model has learned, we will get slightly different predictions each time we call this function on a document - we can see this below, where we predict the topic distribution for the article on Barack Obama:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "+--------------------------+---------------------------+\n",
+ "| predictions (first draw) | predictions (second draw) |\n",
+ "+--------------------------+---------------------------+\n",
+ "| 0.0403225806452 | 0.0456989247312 |\n",
+ "| 0.0591397849462 | 0.0376344086022 |\n",
+ "| 0.0215053763441 | 0.0161290322581 |\n",
+ "| 0.10752688172 | 0.177419354839 |\n",
+ "| 0.604838709677 | 0.594086021505 |\n",
+ "| 0.0134408602151 | 0.0161290322581 |\n",
+ "| 0.0483870967742 | 0.0376344086022 |\n",
+ "| 0.0510752688172 | 0.0268817204301 |\n",
+ "| 0.0188172043011 | 0.0215053763441 |\n",
+ "| 0.0349462365591 | 0.0268817204301 |\n",
+ "+--------------------------+---------------------------+\n",
+ "+-------------------------------+\n",
+ "| topics |\n",
+ "+-------------------------------+\n",
+ "| science and research |\n",
+ "| team sports |\n",
+ "| music, TV, and film |\n",
+ "| American college and politics |\n",
+ "| general politics |\n",
+ "| art and publishing |\n",
+ "| Business |\n",
+ "| international athletics |\n",
+ "| Great Britain and Australia |\n",
+ "| international music |\n",
+ "+-------------------------------+\n",
+ "[10 rows x 3 columns]\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "obama = gl.SArray([wiki_docs[int(np.where(wiki['name']=='Barack Obama')[0])]])\n",
+ "pred1 = topic_model.predict(obama, output_type='probability')\n",
+ "pred2 = topic_model.predict(obama, output_type='probability')\n",
+ "print(gl.SFrame({'topics':themes, 'predictions (first draw)':pred1[0], 'predictions (second draw)':pred2[0]}))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To get a more robust estimate of the topics for each document, we can average a large number of predictions for the same document:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "def average_predictions(model, test_document, num_trials=100):\n",
+ " avg_preds = np.zeros((model.num_topics))\n",
+ " for i in range(num_trials):\n",
+ " avg_preds += model.predict(test_document, output_type='probability')[0]\n",
+ " avg_preds = avg_preds/num_trials\n",
+ " result = gl.SFrame({'topics':themes, 'average predictions':avg_preds})\n",
+ " result = result.sort('average predictions', ascending=False)\n",
+ " return result"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "+---------------------+-------------------------------+\n",
+ "| average predictions | topics |\n",
+ "+---------------------+-------------------------------+\n",
+ "| 0.591397849462 | general politics |\n",
+ "| 0.137231182796 | American college and politics |\n",
+ "| 0.0524193548387 | Business |\n",
+ "| 0.0503494623656 | team sports |\n",
+ "| 0.0413709677419 | science and research |\n",
+ "| 0.0343817204301 | international athletics |\n",
+ "| 0.0254301075269 | Great Britain and Australia |\n",
+ "| 0.0232258064516 | art and publishing |\n",
+ "| 0.0231720430108 | international music |\n",
+ "| 0.0210215053763 | music, TV, and film |\n",
+ "+---------------------+-------------------------------+\n",
+ "[10 rows x 2 columns]\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "print average_predictions(topic_model, obama, 100)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ What is the topic most closely associated with the article about former US President George W. Bush? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "+---------------------+-------------------------------+\n",
+ "| average predictions | topics |\n",
+ "+---------------------+-------------------------------+\n",
+ "| 0.42769005848 | general politics |\n",
+ "| 0.190029239766 | American college and politics |\n",
+ "| 0.102339181287 | Business |\n",
+ "| 0.0605847953216 | science and research |\n",
+ "| 0.0466666666667 | art and publishing |\n",
+ "| 0.045350877193 | team sports |\n",
+ "| 0.0388888888889 | Great Britain and Australia |\n",
+ "| 0.0366081871345 | international athletics |\n",
+ "| 0.0299707602339 | music, TV, and film |\n",
+ "| 0.0218713450292 | international music |\n",
+ "+---------------------+-------------------------------+\n",
+ "[10 rows x 2 columns]\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "george_bush = gl.SArray([wiki_docs[int(np.where(wiki['name']=='George W. Bush')[0])]])\n",
+ "print average_predictions(topic_model, george_bush, 100)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ What are the top 3 topics corresponding to the article about English football (soccer) player Steven Gerrard? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "+---------------------+-------------------------------+\n",
+ "| average predictions | topics |\n",
+ "+---------------------+-------------------------------+\n",
+ "| 0.49124 | team sports |\n",
+ "| 0.17276 | Great Britain and Australia |\n",
+ "| 0.12868 | international athletics |\n",
+ "| 0.0372 | international music |\n",
+ "| 0.0316 | general politics |\n",
+ "| 0.03112 | music, TV, and film |\n",
+ "| 0.02932 | Business |\n",
+ "| 0.02752 | art and publishing |\n",
+ "| 0.02644 | American college and politics |\n",
+ "| 0.02412 | science and research |\n",
+ "+---------------------+-------------------------------+\n",
+ "[10 rows x 2 columns]\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "steven_gerrard = gl.SArray([wiki_docs[int(np.where(wiki['name']=='Steven Gerrard')[0])]])\n",
+ "print average_predictions(topic_model, steven_gerrard, 100)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Comparing LDA to nearest neighbors for document retrieval\n",
+ "\n",
+ "So far we have found that our topic model has learned some coherent topics, we have explored these topics as probability distributions over a vocabulary, and we have seen how individual documents in our Wikipedia data set are assigned to these topics in a way that corresponds with our expectations. \n",
+ "\n",
+ "In this section, we will use the predicted topic distribution as a representation of each document, similar to how we have previously represented documents by word count or TF-IDF. This gives us a way of computing distances between documents, so that we can run a nearest neighbors search for a given document based on its membership in the topics that we learned from LDA. We can contrast the results with those obtained by running nearest neighbors under the usual TF-IDF representation, an approach that we explored in a previous assignment. \n",
+ "\n",
+ "We'll start by creating the LDA topic distribution representation for each document:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "wiki['lda'] = topic_model.predict(wiki_docs, output_type='probability')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next we add the TF-IDF document representations:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "wiki['word_count'] = gl.text_analytics.count_words(wiki['text'])\n",
+ "wiki['tf_idf'] = gl.text_analytics.tf_idf(wiki['word_count'])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For each of our two different document representations, we can use GraphLab Create to compute a brute-force nearest neighbors model:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "Starting brute force nearest neighbors model training.
"
+ ],
+ "text/plain": [
+ "Starting brute force nearest neighbors model training."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "Starting brute force nearest neighbors model training.
"
+ ],
+ "text/plain": [
+ "Starting brute force nearest neighbors model training."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "model_tf_idf = gl.nearest_neighbors.create(wiki, label='name', features=['tf_idf'],\n",
+ " method='brute_force', distance='cosine')\n",
+ "model_lda_rep = gl.nearest_neighbors.create(wiki, label='name', features=['lda'],\n",
+ " method='brute_force', distance='cosine')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's compare these nearest neighbor models by finding the nearest neighbors under each representation on an example document. For this example we'll use Paul Krugman, an American economist:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "Starting pairwise querying.
"
+ ],
+ "text/plain": [
+ "Starting pairwise querying."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |
"
+ ],
+ "text/plain": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 0 | 1 | 0.00169288 | 16.669ms |
"
+ ],
+ "text/plain": [
+ "| 0 | 1 | 0.00169288 | 16.669ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Done | | 100 | 324.125ms |
"
+ ],
+ "text/plain": [
+ "| Done | | 100 | 324.125ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " | query_label | \n",
+ " reference_label | \n",
+ " distance | \n",
+ " rank | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Paul Krugman | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Elise Brezis | \n",
+ " 0.744498017262 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Maitreesh Ghatak | \n",
+ " 0.81564984831 | \n",
+ " 3 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Kai A. Konrad | \n",
+ " 0.823700564406 | \n",
+ " 4 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " David Colander | \n",
+ " 0.834625927759 | \n",
+ " 5 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Richard Blundell | \n",
+ " 0.837934267874 | \n",
+ " 6 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Gordon Rausser | \n",
+ " 0.83941534706 | \n",
+ " 7 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Edward J. Nell | \n",
+ " 0.842178500015 | \n",
+ " 8 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Robin Boadway | \n",
+ " 0.842374260596 | \n",
+ " 9 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Tim Besley | \n",
+ " 0.843088109253 | \n",
+ " 10 | \n",
+ "
\n",
+ "
\n",
+ "[10 rows x 4 columns]
\n",
+ "
Starting pairwise querying.
"
+ ],
+ "text/plain": [
+ "Starting pairwise querying."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |
"
+ ],
+ "text/plain": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 0 | 1 | 0.00169288 | 2.839ms |
"
+ ],
+ "text/plain": [
+ "| 0 | 1 | 0.00169288 | 2.839ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Done | | 100 | 24.178ms |
"
+ ],
+ "text/plain": [
+ "| Done | | 100 | 24.178ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " | query_label | \n",
+ " reference_label | \n",
+ " distance | \n",
+ " rank | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Paul Krugman | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Allen Feldman | \n",
+ " 0.00264054322093 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Cleo Paskal | \n",
+ " 0.00503243838456 | \n",
+ " 3 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Lene Auestad | \n",
+ " 0.00725301227925 | \n",
+ " 4 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Ben Klemens | \n",
+ " 0.00878201993564 | \n",
+ " 5 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Steve Lawrence (computer
scientist) ... | \n",
+ " 0.0109690477594 | \n",
+ " 6 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Randall Hansen | \n",
+ " 0.0110336501005 | \n",
+ " 7 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Metta Spencer | \n",
+ " 0.0110450122822 | \n",
+ " 8 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Steven Skiena | \n",
+ " 0.0114691009306 | \n",
+ " 9 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Mohamed Osman Elkhosht | \n",
+ " 0.0115525590812 | \n",
+ " 10 | \n",
+ "
\n",
+ "
\n",
+ "[10 rows x 4 columns]
\n",
+ "
Starting pairwise querying.
"
+ ],
+ "text/plain": [
+ "Starting pairwise querying."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |
"
+ ],
+ "text/plain": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 0 | 1 | 0.00169288 | 20.877ms |
"
+ ],
+ "text/plain": [
+ "| 0 | 1 | 0.00169288 | 20.877ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Done | | 100 | 302.418ms |
"
+ ],
+ "text/plain": [
+ "| Done | | 100 | 302.418ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Get a list of 'reference_label' based on knn\n",
+ "alex_rodriguez_tfidf = list(model_tf_idf.query(wiki[wiki['name'] == 'Alex Rodriguez'], label='name', k=5000)['reference_label'])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 40,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "value of k: 52\n"
+ ]
+ }
+ ],
+ "source": [
+ "print 'value of k:', alex_rodriguez_tfidf.index('Mariano Rivera')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ Using the LDA representation, compute the 5000 nearest neighbors for American baseball player Alex Rodriguez. For what value of k is Mariano Rivera the k-th nearest neighbor to Alex Rodriguez? (Hint: Once you have a list of the nearest neighbors, you can use `mylist.index(value)` to find the index of the first instance of `value` in `mylist`.)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 41,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "Starting pairwise querying.
"
+ ],
+ "text/plain": [
+ "Starting pairwise querying."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |
"
+ ],
+ "text/plain": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 0 | 1 | 0.00169288 | 3.993ms |
"
+ ],
+ "text/plain": [
+ "| 0 | 1 | 0.00169288 | 3.993ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Done | | 100 | 23.926ms |
"
+ ],
+ "text/plain": [
+ "| Done | | 100 | 23.926ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "value of k: 729\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Get a list of 'reference_label' based on knn\n",
+ "alex_rodriguez_lda = list(model_lda_rep.query(wiki[wiki['name'] == 'Alex Rodriguez'], label='name', k=5000)['reference_label'])\n",
+ "print 'value of k:', alex_rodriguez_lda.index('Mariano Rivera')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Understanding the role of LDA model hyperparameters\n",
+ "\n",
+ "Finally, we'll take a look at the effect of the LDA model hyperparameters alpha and gamma on the characteristics of our fitted model. Recall that alpha is a parameter of the prior distribution over topic weights in each document, while gamma is a parameter of the prior distribution over word weights in each topic. \n",
+ "\n",
+ "In the video lectures, we saw that alpha and gamma can be thought of as smoothing parameters when we compute how much each document \"likes\" a topic (in the case of alpha) or how much each topic \"likes\" a word (in the case of gamma). In both cases, these parameters serve to reduce the differences across topics or words in terms of these calculated preferences; alpha makes the document preferences \"smoother\" over topics, and gamma makes the topic preferences \"smoother\" over words.\n",
+ "\n",
+ "Our goal in this section will be to understand how changing these parameter values affects the characteristics of the resulting topic model.\n",
+ "\n",
+ "__Quiz Question:__ What was the value of alpha used to fit our original topic model? "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 43,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "5.0"
+ ]
+ },
+ "execution_count": 43,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "topic_model['alpha']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ What was the value of gamma used to fit our original topic model? Remember that GraphLab Create uses \"beta\" instead of \"gamma\" to refer to the hyperparameter that influences topic distributions over words."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 44,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.1"
+ ]
+ },
+ "execution_count": 44,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "topic_model['beta']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We'll start by loading some topic models that have been trained using different settings of alpha and gamma. Specifically, we will start by comparing the following two models to our original topic model:\n",
+ " - tpm_low_alpha, a model trained with alpha = 1 and default gamma\n",
+ " - tpm_high_alpha, a model trained with alpha = 50 and default gamma"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 45,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "tpm_low_alpha = gl.load_model('topic_models/lda_low_alpha')\n",
+ "tpm_high_alpha = gl.load_model('topic_models/lda_high_alpha')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Changing the hyperparameter alpha\n",
+ "\n",
+ "Since alpha is responsible for smoothing document preferences over topics, the impact of changing its value should be visible when we plot the distribution of topic weights for the same document under models fit with different alpha values. In the code below, we plot the (sorted) topic weights for the Wikipedia article on Barack Obama under models fit with high, original, and low settings of alpha."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 46,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNXV//HPYVgU2UFRGBhQwR1/IiK4seUhoEE0Piig\nxqBRNEET8+SJQUIgRqMxxhiiPK7BhaBx30CDCY47YgRxCZsQBhhwQUABlfX8/qjbQ88w01Mj1HQz\n832/Xv2arqrbVaebYU7XrVv3mLsjIiKSi+pkOwAREZGKKEmJiEjOUpISEZGcpSQlIiI5S0lKRERy\nlpKUiIjkrESTlJndY2Yfm9m7GdpMMLNFZvaOmf2/JOMREZE9S9JnUpOAb1e00cwGAge5eydgJHB7\nwvGIiMgeJNEk5e6vAmszNBkM3B/avgk0NbPWScYkIiJ7jmxfk2oLLE9bLg7rREREsp6kREREKlQ3\ny8cvBtqlLeeHdTsxM00yKCJSg7m7lV1XHWdSFh7leRr4HoCZ9QDWufvHFe3I3WM9xo0bF7ttdT4U\n154dk+La82NSXLkbU0USPZMysylAb6ClmS0DxgH1o3zjd7r7NDM71cw+BDYCI5KMR0RE9iyJJil3\nHx6jzagkYxARkT1XjRw40bt372yHUC7FFV8uxgSKqypyMSZQXFWRCzFZpr7AXGJmvqfEKiIiVWNm\neDkDJ7I9uk9EaokOHTpQVFSU7TAkywoKCli6dGns9jqTEpFqEb4pZzsMybKKfg8qOpOqkdekRESk\nZlCSEhGRnKUkJSIiOUtJSkRqvY4dOzJjxoysHb9OnTosWbJkt7etCZSkRCRr8gs6YGaJPfILOmT7\nLcZiVtHMcbvWtibQEHQRyZriZUVcP3tzYvsf3bV+Yvvenaoy6rG2jZDUmZSISJrNmzfzk5/8hLZt\n25Kfn8+VV17Jli1bgGgGhieeeAKA1157jTp16vDcc88BMGPGDI455phy9/nWW29xwgkn0Lx5c9q2\nbcvll1/O1q1by207YsQILrvsMvr370+TJk3o06cPy5YtK9XmhRdeoHPnzrRo0YJRo3bMLLdkyRL6\n9etHq1at2G+//TjvvPP44osvdvkzySYlKRGRNNdeey2zZs3i3XffZe7cucyaNYtrr70WgF69elFY\nWAjAyy+/zEEHHcTLL78MwEsvvVThNEJ5eXnccsstrFmzhjfeeIMZM2YwceLECmOYMmUK48aN47PP\nPuPoo4/m3HPPLbV96tSpvP3228ydO5eHH36Y6dOnA9FZ1tVXX81HH33EvHnzWLFiBePHj9+1DyTL\nlKRERNKkEkTLli1p2bIl48aN44EHHgCiJPXSSy8BUZIaPXp0yfJLL71Er169yt1n165d6d69O2ZG\n+/btueSSS0peV57TTjuNE088kXr16nHdddfxxhtvUFy8o9Te6NGjady4Me3ataNPnz688847ABx0\n0EH069ePunXr0rJlS6688sqMx9kTKEmJiKRZuXIl7du3L1kuKChg5cqVAPTs2ZOFCxfyySefMHfu\nXL73ve+xfPlyPvvsM2bNmsUpp5xS7j4XLVrEoEGDOOCAA2jWrBljxoxh9erVFcbQrt2OWrD77LMP\nLVq0KIkBoHXr1iXPGzZsyIYNGwD45JNPGDZsGPn5+TRr1ozzzjsv43H2BEpSIiJp2rRpU2qOwaKi\nItq0aQPA3nvvzbHHHsuf/vQnjjzySOrWrUvPnj25+eabOfjgg2nRokW5+7zssss47LDDWLx4MevW\nreO6667LOABi+fLlJc83bNjAmjVraNu2baWxX3311dSpU4cPPviAdevWMXny5D1+oIWSlIhImmHD\nhnHttdeyevVqVq9ezW9+8xvOP//8ku2nnHIKt956a0nXXu/evUstl2f9+vU0adKEhg0bMn/+fP7v\n//4vYwzTpk3j9ddfZ/PmzYwdO5aePXuWJMpM1q9fT6NGjWjcuDHFxcX8/ve/j/muc5eSlIjUeun3\nHv3yl7+kW7dudOnShaOPPppu3boxZsyYku29evViw4YNJV17qeVMSeqmm27ir3/9K02aNGHkyJEM\nHTq0wuMDDB8+nPHjx9OyZUvmzJnD5MmTK2ybbty4cbz99ts0a9aMQYMGcdZZZ8X7AHKYZkEXkWpR\n3uzX+QUdKF6WXPmOtu0LWFG0NLH9J2HEiBG0a9eOa665JtuhJKKqs6DrZl4RyZo9LYFI9VN3n4hI\nDqlt0x5VRt19IlItVPRQQEUPRUSkBlGSEhGRnKUkJSIiOUtJSkREcpaSlIiI5CwlKRGRKrrsssu4\n7rrrdnvbTIqKiqhTpw7bt2/f5X1V1YgRI/jVr34Vq23Hjh2ZMWPGbju2buYVkazpkJ9PUVoJit2t\noG1blq5Ysdv3W9nce9+0bWVq4z1UsZKUmQ0FDnL368ysHbCfu7+dbGgiUtMVFRezLsGifM0S2Pf2\n7dupU0edUNWl0k/azG4F+gDnhVUbgduTDEpEpDrNnz+fPn360Lx5c4466iieeeaZkm0jRozghz/8\nIaeddhqNGzemsLBwp+6vG2+8kTZt2pCfn88999xDnTp1WLJkScnrU21feukl2rVrx80330zr1q1p\n27Yt9957b8l+pk2bRteuXWnatCkFBQX8+te/jv0eOnbsyE033cTRRx9N48aNufjii/nkk0849dRT\nadKkCf379+fzzz8vaf/0009z5JFH0qJFC/r27cv8+fNLts2ZM4djjz2Wpk2bMnToUL7++utSx3r2\n2Wc55phjaN68OSeddBLvvfde7DirKs7XgRPcfSTwNYC7rwHqJxaRiEg12rp1K4MGDWLAgAF8+umn\nTJgwgXPPPZdFixaVtHnwwQcZO3Ys69ev58QTTyz1+ueff55bbrmFGTNm8OGHH1JYWJixW+6jjz5i\n/fr1rFy5krvvvpsf/ehHJcmjUaNGPPDAA3z++edMnTqV22+/naeffjr2e3n88cf55z//ycKFC3n6\n6ac59dRTueGGG1i9ejXbtm1jwoQJACxcuJDhw4czYcIEPv30UwYOHMigQYPYunUrW7Zs4cwzz+SC\nCy5gzZo1DBkyhMcee6zkGHPmzOGiiy7irrvuYs2aNYwcOZLTTz+dLVu2xI6zKuIkqS1mVgdwADNr\nCVT/lTsRkQTMnDmTjRs3ctVVV1G3bl369OnDd77zHR588MGSNoMHD6ZHjx4ANGjQoNTrH3nkEUaM\nGMGhhx7KXnvtxfhKuhjr16/P2LFjycvLY+DAgTRq1IgFCxYAUa2qI444AoAjjzySoUOHVqn8++WX\nX06rVq044IADOPnkkzn++OPp0qUL9evX58wzz2TOnDkAPPzww3znO9+hb9++5OXl8bOf/Yyvv/6a\n119/nZkzZ7J161auuOIK8vLyOOusszjuuONKjnHXXXdx6aWX0q1bN8yM888/nwYNGjBz5szYcVZF\nnCR1G/AYsK+Z/Rp4FfhdItGIiFSzlStXlirXDlHJ+OK0AR1lt2d6fbt27TLOUdiyZctS17TSy7+/\n+eab9O3bl/32249mzZpxxx13VKn8e3pZ+b333nun5dRxVq5cSUFBQck2MyM/P5/i4mJWrly5UxXg\n9LZFRUX84Q9/oEWLFrRo0YLmzZuzYsWKUuXtd6dKk5S73w/8ErgJWAsMcfeHEolGRKSatWnTplS5\ndoBly5aV+kOdqfvugAMOYEXaCMJly5Z941F45557LmeccQbFxcWsW7eOkSNHJjIpb5s2bSgqKl3H\na/ny5bRt23an9wPRe0pp164dY8aMYc2aNaxZs4a1a9eyYcMGzjnnnN0eJ2RIUmbWJPUAlgOTgL8A\ny8I6EZE93vHHH0/Dhg258cYb2bp1K4WFhTz77LMMGzYs1uvPPvtsJk2axPz58/nyyy+59tprv3Es\nGzZsoHnz5tSrV49Zs2YxZcqUUtt3V8I6++yzmTp1Ki+++CJbt27lpptuYq+99uKEE06gZ8+e1KtX\njz//+c9s3bqVxx9/nFmzZpW89uKLL+b2228vWbdx40amTZvGxo0bd0tsZWU6k/oAeD/8TD1/P+25\niMger169ejzzzDNMmzaNVq1aMWrUKB544AE6deoElH8Wlb5uwIABXHHFFfTp04fOnTvTs2dPYOdr\nVxVJ39fEiRMZO3YsTZs25dprr93p7CTTGVrZbZnadu7cmcmTJzNq1Cj23Xdfpk6dyjPPPEPdunWp\nV68ejz/+OJMmTaJly5Y88sgjpcrQH3vssdx1112MGjWKFi1a0LlzZ+67775Yx/0mVE9KRKpFeXWE\n9tSbeTOZP38+Rx11FJs2bdL9VOXY7fWkzOx0M2uattzMzL6zy5GKSK23dMUK3D2xR3UlqCeffJLN\nmzezdu1arrrqKk4//XQlqN0kzqd4jbuX3AHm7uuA38Q9gJkNMLP5ZrbQzK4qZ3sTM3vazN4xs/fM\n7Ptx9y0ikgvuuOMO9ttvPzp16kS9evWYOHFitkOqMSrt7jOzue5+dJl177n7UZXuPLq/aiHQD1gJ\nvAUMdff5aW1GA03cfbSZtQIWAK3dfWuZfam7T2QPpvLxAsmUj59jZjeaWUF4/B6YEzOe7sAidy9y\n9y3AQ8DgMm0caByeNwY+K5ugRESkdoqTpEaFdk+FB8APY+6/LdHw9ZQVYV26W4HDzWwlMBf4ccx9\ni4hIDVfpLOjuvgH4WYIxfBuY4+59zewg4AUz6xKOKyIitViFScrM/uDu/2NmTxDm7Uvn7t+Nsf9i\noH3acn5Yl24EcH3Y52Iz+w9wKPCvsjtLnxOrd+/e9O7dO0YIIiKSawoLCyksLKy0XYUDJ8ysu7vP\nMrN+5W13939WunOzPKKBEP2AVcAsYJi7z0trcxvwibv/2sxaEyWno8Ns6+n70sAJkT2YBk4I7MaB\nE+6emgfjMHf/Z/oDOCxOMO6+jeia1nSimSoecvd5ZjbSzC4Jza4FTjCzd4EXgJ+XTVAiIknKVPL8\n1Vdf5bDDYv3JK6kXtTtUZV+787i5Jk5l3guJBjeku6icdeVy9+eBQ8qsuyPt+Sqi61IiUst0KNif\nomUfJ7b/gvatWVr00S7t46STTmLevHmVNwx257RAVdlXTS0tn+ma1DnAUOBAM3s8bVNjYF3SgYlI\nzVe07GM8/t//KrPDkkuAUj0yDUGfRVRLalH4mXqMAfonH5qISPWZM2cORx99NM2bN2fYsGFs3rwZ\n2Lkrbfbs2SUl3s8++2yGDh1aqpS8u1dYHr6se++9l8MPP5wmTZpw8MEHc+edd1bYtmPHjtxwww0c\nccQRtGzZkosuuqgkxsqOuytl6bMt0zWp/wAvAuvKXJOaFW7MFRGpMR555BGmT5/Of/7zH+bOnVvq\nj3yqK23Lli1897vf5cILL2TNmjUMGzaMJ554otR+MpWHL6t169ZMmzaNL774gkmTJnHllVfyzjvv\nVBjjlClTeOGFF1i8eDELFiwoVRYkybL02ZTxZt4w8CFP9aNEpKb78Y9/TOvWrWnWrBmDBg0qN1m8\n8cYbbNu2jVGjRpGXl8eZZ55J9+7dS7XJVB6+rIEDB9KhQwcATj75ZPr3788rr7xSYYyXX345bdq0\noVmzZowZM6ZUifsky9JnU5yBE58Dc81sOlBS1crdf5pYVCIi1Sy91HrDhg1ZtWrVTm1WrVq1U2n1\nsqPqMpWHL+u5557jmmuuYeHChWzfvp2vvvqKLl26VBhjfn5+yfOCgoJSJdsrK0s/evRo3n//fTZv\n3szmzZsZMmRIhcfJJXGmRXqWaJj4LEoXQBQRqVUOOOAAisvUvypbej6uzZs389///d/8/Oc/59NP\nP2Xt2rUMHDgw471k6ccqKiqiTZs2sY5VXWXpk1BpknL3e9IfwHNA08peJyJS0/Ts2ZO8vDxuu+02\ntm3bxlNPPVWqtHpVpM5oWrVqRZ06dXjuueeYPn16xtfcdtttFBcXs2bNGn77298ydOjQWMeqrCx9\nLotVlcvMWpjZJWb2IvA6UJBsWCIi1SfuPUap0up33303zZs3Z8qUKQwaNChjqfiK9t2oUSMmTJjA\nkCFDaNGiBQ899BCDB5ctElHa8OHD6d+/PwcffDCdOnVizJgxsY5bWVn6XJZpWqR9gDOA4cARRDOg\n/7e7l53FvFpoWiSRPVu55eP3gJt5K9OjRw8uu+wyLrjggkSP07FjR+655x769u2b6HGSVtVpkTIN\nnPiEaB698cBL7r7dzE7fXYF+E3G/7bRtX8CKoqXJBiMiuyzpBJKEl19+mUMOOYRWrVoxefJk3nvv\nPQYMGJDtsGqsTElqHNGMEzcDD5rZ3yhnNvTqdP3szZU3AkZ3rZ9wJCJSWy1YsICzzz6bL7/8kgMP\nPJDHHnus1MjApNTUaY8qE6d8fCdgGGGKJKIZJ55w9yXJh1cqDq9KklLXoEhu0SzoAgmUj3f3Re5+\njbsfDvQA9gMqLdMhIiKyq2KN7ktx93fc/Sp375hUQCIiIilVSlIiIiLVSUlKRERyVpy5+0REdllB\nQUGtHaEmOxQUVG0uiEqTlJkdBFwHHA7slVrv7p2rGpyI1F5Lly7NdgiyB4rT3XcvMAkwYCDwMPC3\nBGMSEREB4iWphu7+dwB3X+zuvyRKViIiIomKc01qk5nVARab2aVAMdA42bBERETiJakrgX2AK4iu\nTTUFLkwyKBEREYiRpNz9zfB0PXB+suGIiIjsEGd0X1dgNFENqZL27t41wbhERERidfc9SJSk3gO2\nJxuOiIjIDnGS1Gp3fzzxSERERMqIk6R+bWa3E818vim10t2fTiwqERER4iWpc4EuRMPOU919DihJ\niYhIouIkqR7ufkjikYiIiJQRZ8aJN81MSUpERKpdnDOpY4B3zexDomtSBriGoIuISNLiJKkzEo9C\nRESkHHFmnFgMYGYtSCvVISIikrRKr0mZ2WlmthBYAbwJLAdmJB2YiIhInIET1wEnAgvcvR0wAHgl\n0ahERESIl6S2uvunQB0zM3d/AeiecFwiIiKxBk58bmaNgNeA+83sE+CrZMMSERGJdyZ1BvA18GOg\nkKjo4aAEY9plDfLyMLPYjw75+dkOWUREyhFndN96M9sXOA5YCTwduv9iMbMBwC1ECfEed/9dOW16\nA38E6gGfunufuPsvz6Zt21g3fnzs9s2q0FZERKpPnNF9I4DZwHDgPOBfZnZBnJ2HsvO3At8GjgCG\nmdmhZdo0BW4DvuPuRwJDqvQORESkxopzTeoXQNfU2VM4q3oVuC/Ga7sDi9y9KLz2IWAwMD+tzXDg\nMXcvBnD31fHDFxGRmizONak1wLq05XVhXRxtie6rSlkR1qXrDLQwsxfN7C0zU4l6EREBMpxJmdkV\n4ekC4A0ze5KoRMcZwPu7OYauQF9gn3CsN9z9w914DBER2QNl6u7bN/xcHh4NwvLzVdh/MdA+bTk/\nrEu3gqj679fA12b2MnA0sFOS+sft15Q8P7BbLw7s1qsKoYiISK4oLCyksLCw0nbm7pU3MtsLICSS\n2Mwsj+hMrB+wCpgFDHP3eWltDgX+TDSTRQOiqZfOcfd/l9mXXz97c6zjju5av8qj++J8DiIikgwz\nw92t7PqM16TM7GIzWwJ8BHxkZovN7JK4B3X3bcAoYDrwAfCQu88zs5Gp/bj7fODvwLvATODOsglK\nRERqp0zXpEYDvYEB7r4wrOsM/MnMWrr79XEO4O7PA4eUWXdHmeWbgJuqFrqIiNR0mc6kvg+ckUpQ\nAOH5WcCIhOMSERHJmKTc3Xeao8/dvwS2JxeSiIhIJFOSWhWmKyrFzHoRXaMSERFJVKYh6FcAT5rZ\ni8DbYV03outUKikvIiKJq/BMyt3fA44kGjZ+aHjMAo4K20RERBKVce6+cE3qzmqKRUREpJQ4c/eJ\niIhkhZKUiIjkrMpmnMgzs/urKxgREZF0GZNUmNboQDOrV03xiIiIlIhT9HAx8IqZPQVsTK109wmJ\nRSUiIkK8JLUsPBqGh4iISLWoNEm5+1gAM9s7LO80VZKIiEgSKh3dZ2aHm9lbwCJgkZm9aWaHJR+a\niIjUdnGGoN8JXO3u+e6eD4wB7ko2LBERkXhJqrG7v5BacPd/AI2TC0lERCQSJ0ktNbPRZpYfHr8A\nliYcl4iISKwkdSHQDpgGTAXywzoREZFEZSoff6+7fx8Y5u4/rL6QREREIpnOpLqb2X7AxWbW2Mya\npD+qK0AREam9Mt0ndTfwGtAe+ACwtG0e1ouIiCQmU9HDm929E3C/u7d393ZpDyUoERFJXKUDJ9z9\n4uoIREREpCzVkxIRkZylJCUiIjkrztx9l5lZ0+oIJlsa1Aczi/XoULB/tsMVEak14pTqKABmm9mb\nwF/CtEg1yqbN4PPitbXDPk42GBERKRFn4MQvgE7AX4FLzWyRmV1jZh0Sjk1ERGq5WNek3H070Xx9\nS4HtwAHAU2Z2fWKRiYhIrVdpd5+Z/Qi4APgCuAcY4+6bzKwO8CEwOtkQRUSktopzTaoN0fx9i9NX\nuvt2Mzs9mbBERETidfe1LZugzOxeAHd/P4mgREREIF6S6pK+ELr5jksmHBERkR0qTFJmdpWZrQW6\nmNma8FgLrCaqLSUiIpKoTGdSNwL7An8MP/cFWrl7C3f/3+oITkREardMAycOdvdFZvYAcERqpVlU\nscPd3004NhERqeUyJanRRGXibytnmwOnJBKRiIhIUGGScvcLw8+Tqy8cERGRHSpMUpXdA+XuT8c5\ngJkNAG4huv51j7v/roJ2xwGvA+e4++Nx9i0iIjVbpu6+IRm2OVBpkgrD1W8F+gErgbfM7Cl3n19O\nuxuAv1casYiI1BqZuvvO3w377w4scvciADN7CBgMzC/T7nLgUXT/lYiIpMnU3TfM3R80syvK2+7u\nE2Lsvy2wPG15BVHiSj9OG+AMd+9jZqW2iYhI7Zapu695+LlvwjHcAlyVtmwJH09ERPYQmbr7Joaf\nY3dh/8VA+7Tl/LAuXTfgIYtuwGoFDDSzLeUNzPjH7deUPD+wWy8O7NZrF0ITEZFsKSwspLCwsNJ2\n5u6ZG0TFDf8I9AyrXgP+x92XVrpzszxgAdHAiVXALKIZ1cutg2tmk4BnyhvdZ2Z+/ezNlR0SgNFd\n67Nu/PhYbQGajR9fhcq8UNlnJiIiVWNmuPtOPWlxJph9kGgkX/vweCasq5S7bwNGAdOBD4CH3H2e\nmY00s0vKe0mc/YqISO0Qp57UPu4+KW35XjO7Mu4B3P154JAy6+6ooO2FcfcrIiI1X6bRfU3C02lm\n9jPgIaIznXOAqdUQm4iI1HKZzqQ+IEpKqT7CH6dtc+DqpIISERGBzKP72lVnICIiImXFuSaFmR0K\nHA7slVrn7lOSCkpERARiJCkz+yXQHziUaG69bwOvAkpSIiKSqDhD0M8B+gCrwnx+RwP7JBqViIgI\n8ZLUV+F+p61m1hj4CChINiwREZF416TmmFkz4C/Av4AviGaOEBERSVSlScrdR4ant5nZ34Em7j47\n2bBERETij+47HTiJ6P6oVwElKRERSVyl16TM7M9EN/IuAj4ErjCzOLWkREREdkmcM6lvAYd7mPrb\nzP4CvJ9oVCIiIsQb3fcfojpQKQcAi5MJR0REZIdME8w+QXQNai9gnpnNDJt6AG9WQ2wiIlLLZeru\nu7XaohARESlHpglm/5l6bmatiMq8A/zL3VcnHZiIiEic0X1nEQ05Px/4HvAvMzsz6cBERETijO77\nFXCcu38MYGaticrBP5FkYCIiInFG99VJJajgk5ivExER2SVxzqReMLOpwINheShRyQ4REZFExUlS\n/wMMIZoWCeA+4NHEIhIREQkyJikzywOed/f/Ah6unpBEREQiGa8thTpSeWbWpJriERERKRGnu+9z\nYK6ZTQc2pla6+08Ti0pERIR4SerZ8BAREalWlV2TOgr4DPjA3RdVT0giIiKRCq9JmdnVwJPAuUTD\n0C+stqhERETIfCZ1LtDF3Tea2b7ANOAv1ROWiIhI5tF9m9x9I4C7f1pJWxERkd0u05nUgWb2eHhu\nwEFpy7j7dxONTEREar1MSeqsMsuqLyUiItUqVj0pERGRbNB1JhERyVlKUiIikrOUpEREJGfFKR//\nvJk1S1tuHupLiYiIJCrOmVRrd1+XWnD3tUCb5EISERGJxElS280sP7VgZu0TjEdERKREnCT1K+A1\nM5tkZvcCLwNXxz2AmQ0ws/lmttDMripn+3Azmxser4ZJbUVERCov1eHuU82sO9AzrPq5u38SZ+dm\nVofoJuB+wErgLTN7yt3npzVbApzi7p+b2QDgLqBHVd6EiIjUTJlmQe8UfnYBWhMlkyXA/mFdHN2B\nRe5e5O5bgIeAwekN3H2mu38eFmcCbav2FkREpKbKdCb1C+Ai4LZytjlwSoz9twWWpy2vIEpcFfkB\n8FyM/YqISC2QaVqki8LPk6sjEDPrA4wATqqO44mISO6r9JqUmTUARhIlDwdeAe5y900x9l8MpI8G\nzA/ryh6jC3AnMCAMcS/XP26/puT5gd16cWC3XjFCEBGRXFNYWEhhYWGl7czdMzcwewjYBEwOq4YD\ne7v70Ep3bpYHLCAaOLEKmAUMc/d5aW3aA/8Eznf3mRn25dfP3lzZIQEY3bU+68aPj9UWoNn48eyI\nKDM7DCr7zEREpGrMDHe3susrPZMiqs57eNryC2b27zgHdfdtZjYKmE40SOMed59nZiOjzX4nMBZo\nAUw0MwO2uHum61YiIlJLxElSc83sOHd/C8DMjgXmxD2Auz8PHFJm3R1pzy8GLo67PxERqT3iJKmj\ngDfNbElY7gjMM7M5RGdDXROLTkREarU4SWpw5U1ERER2vzgzTiw2syOA1FD0V9z9g2TDEhERiVeq\nYxTwCNFQ8vbAw2b2w6QDExERidPddwnQ3d03AJjZb4HXgYlJBiYiIhJnFnQD0m9Q2hLWiYiIJKrC\nMykzq+vuW4EHiEb3PRY2nQncVx3BiYhI7Zapu28W0NXdbzSzQnbMqXdp6p4pERGRJGVKUiVdeu4+\niyhpiYiIVJtMSWpfM/tpRRvd/eYE4hERESmRKUnlAY3QIAkREcmSTElqlbtfk2G7VFGH/HyKineq\nVFKugrYBcrr/AAAPlElEQVRtWbpiRcIRiYjktljXpGT3KCoujl1CpPVvxxNNCh9PQfvWLC366BtG\nJiKSmzIlqX7VFoXsZNNmYte4ArDDPk4uGBGRLKnwZl53X1OdgYiIiJQVZ8YJySC/oANmFushIiJV\nE2fuPsmgeFkRVSlrLyIi8elMSkREcpaSlIiI5CwlKRERyVlKUiIikrOUpEREJGcpSYmISM5SkhIR\nkZylJCUiIjlLSUpERHKWkpSIiOQsJakaqCrzCZoZ+QUdYu+7Q8H+sffboWD/5N7kHhBXVWKq7s9L\nZE+huftqoKrMJwhVm1OwaNnHsUuIVGf5kFyMqyoxgcqtiJRHZ1IiIpKzlKREaplc7BoVqYi6+4QG\neXmqd1WL5GLXqEhFlKSETdu2sW78+Fhtm8VsJyKyO6i7T3JSh/z8Ko2ME5GaSWdSkpOKiotjn92B\nzvBqgg4F+1O0LF73YkH71iwt+ijhiHIzJsjduJKgJCVSBR3y8ykqLs52GDvJ1biqIhevleViTJCb\ncVUlcUL85KkkJdUmv6ADxcuKsh3GLqnKGV51nt3lalxSeyR1X6CSlFSbqtxkXJUbjCU31YSzO8m+\nxJOUmQ0AbiEapHGPu/+unDYTgIHARuD77v5O0nGJSLJy9bpiribPXI0r2xJNUmZWB7gV6AesBN4y\ns6fcfX5am4HAQe7eycyOB24HeiQZl0i6mtANKfHlatdoLsa1/7778vHq1dVyrIokfSbVHVjk7kUA\nZvYQMBiYn9ZmMHA/gLu/aWZNzay1u+suQqkW6oYUKd/Hq1dnPXEmfZ9UW2B52vKKsC5Tm+Jy2ojU\nKlWdyT4X4xLZHTRwQiQHJTmT/a7I1bPOXOyyzcWYIHfjqoi5e3I7N+sBjHf3AWH5F4CnD54ws9uB\nF939b2F5PtCrbHefmSUXqIiIZJ2773QKnvSZ1FvAwWZWAKwChgLDyrR5GvgR8LeQ1NaVdz2qvOBF\nRKRmSzRJufs2MxsFTGfHEPR5ZjYy2ux3uvs0MzvVzD4kGoI+IsmYRERkz5Fod5+IiMiuqHGzoJvZ\nADObb2YLzeyqbMcDYGb3mNnHZvZutmNJMbN8M5thZh+Y2XtmdkW2YwIwswZm9qaZzQlxjct2TClm\nVsfMZpvZ09mOJcXMlprZ3PB5zcp2PCnhVpJHzGxe+B07Pgdi6hw+p9nh5+e58HtvZlea2ftm9q6Z\n/dXMcuI+BzP7cfg/mNW/DzXqTCrcPLyQtJuHgaHpNw9nKa6TgA3A/e7eJZuxpJjZ/sD+7v6OmTUC\n3gYGZ/uzAjCzhu7+pZnlAa8BV7h71v8Am9mVwLFAE3c/PdvxAJjZEuBYd1+b7VjSmdm9wEvuPsnM\n6gIN3f2LLIdVIvytWAEc7+7LK2ufYBxtgFeBQ919s5n9DZjq7vdnK6YQ1xHAg8BxwFbgOeBSd19S\n3bHUtDOpkpuH3X0LkLp5OKvc/VUgp/6IuPtHqemn3H0DMI8cuT/N3b8MTxsQXTfN+jcpM8sHTgXu\nznYsZRg59v/YzJoAJ7v7JAB335pLCSr4FrA4mwkqTR6wTyqZE33BzrbDgDfdfZO7bwNeBr6bjUBy\n6pd7N4hz87CUYWYdgP8HvJndSCKhW20O8BHwgru/le2YgD8C/0sOJMwyHHjBzN4ys4uzHUzQEVht\nZpNC19qdZrZ3toMq4xyiM4WscveVwB+AZUQTGaxz939kNyoA3gdONrPmZtaQ6Atau2wEUtOSlFRR\n6Op7FPhxOKPKOnff7u7HAPnA8WZ2eDbjMbPTgI/DmaeFR6440d27Ev0R+VHoWs62ukBX4LYQ25fA\nL7Ib0g5mVg84HXgkB2JpRtTbUwC0ARqZ2fDsRgWh2/93wAvANGAOsC0bsdS0JFUMtE9bzg/rpByh\ne+FR4AF3fyrb8ZQVuoheBAZkOZQTgdPD9Z8HgT5mltVrBinuvir8/BR4gqjLO9tWAMvd/V9h+VGi\npJUrBgJvh88s274FLHH3NaFb7XHghCzHBIC7T3L3bu7eG1hHdL2/2tW0JFVy83AYITOU6GbhXJBr\n38AB/gL8293/lO1AUsyslZk1Dc/3Bv6L0hMSVzt3v9rd27v7gUS/UzPc/XvZjAmiASbhTBgz2wfo\nT9RNk1XhZvzlZtY5rOoH/DuLIZU1jBzo6guWAT3MbC+LJjzsR3R9OOvMbN/wsz1wJjAlG3HUqLn7\nKrp5OMthYWZTgN5ASzNbBoxLXVTOYkwnAucC74XrPw5c7e7PZzMu4ADgvjD6qg7wN3efluWYclVr\n4IkwZVhd4K/uPj3LMaVcAfw1dK0tIUdu0g/XV74FXJLtWADcfZaZPUrUnbYl/Lwzu1GVeMzMWhDF\n9cNsDX6pUUPQRUSkZqlp3X0iIlKDKEmJiEjOUpISEZGcpSQlIiI5S0lKRERylpKUiIjkLCWpWsTM\nWqSVKlhlZivSlqt0z1woP9JpF+NpaGYv7so+0vZ1ZVVLHJhZPzN7opz1F5nZH3dHXFWMp9LP1Mwe\nMLOdZmA3s45mds43OObNoRTDb8us72Nmu2X2Cos8b2ZrzezxMtsOtKg0y0Izmxxmvk9tm2hmi8zs\nHTOrtuoBZnZQuHewou0NzOylcPOtJExJqhYJU68cE+ZT+z/g5tSyu2+t4r4ucvdFuxjSD4CHd3Ef\nhD9sPwX2+gYvr+hGwWq/gXAXP9ODiGbDiC38kR3h7ke5+9VlNvcFen7DWErx6GbM3wEXlLP598AN\n7t4Z+Ar4fohtENDW3TsBPwIm7o5YypOeGNNU+O/v7puAQmBIUjHJDkpStVepb4Fm9vPwjfrdMGtH\n6hvl+2b2oJn928weMrMGYdsrqW+3Znaamb0dzsqeD+v6hm/As83sXxXMgn0u8FRo3ybsc3aIoUdY\nf15YftfMrgvr8sK38j+a2TtEs5PvB7xiZtNDm4Fm9no49oOp44dY55vZv8hcxqWDmRWa2QIzuzq8\n9joz+1HaZ3aDmV1W5nP8hZldGp7/2cz+Hp7/l0U1llKFOcuLLf0zHRmO/YaZ3WVmN6cdpq+ZvWZm\nH5pZ6j1cD/QOn9+oMjGZmf0h/PvONbNUyYVniSY0nZ22DjM7kOgLxM/Cth5m1sGiIpnvmNnfLaqD\nlDqzmxjey3wzK3eeRXd/EdhYJq46wCnAk2HVfcAZ4flg4P7w2teA1mbWsszrh5rZ78Lz/zGzBeF5\nJzMrDM/7h9/LuWZ2h4UeAzNbbmbXm9nbwBlm1i20mQ1cmnaMo8xsVvgc3rGoYgBEv7fnlvdeZTdz\ndz1q4QMYB/w0PO9ONB1LfaAR0TxrRxB9O98OHBfa3UdUgBDgFaAL0dQ8RUB+WN8s/JyW9rqGhNlN\n0o7fAFiRtvxz4H/DcwuvaQv8B2hOVHOnkGi277wQ1+C01y8DGofn+4a2e4Xlq4lm4d6bqJRLh7D+\nUeDxcj6bi0K7JiGOD8J7PQiYFdrUARYDTcu89kSi6YkgKmY3M7yfa4imBio3tjKfaT7RVEJNiKY7\neo3orBfggbT9HwXMC8/7lfdewraziQrpEf69lgGtwue4poLX/Cb1b5327zk0PL8YeCQtnqfD885h\n3/Uq2GepGEMs/05b7gDMDs+fA7qnbSsEupTZX1vgtfD8CaJSM/sCFwK/LuffezLR9D6E9T9J29f7\nQI/w/Oa0OCYCQ8LzekD98DyPaGb8rP9frukPnUkJwEnAY+6+2aNyHU8CJ4dtS3xHPafJoW26nkQT\nrq4AcPd1Yf1rwITwrb6ph//ZafYD1qQtvwX8wMzGAkd5VPjweOCf7r7WoxmipxB98wbY5KVnbk+f\nwPcE4HDgdYuuLQwn+gN4OLDA3ZeGdn/N8Jn83d2/CHE8CZzk7ouBLyyqWjqQqCjc52Ve9xZwnEWT\n5G4Iy8cSfZ6vVBBbQZl9pN73Fx51wz5aZvuTAO7+HlF5h8qcRJhQ1aPJX18BuoVtca+rHA/8LTy/\nn9K/Bw+HfS8kSlK7dK0yLncvBlpYNB/f/iGOXuz4rA+j9L/3/ez4/YHwfsIZ2l7uPjOsfyCtzevA\nWDP7X6C9u28Ox94GbE/1LEhylKSkqsrrq9/pD527X0f0jbsRMNPMDirT5CvSriF51B3UG1hFNMHs\nsIr2nfb6ihjwnEfX2o5x9yPd/dK0bXGUfZ+p5XuIzohGEM0iX7pR9EdsJfA9ojOpV4jOINq7+4cV\nxHZZ2f1UEuemmO0qkv6auNfeMrVL32ZV2OenQCuzkgEI6aV1iildZK+isjszic58PyD6rE8mSqiv\np8VTkY0ZtgHg7pOJuiA3Ac9b6Xpd9T26PiUJUpISiP5zn2nRqKVGRNcDXgnbOprZseH58LT1Ka8T\nXQtpD2BmzcPPA939fXe/AZgNHJL+IndfDeyddo2gPVH3yd3AvcAxRN03vS2qDlqXaGBAYdhF2T8+\nXxB1j6Vi6mVmHcO+G5rZwUTdmKlSLkZUsqEi/c2sSfiWPpjozBCiej+DgKO94gqqrwA/Iyq5/SrR\nhf9UbaWKYks3K7zvJhbNIp6pbHfqc1gPNM4Qz9Bwbao10dlcKp6K/oivZ8fnCVEyODs8Pz+8t5Qh\n4b10JkomFQ3+KFWuxt23h9hS7+8CwjVKohI73wv7PQn4yN0/K2efrxJ91i8R/Z59G1gfzoDnEf17\ndwhtz2PH70+JsN+vzOz4sKrkWpOZdXT3Je4+gegaXuqa4X6oVl21UJISQnfeg0R/uF4nqqj6Qdg8\nD/ipmf2bqI//rtTLwms/AS4DngrdV5PD9p+FC/XvEP3BK6+ExD/YUeCtH5C6cH0m8OfQnTOWHX+A\nXvcdpUTKflu/C/iHmU0PMf0A+Fs4/mtAJ3f/KsT6PFEiWJnhY3mL6A/lHKJrQO+G97uJ6A90pnpE\nrxB1Z870qDz45vCa1Od1UdnY0t+Tuy8nGvX2VnjdYuDz9DZpUstzgLphkMCoMm0eJarJ9S7Rv8OV\n4UtCeftLeQo426IBMT2IEu3IEPMQ4Mq0tsUWDUR5CrjYyxkpamavE3Wv9jezZWbWJ2z6OXCVmS0E\n9iH6ggLwDLDSzD4EbgvHL88rRInx5XDcFez4rL8i+qyfMLO5wNfA3RW87wuBO8PvX3oF2uEWDR6a\nQ/TvlPr97gNMrSAm2Y1UqkMqFLroHvWolHsS++8GXObuFyWx/ySEEWlziAZtLE3wOPu4+8ZwBvkU\nMNHdc+6Popk9QDSIIleKi1YLM3uSKNn/J9ux1HQ6k5LKJPYtxqPy4q8mtf/dzcyOBD4EpiWZoILf\nhG/1c4H5uZigglr3Ldeim8YfUYKqHjqTEhGRnKUzKRERyVlKUiIikrOUpEREJGcpSYmISM5SkhIR\nkZylJCUiIjnr/wNRBmUj3gTArwAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "a = np.sort(tpm_low_alpha.predict(obama,output_type='probability')[0])[::-1]\n",
+ "b = np.sort(topic_model.predict(obama,output_type='probability')[0])[::-1]\n",
+ "c = np.sort(tpm_high_alpha.predict(obama,output_type='probability')[0])[::-1]\n",
+ "ind = np.arange(len(a))\n",
+ "width = 0.3\n",
+ "\n",
+ "def param_bar_plot(a,b,c,ind,width,ylim,param,xlab,ylab):\n",
+ " fig = plt.figure()\n",
+ " ax = fig.add_subplot(111)\n",
+ "\n",
+ " b1 = ax.bar(ind, a, width, color='lightskyblue')\n",
+ " b2 = ax.bar(ind+width, b, width, color='lightcoral')\n",
+ " b3 = ax.bar(ind+(2*width), c, width, color='gold')\n",
+ "\n",
+ " ax.set_xticks(ind+width)\n",
+ " ax.set_xticklabels(range(10))\n",
+ " ax.set_ylabel(ylab)\n",
+ " ax.set_xlabel(xlab)\n",
+ " ax.set_ylim(0,ylim)\n",
+ " ax.legend(handles = [b1,b2,b3],labels=['low '+param,'original model','high '+param])\n",
+ "\n",
+ " plt.tight_layout()\n",
+ " \n",
+ "param_bar_plot(a,b,c,ind,width,ylim=1.0,param='alpha',\n",
+ " xlab='Topics (sorted by weight of top 100 words)',ylab='Topic Probability for Obama Article')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here we can clearly see the smoothing enforced by the alpha parameter - notice that when alpha is low most of the weight in the topic distribution for this article goes to a single topic, but when alpha is high the weight is much more evenly distributed across the topics.\n",
+ "\n",
+ "__Quiz Question:__ How many topics are assigned a weight greater than 0.3 or less than 0.05 for the article on Paul Krugman in the **low alpha** model? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 52,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "+---------------------+-------------------------------+\n",
+ "| average predictions | topics |\n",
+ "+---------------------+-------------------------------+\n",
+ "| 0.629444444444 | art and publishing |\n",
+ "| 0.0176543209877 | Great Britain and Australia |\n",
+ "| 0.0161111111111 | international athletics |\n",
+ "| 0.0146913580247 | American college and politics |\n",
+ "| 0.0136419753086 | science and research |\n",
+ "| 0.0131481481481 | team sports |\n",
+ "| 0.0129012345679 | general politics |\n",
+ "| 0.010987654321 | Business |\n",
+ "+---------------------+-------------------------------+\n",
+ "[? rows x 2 columns]\n",
+ "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
+ "You can use sf.materialize() to force materialization.\n",
+ "8\n"
+ ]
+ }
+ ],
+ "source": [
+ "paul_krugman = gl.SArray([wiki_docs[int(np.where(wiki['name']=='Paul Krugman')[0])]])\n",
+ "paul_krugman_low_alpha = average_predictions(tpm_low_alpha, paul_krugman, 100)\n",
+ "print paul_krugman_low_alpha[(paul_krugman_low_alpha['average predictions'] > 0.3) |\n",
+ " (paul_krugman_low_alpha['average predictions'] < 0.05)\n",
+ " ]\n",
+ "print len(paul_krugman_low_alpha[(paul_krugman_low_alpha['average predictions'] > 0.3) |\n",
+ " (paul_krugman_low_alpha['average predictions'] < 0.05)\n",
+ " ])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ How many topics are assigned a weight greater than 0.3 or less than 0.05 for the article on Paul Krugman in the **high alpha** model? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 53,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "+---------------------+--------------------+\n",
+ "| average predictions | topics |\n",
+ "+---------------------+--------------------+\n",
+ "| 0.629444444444 | art and publishing |\n",
+ "| 0.010987654321 | Business |\n",
+ "+---------------------+--------------------+\n",
+ "[? rows x 2 columns]\n",
+ "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
+ "You can use sf.materialize() to force materialization.\n",
+ "2\n"
+ ]
+ }
+ ],
+ "source": [
+ "paul_krugman_high_alpha = average_predictions(tpm_high_alpha, paul_krugman, 100)\n",
+ "print paul_krugman_low_alpha[(paul_krugman_high_alpha['average predictions'] > 0.3) |\n",
+ " (paul_krugman_high_alpha['average predictions'] < 0.05)\n",
+ " ]\n",
+ "print len(paul_krugman_low_alpha[(paul_krugman_high_alpha['average predictions'] > 0.3) |\n",
+ " (paul_krugman_high_alpha['average predictions'] < 0.05)\n",
+ " ])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Changing the hyperparameter gamma\n",
+ "\n",
+ "Just as we were able to see the effect of alpha by plotting topic weights for a document, we expect to be able to visualize the impact of changing gamma by plotting word weights for each topic. In this case, however, there are far too many words in our vocabulary to do this effectively. Instead, we'll plot the total weight of the top 100 words and bottom 1000 words for each topic. Below, we plot the (sorted) total weights of the top 100 words and bottom 1000 from each topic in the high, original, and low gamma models."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now we will consider the following two models:\n",
+ " - tpm_low_gamma, a model trained with gamma = 0.02 and default alpha\n",
+ " - tpm_high_gamma, a model trained with gamma = 0.5 and default alpha"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 54,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "del tpm_low_alpha\n",
+ "del tpm_high_alpha\n",
+ "tpm_low_gamma = gl.load_model('topic_models/lda_low_gamma')\n",
+ "tpm_high_gamma = gl.load_model('topic_models/lda_high_gamma')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 55,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmclWX9//HXm1XJDaRUZmRAUVNxQ1JQFNQyxFzQHwiS\nW2VIkoUtZqZImaaZlanh9nXBfV/SzFzGDRQMxDQQl9hdQVQwZfv8/rjvGc6MM4d7YM6cw/B+Ph7n\nMfdynev+nDMz53Pu677u61JEYGZmVopaFDsAMzOz+jhJmZlZyXKSMjOzkuUkZWZmJctJyszMSpaT\nlJmZlayCJylJ/SVNlzRD0hn1lOknaYqkVyQ9WeiYzMxs3aBC3iclqQUwAzgImA9MAoZExPScMpsC\n44GDI2KepI4R8UHBgjIzs3VGoc+k9gJej4hZEbEMuA04olaZY4G7I2IegBOUmZlVKXSSKgPm5KzP\nTbfl2h7oIOlJSZMkHVfgmMzMbB3RqtgBkMTQAzgQ+BIwQdKEiHijuGGZmVmxFTpJzQM656yXp9ty\nzQU+iIjPgM8kPQ3sBtRIUpI8yKCZWTMWEaq9rdDNfZOAbpIqJLUBhgAP1CpzP9BHUktJ7YC9gWl1\nVRYRmR6jR4/OXLYpH45r3Y7Jca37MTmu0o2pPgU9k4qIFZJGAo+SJMRrI2KapOHJ7rgqIqZL+gfw\nMrACuCoi/lPIuMzMbN1Q8GtSEfEIsEOtbVfWWr8YuLjQsZiZ2bqlWY440a9fv2KHUCfHlV0pxgSO\nqyFKMSZwXA1RCjEV9GbexiQp1pVYzcysYSQRdXScKIUu6Ga2HujSpQuzZs0qdhhWZBUVFcycOTNz\neZ9JmVmTSL8pFzsMK7L6/g7qO5NqltekzMyseXCSMjOzkuUkZWZmJctJyszWe127duWJJ54odhhW\nBycpMyua8oouSCrYo7yiS7Ffoq0ld0E3s6KZN3sWF0xeWrD6z+zRpmB1W9PwmZSZWY6lS5fy4x//\nmLKyMsrLyxk1ahTLli0DkhEY7r33XgCee+45WrRowd///ncAnnjiCfbYY4866/zss8844YQT6NCh\nAzvvvDO///3v2Xrrrav3X3jhhXTr1o1NNtmE7t27c99991Xvu+GGG+jTpw+nn3467du3p1u3bkyY\nMIEbbriBzp07s+WWW3LjjTdWlz/ppJM49dRTGTBgABtvvDH77bcf7777LqNGjaJDhw7stNNOTJ06\nNdOxS4GTlJlZjvPOO4+JEyfy8ssvM3XqVCZOnMh5550HQN++famsrATg6aefZtttt+Xpp58G4Kmn\nnqp3GKFzzz2X2bNnM3PmTP75z39y0003Ia26Jahbt24899xzfPzxx4wePZpvf/vbvPvuu9X7J06c\nyO67787ChQsZOnQoQ4YM4cUXX+TNN99k3LhxjBw5kk8//bS6/J133sn555/PggULaNOmDb1796Zn\nz54sWLCAo48+mlGjRmU+drE5SZmZ5bjlllsYPXo0m2++OZtvvjmjR49m3LhxQJKknnrqKSBJUmee\neWb1+lNPPUXfvn3rrPPOO+/krLPOYpNNNqFTp06cdtppNfYfffTRbLHFFgAMGjSI7bbbjokTJ1bv\n79q1K8cffzySOOaYY5g7dy6jR4+mdevWfOMb36BNmza88caqKfgGDhzI7rvvTps2bRg4cCAbbrgh\nw4YNq37+Sy+9lPnYxeYkZWaWY/78+XTuvGqu1oqKCubPnw9A7969mTFjBu+99x5Tp07l+OOPZ86c\nOSxYsICJEyey//7711tneXl59XpuUx/AjTfeyB577EH79u1p3749r776Kh988EH1/qokArDhhhsC\n0LFjxxrbFi9eXG/52uu5ZVd37GJzkjIzy9GpU6caYwzOmjWLTp06AckH/J577smf//xnunfvTqtW\nrejduzeXXHIJ3bp1o0OHDvXWOXfu3Or12bNn11j+/ve/zxVXXMGHH37Ihx9+yM4779wkQ0gV89hZ\nOUmZmeUYOnQo5513Hh988AEffPABv/nNbzjuuOOq9++///5cdtll1U17/fr1q7Fel0GDBnHBBRew\naNEi5s2bx+WXX169b8mSJbRo0YKOHTuycuVKrrvuOl555ZW8Ma5tEql6/pocu6k5SZnZei+3E8Ov\nfvUrevbsya677spuu+1Gz549Oeuss6r39+3bl8WLF1c37VWt50tS55xzDmVlZXTt2pWDDz6YQYMG\n0bZtWwB23HFHfvKTn9CrVy+23HJLXn31Vfr06ZM53rrWs77eNTl2U/Mo6GbWJOoa/bq8ogvzZhdu\n+o6yzhXMnTWzYPWvqbFjx3L77bfz5JNPFjuUJudR0M1snTF31kwiomCPUklQ77zzDuPHjycieO21\n1/jDH/7AUUcdVeyw1gkeccLMrMCWLl3K8OHDmTlzJpttthlDhw5lxIgRxQ5rneDmPjNrEp700MDN\nfWZm1ow4SZmZWclykjIzs5LlJGVmZiXLScrMzEqWk5SZWQONGDGC3/72t41eNp9Zs2bRokULVq5c\nudZ1NdRJJ53EOeeck6ls165deeKJJxrt2L5PysyKpkt5ObPmzStY/RVlZczMGdi1sfz1r38tSNnV\naejwR82Bk5SZFc2sefNYdO65Bat/swLUvXLlSlq0cCNUU1ntOy3pcEkbp8s/lXSLpF0KH5qZWdOY\nPn06BxxwAO3bt2eXXXbhwQcfrN530kkn8YMf/IBDDz2UjTfemMrKyi80f1100UV06tSJ8vJyrr32\nWlq0aMFbb71V/fyqsk899RRbb701l1xyCVtssQVlZWVcf/311fU8/PDD9OjRg0033ZSKigrGjBmT\n+TV07dqViy++mN12242NN96Yk08+mffee48BAwawySabcPDBB/PRRx9Vl3/ggQfo3r07HTp04MAD\nD2T69OnV+6ZMmcKee+7JpptuypAhQ/jss89qHOtvf/tb9RxUffr04d///nfmOBsqy9eB8yLiE0m9\ngCOBu4Grsh5AUn9J0yXNkHRGHfv7SlokaXL6+FX28M3M1s7y5cs57LDD6N+/P++//z6XXnopw4YN\n4/XXX68uc+utt3L22WfzySefsO+++9Z4/iOPPMKf/vQnnnjiCd544w0qKyvzNsu98847fPLJJ8yf\nP59rrrmGU089tTp5bLTRRowbN46PPvqIhx56iLFjx/LAAw9kfi333HMPjz/+ODNmzOCBBx5gwIAB\n/O53v+ODDz5gxYoVXHrppQDMmDGDY489lksvvZT333+fQw45hMMOO4zly5ezbNkyBg4cyAknnMDC\nhQsZNGgQd999d/UxpkyZwne/+12uvvpqFi5cyPDhwzn88MNZtmxZ5jgbIkuSWpH+PAy4MiLuBtpm\nqVxSC+Ay4JvAzsBQSV+to+jTEdEjfZyXpW4zs8bw/PPPs2TJEs444wxatWrFAQccwLe+9S1uvfXW\n6jJHHHEEvXr1AqieYqPKnXfeyUknncRXv/pVNthgA85dTRNjmzZtOPvss2nZsiWHHHIIG220Ea+9\n9hqQzFW18847A9C9e3eGDBlSPT19Fj/84Q/p2LEjW221Ffvttx977703u+66a/U08lOmTAHgjjvu\n4Fvf+hYHHnggLVu25Kc//SmfffYZ48eP5/nnn2f58uWcdtpptGzZkqOPPpqvfe1r1ce4+uqrOeWU\nU+jZsyeSOO6442jbti3PP/985jgbIkuSelfSn4EhwEOS2gAtM9a/F/B6RMyKiGXAbcARdZRb/64G\nmllJmD9//hemc6+oqGBeToeO2vvzPX/rrbfOO0bh5ptvXuOaVrt27aqnc3/hhRc48MAD+cpXvsJm\nm23GlVde2aCp3LNOGz9//nwqKiqq90mivLycefPmMX/+fMrKymrUm1t21qxZ/OEPf6BDhw506NCB\n9u3bM3fuXObPn585zobIkqQGAS8Ah0fEQuDLwFn5n1KtDJiTsz433VZbb0kvSXpI0k4Z6zYzW2ud\nOnVizpw5NbbNnj27xgd1vua7rbba6gtTw69pL7xhw4Zx5JFHMm/ePBYtWsTw4cMLMihvp06dmDWr\n5jxec+bMoays7AuvB2pOd7/11ltz1llnsXDhQhYuXMiHH37I4sWLOeaYYxo9TsiTpCS1k9SOpLnv\nPuC/6fqHQON1god/AZ0jYneSpsH7GrFuM7O89t57b9q1a8dFF13E8uXLqays5G9/+xtDhw7N9PzB\ngwdz3XXXMX36dD799FPOO2/Nr1gsXryY9u3b07p1ayZOnMgtt9xSY39jJazBgwfz0EMP8eSTT7J8\n+XIuvvhiNthgA/bZZx969+5N69at+ctf/sLy5cu55557mDhxYvVzTz75ZMaOHVu9bcmSJTz88MMs\nWbKkUWKrLV8X9DeBqndkC+BTkma5DYF3gU4Z6p8HdM5ZL0+3VYuIxTnLf5d0haQO6VlbDbltvf36\n9aNfv34ZQjAzq1/r1q158MEHGTFiBOeffz7l5eWMGzeO7bbbDqj7LCp3W//+/TnttNM44IADaNmy\nJWeffTbjxo37wrWr+uTWdcUVV3D66aczcuRI+vbtyzHHHMOiRYvqLJuvntWV3X777bnpppsYOXIk\n8+fPZ/fdd+fBBx+kVaskJdxzzz1873vf41e/+hUDBgzg6KOPrn7unnvuydVXX83IkSN544032HDD\nDenTpw99+/Zd7XFzVVZWUllZudpyq51PStIVwGMRcU+6PhA4KCJGrrZyqSXwGnAQ8DYwERgaEdNy\nymwREe+my3sBd0RElzrq8nxSZuuwuuYRWldv5s1n+vTp7LLLLnz++ee+n6oODZ1PKkuS+ndE7FJr\n28sRsWvGgPoDfyZpWrw2In4naTgQEXGVpFOBEcAy4H/AqIh4oY56nKTM1mHNedLD++67jwEDBrBk\nyRJOPPFEWrVqVaPbtq1SiCT1GPAIcFO6aRgwICIOWvtws3OSMlu3NeckdcghhzBhwgRatWpFv379\nuPzyy2v0rLNVCpGkvgycB+xPco3qaeCciHivUSLOyEnKbN3WnJOUZdfQJJV37L70mtKPI2J444Vo\nZmaWTd6rehGxAvh6E8ViZmZWQ5ZR0F+UdCdwB1DdET4iHi5YVGZmZmRLUh2A5cBROdsCcJIyM7OC\nWm3HiVLhjhNm6zZ3nDBoeMeJLPNJbSnpVklz08fNkrZspHjNzIou35Tnzz77LDvuuGOmeqrmi7LG\nk+V26OtJup1vnz6eSbeZma2VLhVbIqlgjy4Va/99uk+fPkybNm31BVPr4xTvhZTlmtSWEfHXnPWx\nkk4pVEBmtv6YNftdIvvnf4Npx3cLV7k1iSxnUosk/b+qFUlHA4vylDczW+dMmTKF3Xbbjfbt2zN0\n6FCWLl0KfLEJb/LkydVTvA8ePJghQ4bUmEo+IuqdHr62mTNn0rdvXzbddFMOPvhgRo4cyXHHHVe9\nf/DgwWy11Va0b9+efv368Z///Kd630knncSpp57KgAED2Hjjjdlvv/149913GTVqFB06dGCnnXZi\n6tSp1eUbOr18vmM3pSxJ6jvA9yUtlLQAODndZmbWbNx55508+uij/Pe//2Xq1Kk1kktVE96yZcs4\n6qij+M53vsPChQsZOnQo9957b4168k0PX9uxxx5Lr169WLBgAaNHj2bcuHE1mgsHDBjAm2++yXvv\nvUePHj0YNmzYF2I+//zzWbBgAW3atKF379707NmTBQsWcPTRRzNq1Kga5bNOL5/l2E0l33xSXwKI\niLci4uCI6BARm0dE/4h4q+lCNDMrvB/96EdsscUWbLbZZhx22GG89NJLXygzYcIEVqxYwciRI2nZ\nsiUDBw5kr732qlEm3/TwuebMmcOLL77ImDFjaNWqFfvuuy+HH354jTInnngi7dq1o3Xr1pxzzjlM\nnTqVTz75pHr/wIED2X333aunh99www0ZNmwYkjjmmGO+8BqyTi+f5dhNJd+Z1BxJkyX9RdJQSe6y\nYmbNVu6AsLlTuud6++23vzC1eu3efPmmh881f/58OnTowAYbbFBnXStXruQXv/gF3bp1Y7PNNqNr\n165IqjGdfNbp4htaPsuxm0q+JLU5cCLwKnAI8KSkOZJul3RaUwRnZlZKttpqK+bVmv+q9tTzDalr\n4cKFfPbZZ3XWdfPNN/Pggw/yxBNPsGjRImbOnElENMm9ZsU8dm31JqlIvBwRYyPieJJR0C8E9gQu\nbqoAzcxKRe/evWnZsiWXX345K1as4P77768xtXpDdO7cmZ49e3LuueeybNkyJkyYwIMPPli9f/Hi\nxbRt25b27duzZMkSzjzzzAZ3b1/TpNIYx24s+a5J9ZA0UtItkiYCVwAbAd8D2jdVgGZmhZb1A7h1\n69bcc889XHPNNbRv355bbrmFww47LO9U8fnqvvnmmxk/fjwdO3bknHPOYciQIdV1HX/88XTu3Jmy\nsjK6d+/OPvvs07AXVevYDZlevjGO3VjqHRZJ0kpgMvBHkindlzVlYHXE42GRzNZhdU4fX7Els2YX\n7l6mis5bMHPWOwWrH6BXr16MGDGCE044Ya3rGjJkCDvuuCOjR49uhMhKU6NNeiipC7BP+tiTZHr3\nicAEYEJEzG+0qDNwkjJbtzWXsfuefvppdthhBzp27MhNN93ED37wA9566601mon3xRdfpEOHDnTt\n2pV//OMfHHXUUUyYMIHddtutAJGXhkab9DAiZgIzgVvSCjYi6UhxIdAVaNkYAZuZrUtee+01Bg8e\nzKeffso222zD3XffvcZTxb/zzjscddRRLFy4kPLycsaOHdusE9SayHcmtSGwN6vOpvYC3gbGA89F\nxE1NFWQaj8+kzNZhzeVMytZOYzb3fQhMImneew54PiI+btxws3OSMlu3OUkZNGJzH7B5RKxszODW\nVtYeOGWdK5g7a2ZhgzEzs4LLd02qpBIUwAWTl2Yqd2aPNgWOxMzMmkKWAWbNzMyKIst8UmZma62i\nosITAhoVFRUNKp83SUnqCxwJVI2oOA+4PyIq1yQ4M1t/zZw5s9gh2Doo37BIFwHnAFOBq9LHVOBs\nSb9vmvBKX3lFl8xTWZdXdCl2uGZm65R8Z1JHRsT2tTdKugGYAfysYFGtQ+bNnlVyHTrKK7owb/as\nzOXdG9LMSlW+JLVU0m4RMbXW9l2BzwsYk62lhiROcG9IMytd+ZLU94AbJAVQNclJZyDSfWZmZgWV\n7z6p54Hd04FmqztOpGP6mZmZFVyW+6Q2rfVoEEn9JU2XNEPSGXnKfU3SMklHNfQYZmbWPNV7JiXp\nAGAsMJ+k6zlAuaStgFMi4snVVS6pBXAZcFBazyRJ90fE9DrK/Q74xxq9CjMza5byXZO6DDg0It7I\n3ShpO+B+YKcM9e8FvB4Rs9Ln3gYcAUyvVe6HwF3A1zLGbeughvQ6dI9DM4P8SaoN8FYd2/+b7sui\njFWdLgDmkiSuapI6kXR3P0BSjX3WvJRid30zK235ktRNwARJt7Aq0WwNHJvuayx/AnKvVdU7bspj\nY39dvbxNz75s07NvI4Zh6yuf4Zk1vcrKSiorK1dbLl/vvjGSHiBpnuuebp4HjIiIyRnjmEfSbb1K\nOauub1XpCdymZFCvjsAhkpZFxAO1K/v6KedkPKxZdj7DM2t6/fr1o1+/ftXrY8aMqbNc3rH7ImIK\nMGUt4pgEdJNUQTKr7xBgaK1jbFO1LOk64MG6EpSZma1/1miqDkn3ZSkXESuAkcCjwKvAbRExTdJw\nSd+v6ylrEo+ZmTVP+bqg19d7T9Tq/JBPRDwC7FBr25X1lP1O1nrNzKz5y9fc92/gBeruyNC+MOGY\nmZmtki9JvQYcFxFv1t4haU4d5c2skXgke7NEviT1G+q/H8rTdJgVkEeyN0vk64J+a559txUmHDMz\ns1XWqHefmZlZU3CSMjOzkuUkZWZmJSvviBMAkloDJwN9SG62fRa4JiKWFTg2MzNbz602SQHXkySn\ncen6UGA/koFmzczMCiZLkuoRETvmrP9d0rRCBWRmZlYlyzWplyXtUbUiaXfgpcKFZGZmlshyJvVV\nkmnfq2bo3Q54RdIkICKi5CYqbNuyJcnMH9lUlJUxc+7cAkZkZmZrIkuSGlzwKBrZ5ytWsOjcczOX\n36wBZddGQ5KnE6eZWYYkFRGvSfoqSe8+gGcjYnphw2qeGpI8mypxmpmVstVek5I0AriPpJlve+Be\nScMLHZiZmVmW5r4RwNci4hMASb8BngPqnBPKzMyssWTp3Sfg85z1z6l7jikzM7NGleVM6hZgvKS7\n0vWjgZsKF5I1NXfoMLNSlW/6eEXiAklPsarjxGkRMaFpwrOm4A4dZlaq8p1J/QvoARAR44HxTRKR\nGb7XzcwS+ZKUrztZ0ZTqvW5m1rTyJakvSzqtvp0RcWkB4jEzM6uWr3dfS6Aj8OV6Hma2nimv6IKk\nTI/yii7FDteagXxnUm9HxDlNFomZlbx5s2dxweSlmcqe2aNNgaOx9UG+MylfkzIzs6LKl6QObrIo\nzMzM6lBvkoqI95syELN1QVXX+CyPLuXlxQ7XbJ2X72beVhGxvCmDMSt1pXrjs0cNseYqX8eJiUAP\nSddHxIlNFI+ZrYFSTZ5maytfkmojaTCwn6TDa++MiAcKF5aZWXblFV2YN3tWprJlnSuYO2tmYQOy\nRpMvSZ0KfBvYDBhUa18AmZKUpP7An0iuf10bERfW2n848BtgJbAMGBURz2WKvpG0bUP2ppLOWzBz\n1jsFjsjMGsJd45uvepNURDwFPCXpxYhYo7mjJLUALgMOAuYDkyTdX2tm38eqzsok7QLcAey4Jsdb\nU58vhZiWrax2fLewwZg1Ex5/0RpDlqk6/k/SD4D90/WngKszdqrYC3g9ImYBSLoNOAKoTlIR8WlO\n+Y1IzqjMbB3n8RetMWRJUpcBXwL+L13/NrAH8P0Mzy0D5uSszyVJXDVIOhK4gGS4pUMz1GtmZuuB\nLEmqV0TslrP+qKSpjRlERNwH3CepD3Ae8I26yj029tfVy9v07Ms2Pfs2Zhhmth5wd/3SUFlZSWVl\n5WrLZUlSKyV1iYiZAJK6kL1Jbh7QOWe9PN1Wp4h4VtI2kjpExMLa+79+iocSNLO14+76paFfv370\n69even3MmDF1lss3LFKVM4BnJD0m6XGSa1I/yxjHJKCbpApJbYAh1OoVKGnbnOUeQJu6EpSZ2bqk\nISPGe9T4+q32TCoiHpW0Pat63E2LiP9lqTwiVkgaCTzKqi7o0yQNT3bHVcDRko4HlgL/AwavyQsx\nMyslDekWD+4aX58szX2kSWnymhwgIh4Bdqi17cqc5YuAi9ak7uasIfduge/fMrPmKVOSsqbXkHu3\nwPdvmVnzlOWalJmZWVGsNklJul3SN9WQtidrtqqaITNNVVGx5Xofl5mtnSzNfdcB3wEuk3Q7cH1E\nvFHYsKxUleoQUqUal5mtndWeSUXEIxFxDMlIEe8AT0p6WtJxknxNy8zMCibTNSlJ7YFjgeOAl4Er\ngX2ARwoXmpmZre9WeyYk6U5gF+Bm4OiIqBoj5GZJUwoZnJmZrd+yNNddRTKdRlRtqJpaPiL2KFxo\nZma2vsvS3HdhboJKTSxEMGZmZrnqPZOS9BVgK2DDdDLCqi7omwDtmiA2MzNbz+Vr7juUpOt5OXBF\nzvZPgLMLGZSZmRnknz7+OuA6SYMj4o4mjMmsWfD4i2ZrL19z39CIuBXYStJptfdHxKUFjcxsHefx\nF60hPBlj3fI197VPf3ZsikDMzNZnTTUZY3lFF+bNnpWpbFnnCubOmrnGx2oM+Zr7rkh/+vqTWTPS\nkGZIN0E2Pw2Z56oU5rjK19x3Sb4nRsTpjR+OmRWaxzm0dUm+5r5XmywKM1vv+QzP6pKvue/apgzE\nzNZvPsOzuuRr7vtDRPxE0r1A7REniIijChqZmZmt9/I1992e/rysKQIxMzOrLV9z38T05+OSWgPb\nkZxRvR4Ry5soPjOzovF1suLLMlVHf5KR0GeTjN9XLunkiHi00MGZmRWTr5MVX5apOv4EfD0iZgBI\n2h64H9ixkIGZmZllmapjcVWCAkiXlxQuJDMzy6eqGTLLo0vFlsUOd63k6913eLo4UdIDwB0k16QG\nAS80QWxmZlaH9akZMl9z36Cc5Y+Ab6bLnwAbFywiMzOzVL7efcc1ZSBmZma1Zend1xY4EdgZ2KBq\ne0R8v3BhmZmZZes4cSPQBfgWybWobYHPChiTmZkZkC1JbR8RZ5L08rsW6A/sVdiwzMzMsiWpZenP\nRZJ2JOk08ZWsB5DUX9J0STMknVHH/mMlTU0fz0raJWvdZmbWvGW5mfdaSe2B0cA/gHbAOVkql9SC\nZOy/g4D5wCRJ90fE9JxibwH7R8RH6egWVwO9GvAazMysmVptkoqIK9PFJ4HODax/L5Kx/mYBSLoN\nOAKoTlIR8XxO+eeBsgYew8zMmqnVNvdJai/pj5ImSnpB0sXpmVUWZcCcnPW55E9C3wP+nrFuMzNr\n5rJck7oN+BgYBnyb5Gbe2/M+Yw1IOgA4CfjCdSszM1s/ZbkmVRYRo3PWx0h6JWP986jZRFiebqtB\n0q4kI633j4gP66vssbG/rl7epmdftunZN2MYZmbWUG1btsw+VUlZGTPnzs1cd2VlJZWVlastlyVJ\nPS7p/0XEXQCSjgL+mTGOSUA3SRXA28AQYGhuAUmdgbuB4yLizXyVff2UTP01zMysEXy+YgWLzj03\nU9ktzj83c0KDL86/NWbMmDrL5Rtg9kOSAWUF/FDS8pznLAJGrS6IiFghaSTwKEnT4rURMU3S8GR3\nXAWcDXQArlDyCpdFhO/DMjNbhzRk0FvIPvBtvjOpjtkPV7+IeATYoda2K3OWTwZOboxjmZlZ85Jv\ngNkVVcuSBgD7p6uVaeIxMzMrqCxd0H8L/Jzkptu3gJ9LOq/QgZmZmWXpOHEYsEfVmZWk/wMmA78q\nZGBmZmZZ7pMC2CRn2RMemplZk8hyJnURMFnS4yQ9/fqR9MgzMzMrqLxJKu0S/jjJuH17p5vPiYgv\n3JBrZmbW2PImqYgISf+MiO7APU0Uk5mZGZDtmtRLkvYoeCRmZma1ZLkmtQfJPFBvAktIrktFRPQo\naGRmZraCEh0KAAARKElEQVTey5KkDi94FGZmZnXIN3ZfW5LhiroB/wauzx2FwszMrNDyXZO6HugD\nvA4cCVzcFAGZmZlVydfc1z0idgGQdBXwQtOEZGZmlsh3JrWsaiEiluUpZ2ZmVhD5zqR2k7QwXRaw\ncbpe1buvQ8GjMzOz9Vq+JNWmyaIwMzOrQ6b5pMzMzIoh6yjoZmZmTc5JyszMSpaTlJmZlax8I058\nCERdu3DvPjMzawL5evd1bLIozMzM6pC5d5+kDsAGOZvmFyooMzMzyHBNStKhkmYAc0mGRpoLPFHo\nwMzMzLJ0nPgtsC/wWkRsDXwTeKagUZmZmZEtSS2PiPeBFpIUEf8E9ipwXGZmZpkmPfxI0kbAs8CN\nkt4D/lfYsMzMzLKdSR1JkpR+DFQC84BvFTAmMzMzIFuSOjMiVkTEsoi4NiIuAU4vdGBmZmZZklT/\nOrYd2tiBmJmZ1VZvkpI0XNIUYAdJk3MerwPTsh5AUn9J0yXNkHRGHft3kDRe0meSfIZmZmbV8nWc\nuAN4HLgA+EXO9k8i4r0slUtqAVwGHERy8+8kSfdHxPScYguAH5Jc+zIzM6tW75lURHwYEW9ExCCS\nkSa+kT6+3ID69wJej4hZ6RT0twFH1DrOBxHxL2B5g6M3M7NmLcuIE6cCdwKd08cdkn6Qsf4yYE7O\n+tx0m5mZ2WpluU9qOLBXRCwGkHQ+MB64opCBmZmZZUlSApbmrC9Lt2Uxj+Tsq0p5um2NPDb219XL\n2/TsyzY9+65pVWZmVmTnnnvuasvkm0+qVUQsB8YBL0i6O901ELghYwyTgG6SKoC3gSHA0Dzl8ya/\nr59yTsbDmplZqctNUmPGjKmzTL4zqYlAj4i4SFIl0CfdfkpETMoSQESskDQSeJTk+te1ETFN0vBk\nd1wlaQvgRWBjYKWkHwE7VTUvmpnZ+itfkqo+q4mIiSRJq8Ei4hFgh1rbrsxZfhfYek3qNjOz5i1f\nkvpyvptr0+GRzMzMCiZfkmoJbET2ThJmZmaNKl+Sejsifp1nv5mZWUHlu5nXZ1BmZlZU+ZLUQU0W\nhZmZWR3yjd23sCkDMTMzqy3LfFJmZmZF4SRlZmYly0nKzMxKlpOUmZmVLCcpMzMrWU5SZmZWspyk\nzMysZDlJmZlZyXKSMjOzkuUkZWZmJctJyszMSpaTlJmZlSwnKTMzK1lOUmZmVrKcpMzMrGQ5SZmZ\nWclykjIzs5LlJGVmZiXLScrMzEqWk5SZmZUsJykzMytZTlJmZlaynKTMzKxkOUmZmVnJKniSktRf\n0nRJMySdUU+ZSyW9LuklSbsXOiYzM1s3FDRJSWoBXAZ8E9gZGCrpq7XKHAJsGxHbAcOBsYWMyczM\n1h2FPpPaC3g9ImZFxDLgNuCIWmWOAG4EiIgXgE0lbVHguMzMbB1Q6CRVBszJWZ+bbstXZl4dZczM\nbD3kjhNmZlayFBGFq1zqBZwbEf3T9V8AEREX5pQZCzwZEben69OBvhHxbq26CheomZkVXUSo9rZW\nBT7mJKCbpArgbWAIMLRWmQeAU4Hb06S2qHaCgrqDNzOz5q2gSSoiVkgaCTxK0rR4bURMkzQ82R1X\nRcTDkgZIegNYApxUyJjMzGzdUdDmPjMzs7XR7DpOZLl5uKlJulbSu5JeLnYsVSSVS3pC0quS/i3p\ntGLHBCCpraQXJE1J4xpd7JiqSGohabKkB4odSxVJMyVNTd+vicWOp4qkTSXdKWla+je2dwnEtH36\nPk1Of35UCn/3kkZJekXSy5JultSm2DEBSPpR+j9Y1M+HZnUmld48PAM4CJhPck1sSERML3JcfYDF\nwI0RsWsxY6kiaUtgy4h4SdJGwL+AI4r9XgFIahcRn0pqCTwHnBYRRf8AljQK2BPYJCIOL3Y8AJLe\nAvaMiA+LHUsuSdcDT0XEdZJaAe0i4uMih1Ut/ayYC+wdEXNWV76AcXQCngW+GhFLJd0OPBQRNxYr\npjSunYFbga8By4G/A6dExFtNHUtzO5PKcvNwk4uIZ4GS+hCJiHci4qV0eTEwjRK5Py0iPk0X25Jc\nNy36NylJ5cAA4Jpix1KLKLH/Y0mbAPtFxHUAEbG8lBJU6uvAm8VMUDlaAl+qSuYkX7CLbUfghYj4\nPCJWAE8DRxUjkJL6424EWW4etlokdQF2B14obiSJtFltCvAO8M+ImFTsmIA/Aj+jBBJmLQH8U9Ik\nSScXO5hUV+ADSdelTWtXSdqw2EHVcgzJmUJRRcR84A/AbJKBDBZFxGPFjQqAV4D9JLWX1I7kC9rW\nxQikuSUpa6C0qe8u4EfpGVXRRcTKiNgDKAf2lrRTMeORdCjwbnrmqfRRKvaNiB4kHyKnpk3LxdYK\n6AFcnsb2KfCL4oa0iqTWwOHAnSUQy2YkrT0VQCdgI0nHFjcqSJv9LwT+CTwMTAFWFCOW5pak5gGd\nc9bL021Wh7R54S5gXETcX+x4akubiJ4E+hc5lH2Bw9PrP7cCB0gq6jWDKhHxdvrzfeBekibvYpsL\nzImIF9P1u0iSVqk4BPhX+p4V29eBtyJiYdqsdg+wT5FjAiAirouInhHRD1hEcr2/yTW3JFV983Da\nQ2YIyc3CpaDUvoED/B/wn4j4c7EDqSKpo6RN0+UNgW8ARe3MERG/jIjOEbENyd/UExFxfDFjgqSD\nSXomjKQvAQeTNNMUVXoz/hxJ26ebDgL+U8SQahtKCTT1pWYDvSRtIEkk79W0IscEgKQvpz87AwOB\nW4oRR6FHnGhS9d08XOSwkHQL0A/YXNJsYHTVReUixrQvMAz4d3r9J4BfRsQjxYwL2Aq4Ie191QK4\nPSIeLnJMpWoL4N50yLBWwM0R8WiRY6pyGnBz2rT2FiVyk356feXrwPeLHQtAREyUdBdJc9qy9OdV\nxY2q2t2SOpDE9YNidX5pVl3QzcyseWluzX1mZtaMOEmZmVnJcpIyM7OS5SRlZmYly0nKzMxKlpOU\nmZmVLCep9YikDjlTFbwtaW7OeoPumUunH9luLeNpJ+nJtakjp65RDZ3iQNJBku6tY/t3Jf2xMeJq\nYDyrfU8ljZP0hRHYJXWVdMwaHPOSdCqG82ttP0BSo4xeocQjkj6UdE+tfdsomZplhqSb0pHvq/Zd\nIel1SS9JarLZAyRtm947WN/+tpKeSm++tQJzklqPpEOv7JGOp/ZX4JKq9YhY3sC6vhsRr69lSN8D\n7ljLOkg/2E4HNliDp9d3o2CT30C4lu/ptiSjYWSWfsieFBG7RMQva+0+EOi9hrHUEMnNmBcCJ9Sx\n+/fA7yJie+B/wIlpbIcBZRGxHXAqcEVjxFKX3MSYo97ff0R8DlQCgwoVk63iJLX+qvEtUNLP02/U\nL6ejdlR9o3xF0q2S/iPpNklt033PVH27lXSopH+lZ2WPpNsOTL8BT5b0Yj2jYA8D7k/Ld0rrnJzG\n0Cvd/u10/WVJv023tUy/lf9R0ksko5N/BXhG0qNpmUMkjU+PfWvV8dNYp0t6kfzTuHSRVCnpNUm/\nTJ/7W0mn5rxnv5M0otb7+AtJp6TLf5H0j3T5G0rmWKqamLOu2HLf0+HpsSdIulrSJTmHOVDSc5Le\nkFT1Gi4A+qXv38haMUnSH9Lf71RJVVMu/I1kQNPJOduQtA3JF4ifpvt6SeqiZJLMlyT9Q8k8SFVn\ndlekr2W6pDrHWYyIJ4ElteJqAewP3JduugE4Ml0+Argxfe5zwBaSNq/1/CGSLkyXfyLptXR5O0mV\n6fLB6d/lVElXKm0xkDRH0gWS/gUcKalnWmYycErOMXaRNDF9H15SMmMAJH+3w+p6rdbIIsKP9fAB\njAZOT5f3IhmOpQ2wEck4azuTfDtfCXwtLXcDyQSEAM8Au5IMzTMLKE+3b5b+fDjnee1IRzfJOX5b\nYG7O+s+Bn6XLSp9TBvwXaE8y504lyWjfLdO4jsh5/mxg43T5y2nZDdL1X5KMwr0hyVQuXdLtdwH3\n1PHefDctt0kax6vpa90WmJiWaQG8CWxa67n7kgxPBMlkds+nr+fXJEMD1Rlbrfe0nGQooU1Ihjt6\njuSsF2BcTv27ANPS5YPqei3pvsEkE+mR/r5mAx3T93FhPc/5TdXvOuf3OSRdPhm4MyeeB9Ll7dO6\nW9dTZ40Y01j+k7PeBZicLv8d2CtnXyWwa636yoDn0uV7Saaa+TLwHWBMHb/vm0iG9yHd/uOcul4B\neqXLl+TEcQUwKF1uDbRJl1uSjIxf9P/l5v7wmZQB9AHujoilkUzXcR+wX7rvrVg1n9NNadlcvUkG\nXJ0LEBGL0u3PAZem3+o3jfQ/O8dXgIU565OA70k6G9glkokP9wYej4gPIxkh+haSb94An0fNkdtz\nB/DdB9gJGK/k2sKxJB+AOwGvRcTMtNzNed6Tf0TEx2kc9wF9IuJN4GMls5YeQjIp3Ee1njcJ+JqS\nQXIXp+t7kryfz9QTW0WtOqpe98eRNMPeVWv/fQAR8W+S6R1Wpw/pgKqRDP76DNAz3Zf1usrewO3p\n8o3U/Du4I617BkmSWqtrlVlFxDygg5Lx+LZM4+jLqvd6R2r+vm9k1d8PpK8nPUPbICKeT7ePyykz\nHjhb0s+AzhGxND32CmBlVcuCFY6TlDVUXW31X/igi4jfknzj3gh4XtK2tYr8j5xrSJE0B/UD3iYZ\nYHZofXXnPL8+Av4eybW2PSKie0SckrMvi9qvs2r9WpIzopNIRpGvWSj5EJsPHE9yJvUMyRlE54h4\no57YRtSuZzVxfp6xXH1yn5P12lu+crn71IA63wc6StUdEHKn1plHzUn26pt253mSM99XSd7r/UgS\n6viceOqzJM8+ACLiJpImyM+BR1Rzvq42kVyfsgJykjJI/rkHKum1tBHJ9YBn0n1dJe2ZLh+bs73K\neJJrIZ0BJLVPf24TEa9ExO+AycAOuU+KiA+ADXOuEXQmaT65Brge2IOk+aafktlBW5F0DKhMq6j9\n4fMxSfNYVUx9JXVN624nqRtJM2bVVC4imbKhPgdL2iT9ln4EyZkhJPP9HAbsFvXPoPoM8FOSKbef\nJbnwXzW3Un2x5ZqYvu5NlIwinm/a7qr34RNg4zzxDEmvTW1BcjZXFU99H+KfsOr9hCQZDE6Xj0tf\nW5VB6WvZniSZ1Nf5o8Z0NRGxMo2t6vWdQHqNkmSKnePTevsA70TEgjrqfJbkvX6K5O/sm8An6Rnw\nNJLfd5e07LdZ9fdTLa33f5L2TjdVX2uS1DUi3oqIS0mu4VVdM/wKnquuSThJGWlz3q0kH1zjSWZU\nfTXdPQ04XdJ/SNr4r656Wvrc94ARwP1p89VN6f6fphfqXyL5wKtrConHWDXB20FA1YXrgcBf0uac\ns1n1ATQ+Vk0lUvvb+tXAY5IeTWP6HnB7evzngO0i4n9prI+QJIL5ed6WSSQflFNIrgG9nL7ez0k+\noPPNR/QMSXPm85FMD740fU7V+/Xd2rHlvqaImEPS621S+rw3gY9yy+SoWp8CtEo7CYysVeYukjm5\nXib5PYxKvyTUVV+V+4HBSjrE9CJJtMPTmAcBo3LKzlPSEeV+4OSoo6eopPEkzasHS5ot6YB018+B\nMyTNAL5E8gUF4EFgvqQ3gMvT49flGZLE+HR63Lmseq//R/Je3ytpKvAZcE09r/s7wFXp31/uDLTH\nKuk8NIXk91T1930A8FA9MVkj8lQdVq+0ie6uSKZyL0T9PYEREfHdQtRfCGmPtCkknTZmFvA4X4qI\nJekZ5P3AFRFRch+KksaRdKIolclFm4Sk+0iS/X+LHUtz5zMpW52CfYuJZHrxZwtVf2OT1B14A3i4\nkAkq9Zv0W/1UYHopJqjUevctV8lN43c6QTUNn0mZmVnJ8pmUmZmVLCcpMzMrWU5SZmZWspykzMys\nZDlJmZlZyXKSMjOzkvX/AbawIRe6AqQgAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNW5//HPw66RZRAFYWQgIorghqigGEdNVHBBNCKI\nG/ozuHBNSHKvGhfwRmM0anKNWzReRVzirqBEURE1bqMBcUVwYdgEBEQFr7I9vz/q9NDTzPTUDFPd\nDfN9v179mq6qc6qe7pnpp+vUqXPM3RERESlEjfIdgIiISHWUpEREpGApSYmISMFSkhIRkYKlJCUi\nIgVLSUpERApW4knKzI40s5lmNsvMLqymzI1mNtvM3jGzvWqqa2ZFZjbZzD42s2fNrHVY/1Mze9vM\nZpjZW2Z2SFqd3mb2btjXX5J8zSIiUj8STVJm1gi4CTgC6AkMM7NdM8oMAHZy952BkcBtMepeBDzv\n7rsAU4CLw/ovgaPdfU/gDGB82qFuBc5y9+5AdzM7op5froiI1LOkz6T2A2a7e7m7rwH+AQzKKDMI\nuAfA3d8EWptZ+xrqDgLGhefjgONC/Rnuvig8/wBoYWZNzawD0NLd3wp17knVERGRwpV0kuoEzEtb\nnh/WxSmTrW57d18MEJLS9pkHNrOfA9NCgusU6meLQ0RECkyTfAdQBatDnUpjO5lZT+Bq4Gf1EpGI\niORF0klqAdA5bbk4rMsss2MVZZplqbvIzNq7++LQlLckVcjMioHHgFPdfU4Nx9iImWkwQxGRHHP3\nKk9Qkm7uewvoZmYlZtYMGApMyCgzATgNwMz6AitCU162uhOIOkYAnA48Geq3AZ4CLnT3N1IHCE2C\nX5vZfmZm4XhPVhe0u9f4GDNmTKxyuXgoFsWyOcejWBRLNomeSbn7OjMbBUwmSoh3uvtHZjYy2uy3\nu/skMxtoZp8Aq4AR2eqGXV8DPGRmZwLlwJCw/nxgJ+ByMxtD1Ax4uLsvDdvuBloAk9z9mSRfu4iI\nbLrEr0mFZLBLxrq/ZSyPils3rF8O/LSK9VcBV1Wzr38Du8cOXERE8k4jTtRRaWlpvkOooFiqpliq\nV0jxKJaqKZaI1dQe2NCYmes9ERHJHTPDq+k4UYhd0EVkC9SlSxfKy8vzHYbkUUlJCXPmzKlVHZ1J\nZdCZlEgywrflfIcheVTd30C2MyldkxIRkYKlJCUiIgVLSUpERAqWkpSINHhdu3ZlypQp+Q5DqqAk\nJSJ5U1zSBTNL7FFc0iXfL1E2kbqgi0jeLJhbztXTVie2/4t7N0ts35IbOpMSEUmzevVqfvWrX9Gp\nUyeKi4sZPXo0a9asAaKRFx5//HEAXn31VRo1asQ///lPAKZMmcLee+9d5T6///57Tj/9dNq2bUvP\nnj3505/+xI47bpiY4ZprrqFbt260atWKXr168cQTT1RsGzduHP379+fXv/41RUVFdOvWjddff51x\n48bRuXNnOnTowD333FNRfsSIEZx//vkMHDiQli1bctBBB7F48WJGjx5N27Zt2W233ZgxY0asYxcC\nJSkRkTRXXnklZWVlvPvuu8yYMYOysjKuvPJKAA4++GCmTp0KwMsvv8xOO+3Eyy+/DMBLL71U7fBB\nY8eOZe7cucyZM4fnnnuOe++9l2hChki3bt149dVX+eabbxgzZgynnHIKixcvrtheVlbGXnvtxfLl\nyxk2bBhDhw7l7bff5tNPP2X8+PGMGjWK7777rqL8ww8/zB/+8AeWLVtGs2bN6NevH3369GHZsmWc\ncMIJjB49Ovax801JSkQkzf3338+YMWPYdttt2XbbbRkzZgzjx48HoiT10ksvAVGSuvjiiyuWX3rp\nJQ4++OAq9/nwww9zySWX0KpVKzp27MgFF1xQafsJJ5xA+/btATjxxBPZeeedKSsrq9jetWtXTjvt\nNMyMk046ifnz5zNmzBiaNm3Kz372M5o1a8Ynn3xSUX7w4MHstddeNGvWjMGDB7PVVlsxfPjwivrv\nvPNO7GPnm5KUiEiahQsX0rnzhvlWS0pKWLhwIQD9+vVj1qxZLFmyhBkzZnDaaacxb948li1bRllZ\nGT/5yU+q3WdxcXHFcnpTH8A999zD3nvvTVFREUVFRXzwwQcsXbq0YnsqiQBstdVWALRr167SupUr\nV1ZbPnM5vWxNx843Jakc6FLSIVZPpC4lHfIdqkiD17Fjx0pjDJaXl9OxY0cg+oDfZ599+J//+R96\n9epFkyZN6NevHzfccAPdunWjbdu21e5z/vz5Fctz586t9PwXv/gFt9xyC1999RVfffUVPXv2zMkQ\nUvk8dlxKUjlQPncx/hE1PsrnFk47sEhDNWzYMK688kqWLl3K0qVL+f3vf8+pp55asf0nP/kJN910\nU0XTXmlpaaXlqpx44olcffXVrFixggULFnDzzTdXbFu1ahWNGjWiXbt2rF+/nrvuuov3338/a4yb\nmkRS9ety7FyrVZKyyI+SCkZEJB/SOzFceuml9OnThz322IM999yTPn36cMkll1RsP/jgg1m5cmVF\n015qOVuSuvzyy+nUqRNdu3bl8MMP58QTT6R58+YA9OjRg9/85jf07duXDh068MEHH9C/f//Y8Va1\nHPf11uXYuVbjKOhmdg8wClgLlAHbAn9y9xuSDy/3khgF3cyomPg+W7kem/4NSaRQVTUCdnFJFxbM\nTW76jk6dS5hfPiex/dfVbbfdxoMPPsiLL76Y71ByKqlR0Pdw92+A44DngBLgjE2IU0QEgPnlc3D3\nxB6FkqAWLVrEa6+9hrvz8ccfc/3113P88cfnO6zNQpwRJ5qaWRNgEHCru682s/UJxyUissVYvXo1\nI0eOZM6cObRp04Zhw4Zx7rnn5juszUKcJPV3YC7wPvCSmXUGVmavIiIiKZ07d+a9997LdxibpVrP\nzGvRFbem7p7cgFt5pGtSIsnQzLxSl2tS1Z5JmdkF1W0LbqxdeCIiIrWTrblvu/BzZ2A/YGJYPhp4\nEyUpERFJWJwu6C8DR4cefphZK2Ciu1d/U8BmTM19IslQc58k1QW9PfB92vIPgMbvERGRxMVJUvcB\nb5rZpWZ2KfAaMD7ZsERECte5557LVVddVe9lsykvL6dRo0asX5/7O4BGjBjB5ZdfHqts165dmTJl\nSr0du8Yu6O7+32b2TyA1vO857v5WvUUgIg1Wl+JiyhcsSGz/JZ06MSdtYNf6cuuttyZStia1Hf5o\nS5A1SZlZY+Bdd+8JKDGJSL0qX7CAFWPHJrb/Ngnse/369TRqpLG5cyXrO+3u64DPzKxTjuIREcm5\nmTNncsghh1BUVMTuu+/OxIkTK7aNGDGC8847j6OOOoqWLVsyderUjZq/rr32Wjp27EhxcTF33nkn\njRo14rPPPquonyr70ksvseOOO3LDDTfQvn17OnXqxN13312xn0mTJtG7d29at25NSUkJV1xxRezX\n0LVrV6677jr23HNPWrZsydlnn82SJUsYOHAgrVq14vDDD+frr7+uKD9hwgR69epF27ZtOfTQQ5k5\nc2bFtunTp7PPPvvQunVrhg4dyvfff1/pWE899VTFHFT9+/dP9EblOF8HtgE+MrNnzeyx1COxiERE\ncmjt2rUcc8wxHHnkkXz55ZfceOONDB8+nNmzZ1eUeeCBB7jsssv49ttvOfDAAyvVf+aZZ/jLX/7C\nlClT+OSTT5g6dWrWZrlFixbx7bffsnDhQv7+979z/vnnVySPbbbZhvHjx/P111/z9NNPc9tttzFh\nwoTYr+Wxxx7jhRdeYNasWUyYMIGBAwfyxz/+kaVLl7Ju3TpuvDG6c2jWrFmcfPLJ3HjjjXz55ZcM\nGDCAY445hrVr17JmzRoGDx7M6aefzvLlyznxxBN59NFHK44xffp0zjrrLO644w6WL1/OyJEjOfbY\nY1mzZk3sOGsjTpK6EhgMXAvcnPYQEdnsvfHGG6xatYoLL7yQJk2acMghh3D00UfzwAMPVJQZNGgQ\nffv2BaiYYiPl4YcfZsSIEey66660aNGCsTU0MTZr1ozLLruMxo0bM2DAALbZZhs+/vhjIJqrqmfP\nngD06tWLoUOHVkxPH8d//Md/0K5dO3bYYQcOOugg9t9/f/bYY4+KaeSnT58OwEMPPcTRRx/NoYce\nSuPGjfntb3/L999/z2uvvcYbb7zB2rVrueCCC2jcuDEnnHAC++67b8Ux7rjjDs455xz69OmDmXHq\nqafSvHlz3njjjdhx1kaNScrdXwBmAE3DY0ZYJyKy2Vu4cOFG07mXlJSwIK1DR+b2bPV33HHHrPeD\nbbvttpWuaW299dYV07m/+eabHHrooWy//fa0adOGv/3tb7Wayj3utPELFy6kpKSkYpuZUVxczIIF\nC1i4cCGdOlW+wpNetry8nOuvv562bdvStm1bioqKmD9/PgsXLowdZ23UmKTM7ARgGnAqcBrwtpkN\nTiQaEZEc69ixI/Pmzau0bu7cuZU+qLM13+2www4bTQ1f1154w4cP57jjjmPBggWsWLGCkSNHJnID\ndMeOHSkvrzyP17x58+jUqdNGrwcqT3e/4447cskll7B8+XKWL1/OV199xcqVKznppJPqPU6I19x3\nObCvuw9395OB/YGxiUQjIpJj+++/P1tvvTXXXnsta9euZerUqTz11FMMGzYsVv0hQ4Zw1113MXPm\nTL777juuvPLKOseycuVKioqKaNq0KWVlZdx///2VttdXwhoyZAhPP/00L774ImvXruW6666jRYsW\nHHDAAfTr14+mTZvy17/+lbVr1/LYY49RVlZWUffss8/mtttuq1i3atUqJk2axKpVq+oltkxxklQj\nd1+ctrwkZj0RkYLXtGlTJk6cyKRJk2jXrh2jRo1i/Pjx7LzzzkDVZ1Hp64488kguuOACDjnkELp3\n706/fv2Aja9dVSd9X7fccguXXXYZrVu35sorr9zo7CTbGVptppTv3r079957L6NGjWK77bbj6aef\nZuLEiTRp0oSmTZvy2GOPcdddd7Htttvy8MMPc8IJJ1TU3WeffbjjjjsYNWoUbdu2pXv37owbNy7W\ncesizth91wO7AqmriEOBme7+23qNpEBo7D6RZFQ1btvmejNvNjNnzmT33Xfnhx9+0P1UGeoydl+c\nJGXAECDV7/IV4JF6/yQvEEpSIsnYkgeYfeKJJxg4cCCrVq3ijDPOoEmTJpW6bUukXpOUmY0iGqfv\nHXdvMNPFK0mJJGNLTlIDBgzg9ddfp0mTJpSWlnLzzTdX6lknkfpOUn8BDgC6AdOBV4mS1uvu/nWV\nlbYASlIiydiSk5TEk1RzXwuiSQ8PAPoR9e5b4u57bHLEBUhJSiQZSlJSr9PHZ5RpBjQPj4XAB5sQ\np4iISCzVJikzuwXYA/iOaAT0N4Bb3P3LHMUmIiINXLb+kd2BFsBc4FPgEyUoERHJpazXpMysEdHZ\n1AHh0YPoZt7X3P33OYkwx3RNSiQZuiYldbkmVdN8Uuvd/R3gUeBxYCrRGdZ/bXK0IiIFItuU5//6\n17/o0aNHrP2k5ouS+lNtkjKz88zsXjP7HCgDfg7MIbqxt01uwhORLVmXkg6YWWKPLiUdNjnG/v37\n89FHMZpCgoY4xXuSsvXu2xWYCFzs7vOylBMRqZPyuYtjNYXXlfVYXHMhKWjVnkm5+wXu/qASlIg0\nBNOnT2fPPfekqKiIYcOGsXr1amDjJrxp06ZVTPE+ZMgQhg4dWmkqeXevdnr4THPmzOHggw+mdevW\nHH744YwaNYpTTz21YvuQIUPYYYcdKCoqorS0lA8//LBi24gRIzj//PMZOHAgLVu25KCDDmLx4sWM\nHj2atm3bsttuuzFjxoyK8rWdXj7bsXNJox+KiBDNsDt58mQ+//xzZsyYUSm5pJrw1qxZw/HHH8+Z\nZ57J8uXLGTZsGI8//nil/WSbHj7TySefTN++fVm2bBljxoxh/PjxlZoLBw4cyKeffsqSJUvo3bs3\nw4cP3yjmP/zhDyxbtoxmzZrRr18/+vTpw7JlyzjhhBMYPXp0pfJxp5ePc+xcSTxJmdmRZjbTzGaZ\n2YXVlLnRzGab2TtmtldNdc2syMwmm9nHZvasmbUO69ua2RQz+9bMbsw4xothX9PNbJqZtUvqNYvI\n5ueXv/wl7du3p02bNhxzzDG88847G5V5/fXXWbduHaNGjaJx48YMHjyY/fbbr1KZbNPDp5s3bx5v\nv/02V1xxBU2aNOHAAw/k2GOPrVTmjDPOYOutt6Zp06ZcfvnlzJgxg2+//bZi++DBg9lrr70qpoff\naqutGD58OGbGSSedtNFriDu9fJxj50qsJGVmrcysVW13Hrqw3wQcAfQEhpnZrhllBgA7ufvOwEjg\nthh1LwKed/ddgCnAxWH998ClwG+qCWmYu+/t7r3dPf6czCKyxUsfEDZ9Svd0X3zxxUZTq2f25ss2\nPXy6hQsX0rZtW1q0aFHlvtavX89FF11Et27daNOmDV27dsXMKk0nH3e6+NqWj3PsXMnWu6849O77\nEpgBvGtmS8K6zjH3vx8w293L3X0N8A9gUEaZQcA9AO7+JtDazNrXUHcQkJplaxxwXKj/nbu/BvxQ\n29crIlKTHXbYgQUZ819lTj1fm30tX76c77//vsp93XfffUycOJEpU6awYsUK5syZg7vn5F6zfB47\nU7YP7QeBfwId3b2ru3cBOgHPECWMODoB6b/B+WFdnDLZ6rZPzRbs7ouA7WPGc3do6rs0ZnkRkQr9\n+vWjcePG3Hzzzaxbt44nn3yy0tTqtdG5c2f69OnD2LFjWbNmDa+//joTJ06s2L5y5UqaN29OUVER\nq1at4uKLL6519/a6JpX6OHZ9yZaktnf3+8JZDADuvsbd7wW2SzCmurwTcX4TJ7v77sBBwEFmdkod\njiMiW6C4H8CpqdX//ve/U1RUxP33388xxxyTdar4bPu+7777eO2112jXrh2XX345Q4cOrdjXaaed\nRufOnenUqRO9evXigAMOqN2Lyjh2baaXr49j15ds80k9DHxB1JyWOqPZETgD2MHdf17jzs36AmPd\n/ciwfBHg7n5NWpnbgBfd/cGwPBM4GOhaXV0z+wgodffFZtYh1O+Rts/TgX3c/YJq4qp2u5n5mDFj\nKpZLS0spLS2t6aXW9D5oWCRp8KoaEqdLSQfK5yZ3L1NJ5/bMKV+U2P4B+vbty7nnnsvpp5++yfsa\nOnQoPXr0IP0zaEuS+huYOnUqU6dOrVh/xRVX1GnSw+bAL4iu/6Sa2RYAE4Db3f37KitW3kdj4GPg\nMKKEV0bUeeGjtDIDgfPd/aiQ1P7i7n2z1TWza4DlIWFdCBS5+0Vp+zwd6OPu/5EWRxt3X2ZmTYH7\ngefc/fYqYtbYfSIJ2FLG7nv55ZfZZZddaNeuHffeey/nnXcen332WZ1m4n377bdp27YtXbt25dln\nn+X444/n9ddfZ88990wg8vyr1/mk3P0H4K/hUSfuvs6iaegnEzUt3hmSzMhos9/u7pPMbKCZfQKs\nAkZkqxt2fQ3wkJmdCZQTDdWUerGfAy2BZmY2CDicaCT3Z82sCdAYeB64o66vS0Qaro8//pghQ4bw\n3Xff8eMf/5hHH320zlPFL1q0iOOPP57ly5dTXFzMbbfdtsUmqLqqaRT0w4h6zqWfST3p7s/nILa8\n0JmUSDK2lDMpqbt6PZMys+uBXsB4op51AMXAf5rZQHf/9aaHLCIiUr1s16RmuXv3KtYbMCvcfLvF\n0ZmUSDJ0JiV1OZPK1gX9BzPrXcX63lR/s6yIiEi9yTZVx5nA7aGXX6oLemeioYfOTDowERGRbL37\n3gL6mFkxaR0n3H1+dXVERKpTUlKiCQEbuJKSklrXyXYmldI+PADWsqEThYhIbHPmzMl3CLIZyta7\n7zDgVqL7kFIjKhaHwWXPdfcXchCfiIg0YNnOpP4KHOnun6WvNLOdgKeAHlXWEhERqSfZevc1JTqL\nyjQ3bBMREUlUtjOpccCbZvYAlQeYHQbcnXBcIiIiNQ6LtDtVDDDr7u/mILa80M28IiK5VadhkQDc\n/T3gvUSiEhERqUGdplM3s4k1lxIREdk02bqg71HdJqBPMuGIiIhskK25bzrwKlVP594mmXBEREQ2\nyJakZgJnuvsnmRvMbF4V5UVEROpVtmtSV1B9EhudQCwiIiKVZO2C3hCpC7qISG7VdT4pERGRvFKS\nEhGRgqUkJSIiBavG+aTMrBFwJNAlvby735hcWCIiIvEmPXwScKLhkdYnG46IiMgGcZJUF3ffPfFI\nREREMsS5JvWsmR2aeCQiIiIZ4iSpV4CJZrbSzJab2VdmtjzpwCQZXUo6YGY1PrqUdMh3qCIiNd/M\na2afAyeQcU3K3dclG1p+bOk38xZSLCIisAnzSQXzgen1/sktIiJSgzhJ6hNgiplNAn5IrVQXdBER\nSVrcM6n5QKuEYxEREamkxiTl7pcBmNlWYfn/kg5KREQEYvTuM7PdzOwtYDYw28zeNLMeyYcmIiIN\nXZwu6LcDv3P3YncvBi4B7kg2LBERkXhJqqW7P5dacPfngZbJhSQiIhKJk6TmmNnFZlYcHhcBcxKO\nS0REJFaSOhPYEZgEPA0UAyOSDEoaBo1+ISI1idMF/WB3Py99hZkdDzyWTEjSUJTPXRxz9IvFyQcj\nIgUpzpnUpVWsu6S+AxHJJ53ViRSmas+kzOwIoskOO5nZDWmbWqF5pWQLo7M6kcKUrblvCfA+8D3w\nQdr6b4GLkgxKREQEsiQpd58OTDezH7n7nenbzGwUcFPSwYmISMMW55rUGVWsO6ue4xAREdlItmtS\nJwFDga5mlt6TrxWwIunAREREsl2TKgOWEd0XdXPa+m+B6UkGJSIiAtmvSX0OfA48b2btgD5h02fu\nviYXwYmISMMWZxT044FpwKnAacDbZjY46cBERETijDgxBtjX3RcDmFl7YDLweJKBiYiIxOnd1yiV\noIIlMeuJiIhskjhnUpPN7GnggbA8FHg2uZBEREQicZLUb4ETgf5heRzwSGIRbSa6FBdTvmBBvsMQ\nEdmi1Zik3N2Bh4CHzKyNu9fqHikzOxL4C1ET4Z3ufk0VZW4EBgCrgDPc/Z1sdc2sCHgQKCGa22qI\nu39tZm2JEui+wF3ufkHaMXoDdwMtgEnu/qvavI5M5QsWsGLs2Fhl28QsJyIilVV7bcnM9jOz583s\nITPb08zeBT4xs8VmdnicnZtZI6Lhk44AegLDzGzXjDIDgJ3cfWdgJHBbjLoXAc+7+y7AFODisP57\nolHbf1NFOLcCZ7l7d6B7GEB3i9CluDjWCN5mlu9QRURqJduZ1M1EPftaAy8Cx7j7q2bWExhP1MOv\nJvsBs929HMDM/gEMAmamlRkE3APg7m+aWevQg7BrlrqDgIND/XHAVOAid/8OeM3Mdk4Pwsw6AC3d\n/a2w6h7gOLaQa2uFdFanZlARqU/ZklQTd58EYGaXu/urAO7+gcX/St4JmJe2PJ8ocdVUplMNddun\nehy6+yIz2z5GHPOrOIbUs0JKmCKy+cuWpDzt+f9l2Vbf6tImVa/xjE378CwtLaW0tLQ+dy8i0qBN\nnTqVqVOnxiqbLUntaWbLiZJGy/CcsLxNzFgWAJ3TlovDuswyO1ZRplmWuovMrL27Lw5NeUtixFHV\nMao0Vt/wRUQSk/nl/4orrqi2bLabcpsB2wHtgObheWq5RcxY3gK6mVmJmTUjusdqQkaZCUTDLWFm\nfYEVoSkvW90JbJhC5HTgySqOXXFG5u6LgK9DZxALx6uqjmxB1KFEZPOXbYDZdZu6c3dfFyZInMyG\nbuQfmdnIaLPf7u6TzGygmX1C1AV9RLa6YdfXEHWJPxMoB4akjmlmnwMtgWZmNgg43N1nAudTuQv6\nM5v6+qSw6fqYyOYvzs28myQkg10y1v0tY3lU3Lph/XLgp9XU6VrN+n8Du8eLWkRECkG2+6QST2Ai\nIiLZZLsmVQZgZnfnJhQREZHKsp0tNTOzIcBBZnZs5kZ3z+wAISIiUq+yJanzgVOANkQDzKZzNu6l\nJyIiUq+y9e57CXjJzN7O7OggIiKSC3EmL/xfMzvPzP4RHueqU4VI7dTmnq0uxcX5DlekYMRJNjcB\nPwL+NyyfAuwN/CKpoES2NLpnS6Ru4iSpvu6+Z9ryZDObkVRAIiIiKXGa+9abWZfUQni+PplwRERE\nNoiTpC4EXgkTIL4AvAT8Z7JhiUiS4l4j0/Uxybc408dPNrPuQI+w6iN3z5y6Q0Q2I3GvkeXi+ljc\niTJLOnVizvz5NZaTLUusXnohKU1LOBYRaYAKKWFK4YnT3CciIpIXSlIiIoGu1RWeGpv7zOxBonuk\nJrt7ktPGiwjQvBmxJ2Is6dyeOeWLEo6o4VDTY+GJc03qLuBM4KaQsO5290+SDUuk4fphNVRM71kD\n67E40ViUMCXf4vTuewZ4xsyKgOHAi2H22zuAB9x9bcIxikieFFLClIYp1jWpkKBOBk4F3gX+BhwA\naAp2ERFJTJxrUg8TTbt+H3CCu6duVLjPzKYnGZyIiDRsca5J3Q48n95pwsyauPtad987udBERKSh\ni9Pcd00VvfrKkghGREQi6g4fqfZMysy2B3YAtjKz3YFUF59WwNY5iE1EpMFSd/hItua+o4i6nhcD\nt6St/xa4LMmgREREIPv08XcBd5nZEHd/KIcxiYiIANmb+4a5+wPADmZ2QeZ2d78x0chERKTBy9Zx\noij8bAdsV8VDRCRnUqNfxOpMUNKhwcSypcvW3HdL+KnrTyKSd4U0+kUhxbKly9bcd0O2iu7+6/oP\nR0REamNLH18xW+++D3IWhYiI1MmWflaXrbnvzlwGIiIihalLcTHlCxbUWK6kUyfmzJ9fY7nayNbc\nd727/8bMHgc2mkfK3Y+v10hERKQg5fPG4mzNfQ+GnzfV+1FFRERiyNbcVxZ+vmBmTYGdic6oZmsO\nKRERyYU4U3UcSTQS+lyi8fuKzexsd5+cdHAiItKwxZmq4y/AT919FoCZdQeeBHokGZiIiEicqTpW\nphIUQHi+KrmQREREItl69x0bnpaZ2QTgIaJrUicCb+YgNhERaeCyNfedmPb8a+CI8PxboGViEYmI\niATZevdkHn/QAAAUxUlEQVSdmstAREREMsXp3dccOAPoCbRIrXf3XyQXloiISLyOE/cAXYCjia5F\n7QR8n2BMIiKyGUpiCpM4XdC7u/tJZnaUu99pZvcAr2zSKxERkS1OEoPdxjmTWhN+rjCzHkSdJraP\nF4aIiEjdxTmTutPMioAxwLPA1sDliUYlIiJCjCTl7n8LT18EOicbjoiIyAY1NveZWZGZ/dnMyszs\nTTO7LpxZiYiIJCrONal/AN8Aw4FTiG7mfTBrDRERkXoQ55pUJ3cfk7Z8hZm9n1RAIiIiKXHOpF4w\ns5+nFszseOC55EISERGJVJukzOwrM1sOnAY8ZGarzWw18AhwetwDmNmRZjbTzGaZ2YXVlLnRzGab\n2TtmtldNdcN1sslm9rGZPWtmrdO2XRz29ZGZHZ62/sWwr+lmNs3M2sV9DSIikh/ZzqTaAduFn02B\nrcKjaVhfIzNrRDT9/BFEwyoNM7NdM8oMAHZy952BkcBtMepeBDzv7rsAU4CLQ53dgCFEc10NAG4x\nM0s73DB339vde7v70jivQURE8qfaJOXu61IPokRxVXj8LKyLYz+i6ebL3X0NUSeMQRllBhENvYS7\nvwm0NrP2NdQdBIwLz8cBx4XnxwL/cPe17j4HmB32U+PrFRGRwhOnC/pVwH8Bn4XHf5nZlTH33wmY\nl7Y8P6yLUyZb3fbuvhjA3RexYQSMzDoLMo53d2jquzRm/CIikkdxevcdA+ydOnsys/8FpgFJfdBb\nzUU24jHKnOzuX5jZj4DHzOwUd7+3DscSEZEciZOkAFoBX4XntZnwcAGVR6koDusyy+xYRZlmWeou\nMrP27r7YzDoAS2rYF+7+Rfi5yszuJ2oGrDJJjR07tuJ5aWkppaWl2V6jiIjUwtSy6Gf6Z2114iSp\na4FpZvYC0VlOKXBZzFjeArqZWQnwBTAUGJZRZgJwPvCgmfUFVoTkszRL3QlEc1xdQ9TT8Mm09feZ\n2Z+Jmvm6AWVm1hho4+7LzKwp0bQj1Xajj/PGiYhI3ZSGngKpz9orrrii2rJZk1ToGfcC0bh9+4fV\nl7t75tlQldx9nZmNAiYTXf+6090/MrOR0Wa/3d0nmdlAM/sEWAWMyFY37Poaom7xZwLlRD36cPcP\nzewh4EOi0dvPc3cPEzc+a2ZNgMbA88AdcV6DiIjkT9YkFT7gn3P3XsBjdTmAuz8D7JKx7m8Zy6Pi\n1g3rlwM/rabO1cDVGeu+A/rUKnAREcm7OF2y3zGzvROPREREJEOca1J7A2+Z2adEzXFGdJLVO9HI\nRESkwYuTpI5NPAoREZEqVJukQmeDs4l6yL0H3F2LkSZEREQ2WbZrUncD/YmGFjoOuC4XAYmIiKRk\na+7r5e67A5jZ7cCbuQlJREQkku1Mak3qSRjgVUREJKeynUntGeaTgqhHX8uwnOrd1zbx6EREpEHL\nlqSa5SwKERGRKlSbpNSTT0RE8k2TAIqISMFSkhIRkYKlJCUiIgUr24gTX1H1jLfq3SciIjmRrXdf\nu5xFISIiUoXYvfvMrC3QIm3VwqSCEhERgRjXpMzsKDObBcwnGhppPjAl6cBERETidJy4CjgQ+Njd\ndwSOAF5JNCoRERHiJam17v4l0MjMzN2fA/ZLOC4REZFYkx5+bWbbAP8C7jGzJcD/JRuWiIhIvDOp\n44iS0q+AqcAC4OgEYxIREQHiJamL3X2du69x9zvd/Qbg10kHJiIiEidJHVnFuqPqOxAREZFM2Uac\nGAmcA3Q3s2lpm1oC/046MBERkWwdJx4CXgCuBi5KW/+tuy9JNCoRERGyjzjxFfAVcKKZ9QQOCpte\nAZSkREQkcXFGnDgfeBjoHB4Pmdl5SQcmIiIS5z6pkcB+7r4SwMz+ALwG3JJkYCIiInF69xmwOm15\nTVgnIiKSqGy9+5q4+1pgPPCmmT0aNg0GxuUiOBERadiynUmVAbj7tURNft+Fxznufl0OYssbM6vx\nkQvFJV0KJhYRkXzIdk2q4tPP3csISashuHra6hrLXNy7WeJxLJhbXjCxiIjkQ7YktZ2ZVTv8URge\nSRqI4pIuLJhbnu8wRKSByZakGgPboE4SQmGd1RVSwlQsVSukWGTzli1JfeHu/52zSERiKqSEuTnG\nAsnHU0ixKGFu3mJdkxIR2VwpYVatkGLJJluSOixnUYiINACFlDALKZZsqu2C7u7LcxmIiIhIpjgj\nToiIiOSFkpSIiBQsJSkRESlYSlIiIlKwlKRERKRgKUmJiEjBUpISEZGCpSQlIiIFS0lKREQKlpKU\niIgULCUpEREpWEpSIiJSsBJPUmZ2pJnNNLNZZnZhNWVuNLPZZvaOme1VU10zKzKzyWb2sZk9a2at\n07ZdHPb1kZkdnra+t5m9G/b1l6Rer4iI1J9Ek5SZNQJuAo4AegLDzGzXjDIDgJ3cfWdgJHBbjLoX\nAc+7+y7AFODiUGc3YAjQAxgA3GJmqXmxbgXOcvfuQHczOyKZVy0iIvUl6TOp/YDZ7l7u7muAfwCD\nMsoMAu4BcPc3gdZm1r6GuoOAceH5OOC48PxY4B/uvtbd5wCzgf3MrAPQ0t3fCuXuSasjIiIFKukk\n1QmYl7Y8P6yLUyZb3fbuvhjA3RcB21ezrwVp+5pfQxwiIlJgCrHjRF2mrfd6j0JERPLP3RN7AH2B\nZ9KWLwIuzChzG3BS2vJMoH22usBHRGdTAB2Aj6raP/AMsH96mbB+KHBrNTG7HnrooYceuX1Ul0ea\nkKy3gG5mVgJ8QZQchmWUmQCcDzxoZn2BFe6+2MyWZqk7ATgDuAY4HXgybf19ZvZnoua8bkCZu7uZ\nfW1m+4WYTgNurCpgd6/LmZyIiCQg0STl7uvMbBQwmahp8U53/8jMRkab/XZ3n2RmA83sE2AVMCJb\n3bDra4CHzOxMoJyoRx/u/qGZPQR8CKwBzvNwekSUCO8GWgCT3P2ZJF+7iIhsOtvwGS4iIlJYCrHj\nRMGLc4NyjuK408wWm9m7+YohLZZiM5tiZh+Y2XtmdkEeY2luZm+a2fQQy5h8xZIWUyMzm2ZmE/Ic\nxxwzmxHem7I8x9LazB4ON95/YGb75ymO7uH9mBZ+fp3nv9/RZvZ+GHzgPjNrlsdYfhn+h/L2P60z\nqVoKNxnPAg4DFhJd4xrq7jPzEEt/YCVwj7vvkevjZ8TSAejg7u+Y2TbAv4FB+XhfQjxbu/t3ZtYY\neBW4wN3z9qFsZqOBfYBW7n5sHuP4DNjH3b/KVwxpsdwNvOTud5lZE2Brd/8mzzE1IrpFZX93n1dT\n+QSO3xH4F7Cru682sweBp939njzE0hN4ANgXWAv8EzjH3T/LZRw6k6q9ODco54S7/wvI+4cNgLsv\ncvd3wvOVRD0w83Yvmrt/F542J7r2mrdvY2ZWDAwE/p6vGNIYBfB/b2atgIPc/S6AcAN+XhNU8FPg\n03wkqDSNgR+lEjfRl+F86AG86e4/uPs64GXg+FwHkfc/1s1QnBuUGzQz6wLsBbyZxxgamdl0YBHw\nXNpoI/nwZ+A/yWOiTOPAc2b2lpmdncc4ugJLzeyu0Mx2u5ltlcd4Uk4iOnvIC3dfCFwPzCUajGCF\nuz+fp3DeBw4KY6VuTfRFa8dcB6EkJfUqNPU9AvwynFHlhbuvd/e9gWJg/zCuY86Z2VHA4nCWadTt\nZvX6dKC79yb6wDk/NBnnQxOgN3BziOc7ovsc88bMmhINrfZwHmNoQ9QyUwJ0BLYxs5PzEUtoqr8G\neA6YBEwH1uU6DiWp2lsAdE5bLg7rGrzQPPEIMN7dn6ypfC6EJqQXgSPzFMKBwLHhWtADwCFmlvPr\nCynu/kX4+SXwOFHzdT7MB+a5+9th+RGipJVPA4B/h/cmX34KfObuy0MT22PAAfkKxt3vcvc+7l4K\nrCC6Hp9TSlK1V3GDcuh1M5ToJuJ8KYRv5yn/C3zo7v+TzyDMrF1q+pbQhPQzopFMcs7df+fund39\nx0R/K1Pc/bR8xGJmW4czXczsR8DhRE06ORfG3pxnZt3DqsOI7m/Mp2HksakvmAv0NbMWYQaHw4iu\n7+aFmW0XfnYGBgP35zqGpEec2OLUcJNxTpnZ/UApsK2ZzQXGpC5E5yGWA4HhwHvhWpADv8vTTdM7\nAONCT61GwIPuPikPcRSa9sDjZuZE//v3ufvkPMZzAdEIMU2Bzwg38udDuObyU+AX+YoBwN3LzOwR\noqa1NeHn7XkM6VEza8uGwRFy3rlFXdBFRKRgqblPREQKlpKUiIgULCUpEREpWEpSIiJSsJSkRESk\nYClJiYhIwVKSakDMrG3alARfmNn8tOVa3TMXpgnZeRPj2drMXtyUfaTta3RtpzQws8PM7PEq1p9l\n0ezOORXnPTWz8Wa20SjqZtbVzE6qwzFvCNMw/CFj/e9rMzVD5vHNbG8zO6K28cQ81pAwtcc6M9sj\nY9ulZjbbzD40s8PS1u8bXucsM7s+bX1zi6YLmW1mr4bBgHOiur+/tO3tzezpXMVTqJSkGpAw1Mre\nYay0W4EbUsvuvraW+zrL3WdvYkj/D3hoE/dBmI7j10SzLtdWdTcK5vwGwk18T3ciGtEitjCiwQh3\n393df1fH41Z3/N4kNxTVu0Tj272avtLMdgeOA3YFjib6G0+5FTjd3bsDvdIS2C+AL9x9Z+AW4I8J\nxZyaBiRTtX9nYVSOpWa2b1IxbQ6UpBquSkMpmdl/hW+a74YRNTCznSyafO2B8M30H2bWPGx7JfUt\n1syOMrN/h7OyZ8K6Q83snXCW9nY1I1wPB54M5TuGfU4LMfQN608Jy++a2VVhXWMz+8rM/mxm7xCN\nML498IqZTQ5lBpjZa+HYD6SOH2KdaWZvk32KlS5mNtXMPjaz34W6V5nZ+Wnv2R/N7NyM9/EiMzsn\nPP+rmT0bnv/MovmTUpNmVhVb+ns6Mhz7dTO7w8xuSDvMoeFb/ydmlnoNVwOl4f0blRGTmdn14fc7\nw8xS0y08RTSA6bS0del6h+N/bGYjathX+vEvAC4HTk7t28y2NbMnQ51/WRjw16IztrvCa//czAaZ\n2XVh/xOr+mB395nu/gkbDwc2CHjA3deFOY/KzWyfcHbU3N2nhXLjiZJZqs648PwhYKOzvxp+p3eF\n53H+Tvet7u8vy//Lk8ApVfxuGg5316MBPoAxwK/D8/2Ihl9pBmxDNIZaT6Jvx+uBfUO5cUSTBwK8\nAuxBNNROOVAc1rcJPyel1duaMLpJ2vGbA/PTlv8L+M/w3EKdTsDnQBHRHDtTiUbvbhziGpRWfy7Q\nMjzfLpRtEZZ/RzTC9lZE06x0CesfAR6r4r05K5RrFeL4ILzWnYCyUKYR8CnQOqPugUTDDUE0ed0b\n4fX8N9GwP1XGlvGeFhMNE9SKaPiiV4nOeiH6gE3tf3fgo/D8sKpeS9g2hGjiPMLvay7QLryPy6up\n83vgbaBpiHle+FndviodP7yHN6Qt3wJcHJ7/DHgr7TgvhveoN7AKODRsmwAMzPI3/AqwR9ryrcCQ\ntOW7iUY13x+YlLa+NBUr0bh426dt+5xoYsra/E5j/Z2S5e+Pav5fiAaznpbvz4t8PnQmJQD9gUfd\nfbVH02s8ARwUtn3mG+ZiujeUTdePaNDU+QDuviKsfxW4MXyrb+3hPy7N9sDytOW3gP9nZpcBu3s0\naeH+wAvu/pVHI0LfD/wklP/BK4+0nj7Q7gHAbsBrFo0jeDLQJaz72N3nhHL3ZXlPnnX3b0IcTwD9\n3f1T4BuLZiwdQDQh3NcZ9d4i+sbcmmjW5LeIZuQ9iOhDtarYSjL2kXrd33jUDPtIxvYnANz9PaLp\nHGrSnzBwqkdNSK8AfcK2bIMTP+HuazwaFfwloi8z2fZVUwzjQ73ngB3SzhYmhb+P96LNPiWsf4/o\n95ZLVb0fNf1O4/6dZvv7q+7/ZQnRWJQNlpKU1FZVbegb/WO7+1XA2URnZm+Y2U4ZRf6PtGtI7v4i\n0TfcL4gGhx1W3b7T6lfHgH96dK1tb3fv5e7n1LC/jV5CNct3En17HkE06nvlQu6riWZSPY3oW/cr\nRGcZnX1DE1VmbOdm7qeGOH+IWa466XWyXXtL32ZEZwXZ9pVNtuOkXs96YHXa+vXUbhDsBVSelC81\njU626XUq6lg00O1GU9jH+J1C/L/TKstl+X9pUcU+GhQlKYHon26wRT2dtiFqK38lbOtqZvuE5yen\nrU95jehaRGcAMysKP3/s7u+7+x+BacAu6ZXcfSmwlYVehaH+Ynf/O1Ezzd5EM/uWWjQzaBOiC/NT\nwy4y/9m/IWoeS8V0sJl1Dfve2sy6ETVjpqZZMaKpGapzuJm1smh07PSL9I8BxwB7evUzpr4C/JZo\nuu1/AecTNZ1liy1dWXjdrcIHZ7Ypu1Pvw7dAyyzxDA3Xk9oTnc2l4smWZI4zs6YWTdfQP9Spbl/f\nsuH9p4rlVwjXVszsp8ACd6/qw7e2STe9/ARgWIh5J6Ik8u9wlv99uD5lwKmEa6Ghzhnh+UlEsxtU\nJdvvNO7fabV/f1n+X7qTp+lUCoWSlBCa8x4g+qd7jWi21A/C5o+AX5vZh0Rt6nekqoW6S4BzgSdD\n89W9Yftvw8Xvd4g+sKr653+eDRO6HQbMMLNpRPPW/NXdFwCXETU1TQNe8w1Tf2R+M78DeN7MJoeY\n/h/wYDj+q8DO4UPxXOAZokSwMMvb8hbRB9h0ousR74bX+wPRB1W2eYdeIWrOfMOj6cBXhzqp9+us\nzNjSX5O7zwP+FGJ4meja19fpZdKklqcDTSzqvDIqo8wjRPNpvUv0exgdviRUtb9077PhQ/ny0OxX\n3b6mA43Tjj8F2NOiDjXHE3Wk6GdmM4CxbEgMmWrsVWlmPzezeUTNjM+Y2USA8Dt6guhv9imi33XK\neUTXVGcBH6R9wbidqOlxdihTXS/HbL/TWH+n4e/vHKr++6vu/+UQoEF3Q9dUHVKt8G30EY+mYU9i\n/32Ac939rCT2n4TQ22w60cXwOQke50fuvip8M38SuMXdG/SHVUNkZi8DR7n7t/mOJV90JiU1Sexb\njEdTh/8rqf3XNzPrBXxCdKF/TsKH+304q5wBzFSCanjMbHvg2oacoEBnUiIiUsB0JiUiIgVLSUpE\nRAqWkpSIiBQsJSkRESlYSlIiIlKwlKRERKRg/X9CuYVdWIuo+QAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "a_top = np.sort([sum(tpm_low_gamma.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "b_top = np.sort([sum(topic_model.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "c_top = np.sort([sum(tpm_high_gamma.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "\n",
+ "a_bot = np.sort([sum(tpm_low_gamma.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "b_bot = np.sort([sum(topic_model.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "c_bot = np.sort([sum(tpm_high_gamma.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "\n",
+ "ind = np.arange(len(a))\n",
+ "width = 0.3\n",
+ " \n",
+ "param_bar_plot(a_top, b_top, c_top, ind, width, ylim=0.6, param='gamma',\n",
+ " xlab='Topics (sorted by weight of top 100 words)', \n",
+ " ylab='Total Probability of Top 100 Words')\n",
+ "\n",
+ "param_bar_plot(a_bot, b_bot, c_bot, ind, width, ylim=0.0002, param='gamma',\n",
+ " xlab='Topics (sorted by weight of bottom 1000 words)',\n",
+ " ylab='Total Probability of Bottom 1000 Words')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "From these two plots we can see that the low gamma model results in higher weight placed on the top words and lower weight placed on the bottom words for each topic, while the high gamma model places relatively less weight on the top words and more weight on the bottom words. Thus increasing gamma results in topics that have a smoother distribution of weight across all the words in the vocabulary."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ For each topic of the **low gamma model**, compute the number of words required to make a list with total probability 0.5. What is the average number of words required across all topics? (HINT: use the get_topics() function from GraphLab Create with the _cdf_\\__cutoff_ argument)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 72,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def calculate_avg_words(model, num_words=547462, cdf_cutoff=0.5, num_topics=10):\n",
+ " avg_num_of_words = []\n",
+ " for i in range(num_topics):\n",
+ " avg_num_of_words.append(len(model.get_topics(topic_ids=[i], num_words=547462, cdf_cutoff=.5)))\n",
+ " avg_num_of_words = np.mean(avg_num_of_words)\n",
+ " return avg_num_of_words"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 73,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "252.40000000000001"
+ ]
+ },
+ "execution_count": 73,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "calculate_avg_words(tpm_low_gamma)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ For each topic of the **high gamma model**, compute the number of words required to make a list with total probability 0.5. What is the average number of words required across all topics? (HINT: use the get_topics() function from GraphLab Create with the _cdf_\\__cutoff_ argument)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 74,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "576.20000000000005"
+ ]
+ },
+ "execution_count": 74,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "calculate_avg_words(tpm_high_gamma)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We have now seen how the hyperparameters alpha and gamma influence the characteristics of our LDA topic model, but we haven't said anything about what settings of alpha or gamma are best. We know that these parameters are responsible for controlling the smoothness of the topic distributions for documents and word distributions for topics, but there's no simple conversion between smoothness of these distributions and quality of the topic model. In reality, there is no universally \"best\" choice for these parameters. Instead, finding a good topic model requires that we be able to both explore the output (as we did by looking at the topics and checking some topic predictions for documents) and understand the impact of hyperparameter settings (as we have in this section)."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 2",
+ "language": "python",
+ "name": "python2"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 2
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython2",
+ "version": "2.7.12"
+ },
+ "toc": {
+ "toc_cell": false,
+ "toc_number_sections": false,
+ "toc_threshold": "8",
+ "toc_window_display": false
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/.ipynb_checkpoints/quiz-week5-assignment-checkpoint.ipynb b/machine_learning/4_clustering_and_retrieval/assigment/week5/.ipynb_checkpoints/quiz-week5-assignment-checkpoint.ipynb
new file mode 100644
index 0000000..1bead40
--- /dev/null
+++ b/machine_learning/4_clustering_and_retrieval/assigment/week5/.ipynb_checkpoints/quiz-week5-assignment-checkpoint.ipynb
@@ -0,0 +1,194 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Modeling text topics with Latent Dirichlet Allocation"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 1\n",
+ "\n",
+ "![](\"images/Screen)
\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 2\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 3\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 4\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 5\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 6\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 7\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 8\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 9\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 10\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 11\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 12\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 2",
+ "language": "python",
+ "name": "python2"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 2
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython2",
+ "version": "2.7.12"
+ },
+ "toc": {
+ "toc_cell": false,
+ "toc_number_sections": false,
+ "toc_threshold": "8",
+ "toc_window_display": false
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/.ipynb_checkpoints/quiz-week5-assignment1-checkpoint.ipynb b/machine_learning/4_clustering_and_retrieval/assigment/week5/.ipynb_checkpoints/quiz-week5-assignment1-checkpoint.ipynb
deleted file mode 100644
index 3aa5bc4..0000000
--- a/machine_learning/4_clustering_and_retrieval/assigment/week5/.ipynb_checkpoints/quiz-week5-assignment1-checkpoint.ipynb
+++ /dev/null
@@ -1,214 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Exploring Ensemble Methods"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": true
- },
- "source": [
- "# Question 1"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic01.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 2"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic02.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 3"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic03.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 4"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic04.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 5"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic05.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 6"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic06.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 7"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic07.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 8"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic08.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 9"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic09.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Question 10"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "![](\"images/lec5_quiz01_pic10.png\")
\n",
- "\n",
- "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-classification/exam/FgzAt/exploring-ensemble-methods)*\n",
- "\n",
- ""
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 2",
- "language": "python",
- "name": "python2"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 2
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.10"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/5_lda_blank.ipynb b/machine_learning/4_clustering_and_retrieval/assigment/week5/5_lda_blank.ipynb
new file mode 100644
index 0000000..2387075
--- /dev/null
+++ b/machine_learning/4_clustering_and_retrieval/assigment/week5/5_lda_blank.ipynb
@@ -0,0 +1,723 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Latent Dirichlet Allocation for Text Data\n",
+ "\n",
+ "In this assignment you will\n",
+ "\n",
+ "* apply standard preprocessing techniques on Wikipedia text data\n",
+ "* use GraphLab Create to fit a Latent Dirichlet allocation (LDA) model\n",
+ "* explore and interpret the results, including topic keywords and topic assignments for documents\n",
+ "\n",
+ "Recall that a major feature distinguishing the LDA model from our previously explored methods is the notion of *mixed membership*. Throughout the course so far, our models have assumed that each data point belongs to a single cluster. k-means determines membership simply by shortest distance to the cluster center, and Gaussian mixture models suppose that each data point is drawn from one of their component mixture distributions. In many cases, though, it is more realistic to think of data as genuinely belonging to more than one cluster or category - for example, if we have a model for text data that includes both \"Politics\" and \"World News\" categories, then an article about a recent meeting of the United Nations should have membership in both categories rather than being forced into just one.\n",
+ "\n",
+ "With this in mind, we will use GraphLab Create tools to fit an LDA model to a corpus of Wikipedia articles and examine the results to analyze the impact of a mixed membership approach. In particular, we want to identify the topics discovered by the model in terms of their most important words, and we want to use the model to predict the topic membership distribution for a given document. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Note to Amazon EC2 users**: To conserve memory, make sure to stop all the other notebooks before running this notebook."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Text Data Preprocessing\n",
+ "We'll start by importing our familiar Wikipedia dataset.\n",
+ "\n",
+ "The following code block will check if you have the correct version of GraphLab Create. Any version later than 1.8.5 will do. To upgrade, read [this page](https://turi.com/download/upgrade-graphlab-create.html)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import graphlab as gl\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt \n",
+ "\n",
+ "%matplotlib inline\n",
+ "\n",
+ "'''Check GraphLab Create version'''\n",
+ "from distutils.version import StrictVersion\n",
+ "assert (StrictVersion(gl.version) >= StrictVersion('1.8.5')), 'GraphLab Create must be version 1.8.5 or later.'"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# import wiki data\n",
+ "wiki = gl.SFrame('people_wiki.gl/')\n",
+ "wiki"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the original data, each Wikipedia article is represented by a URI, a name, and a string containing the entire text of the article. Recall from the video lectures that LDA requires documents to be represented as a _bag of words_, which ignores word ordering in the document but retains information on how many times each word appears. As we have seen in our previous encounters with text data, words such as 'the', 'a', or 'and' are by far the most frequent, but they appear so commonly in the English language that they tell us almost nothing about how similar or dissimilar two documents might be. \n",
+ "\n",
+ "Therefore, before we train our LDA model, we will preprocess the Wikipedia data in two steps: first, we will create a bag of words representation for each article, and then we will remove the common words that don't help us to distinguish between documents. For both of these tasks we can use pre-implemented tools from GraphLab Create:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "wiki_docs = gl.text_analytics.count_words(wiki['text'])\n",
+ "wiki_docs = wiki_docs.dict_trim_by_keys(gl.text_analytics.stopwords(), exclude=True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Model fitting and interpretation\n",
+ "In the video lectures we saw that Gibbs sampling can be used to perform inference in the LDA model. In this assignment we will use a GraphLab Create method to learn the topic model for our Wikipedia data, and our main emphasis will be on interpreting the results. We'll begin by creating the topic model using create() from GraphLab Create's topic_model module.\n",
+ "\n",
+ "Note: This may take several minutes to run."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "topic_model = gl.topic_model.create(wiki_docs, num_topics=10, num_iterations=200)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "GraphLab provides a useful summary of the model we have fitted, including the hyperparameter settings for alpha, gamma (note that GraphLab Create calls this parameter beta), and K (the number of topics); the structure of the output data; and some useful methods for understanding the results."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "topic_model"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It is certainly useful to have pre-implemented methods available for LDA, but as with our previous methods for clustering and retrieval, implementing and fitting the model gets us only halfway towards our objective. We now need to analyze the fitted model to understand what it has done with our data and whether it will be useful as a document classification system. This can be a challenging task in itself, particularly when the model that we use is complex. We will begin by outlining a sequence of objectives that will help us understand our model in detail. In particular, we will\n",
+ "\n",
+ "* get the top words in each topic and use these to identify topic themes\n",
+ "* predict topic distributions for some example documents\n",
+ "* compare the quality of LDA \"nearest neighbors\" to the NN output from the first assignment\n",
+ "* understand the role of model hyperparameters alpha and gamma"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Load a fitted topic model\n",
+ "The method used to fit the LDA model is a _randomized algorithm_, which means that it involves steps that are random; in this case, the randomness comes from Gibbs sampling, as discussed in the LDA video lectures. Because of these random steps, the algorithm will be expected to yield slighty different output for different runs on the same data - note that this is different from previously seen algorithms such as k-means or EM, which will always produce the same results given the same input and initialization.\n",
+ "\n",
+ "It is important to understand that variation in the results is a fundamental feature of randomized methods. However, in the context of this assignment this variation makes it difficult to evaluate the correctness of your analysis, so we will load and analyze a pre-trained model. \n",
+ "\n",
+ "We recommend that you spend some time exploring your own fitted topic model and compare our analysis of the pre-trained model to the same analysis applied to the model you trained above."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false,
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "topic_model = gl.load_model('lda_assignment_topic_model')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Identifying topic themes by top words\n",
+ "\n",
+ "We'll start by trying to identify the topics learned by our model with some major themes. As a preliminary check on the results of applying this method, it is reasonable to hope that the model has been able to learn topics that correspond to recognizable categories. In order to do this, we must first recall what exactly a 'topic' is in the context of LDA. \n",
+ "\n",
+ "In the video lectures on LDA we learned that a topic is a probability distribution over words in the vocabulary; that is, each topic assigns a particular probability to every one of the unique words that appears in our data. Different topics will assign different probabilities to the same word: for instance, a topic that ends up describing science and technology articles might place more probability on the word 'university' than a topic that describes sports or politics. Looking at the highest probability words in each topic will thus give us a sense of its major themes. Ideally we would find that each topic is identifiable with some clear theme _and_ that all the topics are relatively distinct.\n",
+ "\n",
+ "We can use the GraphLab Create function get_topics() to view the top words (along with their associated probabilities) from each topic.\n",
+ "\n",
+ "__Quiz Question:__ Identify the top 3 most probable words for the first topic. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__ Quiz Question:__ What is the sum of the probabilities assigned to the top 50 words in the 3rd topic?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's look at the top 10 words for each topic to see if we can identify any themes:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "[x['words'] for x in topic_model.get_topics(output_type='topic_words', num_words=10)]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We propose the following themes for each topic:\n",
+ "\n",
+ "- topic 0: Science and research\n",
+ "- topic 2: Team sports\n",
+ "- topic 3: Music, TV, and film\n",
+ "- topic 4: American college and politics\n",
+ "- topic 5: General politics\n",
+ "- topic 6: Art and publishing\n",
+ "- topic 7: Business\n",
+ "- topic 8: International athletics\n",
+ "- topic 9: Great Britain and Australia\n",
+ "- topic 10: International music\n",
+ "\n",
+ "We'll save these themes for later:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "themes = ['science and research','team sports','music, TV, and film','American college and politics','general politics', \\\n",
+ " 'art and publishing','Business','international athletics','Great Britain and Australia','international music']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Measuring the importance of top words\n",
+ "\n",
+ "We can learn more about topics by exploring how they place probability mass (which we can think of as a weight) on each of their top words.\n",
+ "\n",
+ "We'll do this with two visualizations of the weights for the top words in each topic:\n",
+ " - the weights of the top 100 words, sorted by the size\n",
+ " - the total weight of the top 10 words\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here's a plot for the top 100 words by weight in each topic:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "for i in range(10):\n",
+ " plt.plot(range(100), topic_model.get_topics(topic_ids=[i], num_words=100)['score'])\n",
+ "plt.xlabel('Word rank')\n",
+ "plt.ylabel('Probability')\n",
+ "plt.title('Probabilities of Top 100 Words in each Topic')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the above plot, each line corresponds to one of our ten topics. Notice how for each topic, the weights drop off sharply as we move down the ranked list of most important words. This shows that the top 10-20 words in each topic are assigned a much greater weight than the remaining words - and remember from the summary of our topic model that our vocabulary has 547462 words in total!\n",
+ "\n",
+ "\n",
+ "Next we plot the total weight assigned by each topic to its top 10 words: "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "top_probs = [sum(topic_model.get_topics(topic_ids=[i], num_words=10)['score']) for i in range(10)]\n",
+ "\n",
+ "ind = np.arange(10)\n",
+ "width = 0.5\n",
+ "\n",
+ "fig, ax = plt.subplots()\n",
+ "\n",
+ "ax.bar(ind-(width/2),top_probs,width)\n",
+ "ax.set_xticks(ind)\n",
+ "\n",
+ "plt.xlabel('Topic')\n",
+ "plt.ylabel('Probability')\n",
+ "plt.title('Total Probability of Top 10 Words in each Topic')\n",
+ "plt.xlim(-0.5,9.5)\n",
+ "plt.ylim(0,0.15)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here we see that, for our topic model, the top 10 words only account for a small fraction (in this case, between 5% and 13%) of their topic's total probability mass. So while we can use the top words to identify broad themes for each topic, we should keep in mind that in reality these topics are more complex than a simple 10-word summary.\n",
+ "\n",
+ "Finally, we observe that some 'junk' words appear highly rated in some topics despite our efforts to remove unhelpful words before fitting the model; for example, the word 'born' appears as a top 10 word in three different topics, but it doesn't help us describe these topics at all."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Topic distributions for some example documents\n",
+ "\n",
+ "As we noted in the introduction to this assignment, LDA allows for mixed membership, which means that each document can partially belong to several different topics. For each document, topic membership is expressed as a vector of weights that sum to one; the magnitude of each weight indicates the degree to which the document represents that particular topic.\n",
+ "\n",
+ "We'll explore this in our fitted model by looking at the topic distributions for a few example Wikipedia articles from our data set. We should find that these articles have the highest weights on the topics whose themes are most relevant to the subject of the article - for example, we'd expect an article on a politician to place relatively high weight on topics related to government, while an article about an athlete should place higher weight on topics related to sports or competition."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Topic distributions for documents can be obtained using GraphLab Create's predict() function. GraphLab Create uses a collapsed Gibbs sampler similar to the one described in the video lectures, where only the word assignments variables are sampled. To get a document-specific topic proportion vector post-facto, predict() draws this vector from the conditional distribution given the sampled word assignments in the document. Notice that, since these are draws from a _distribution_ over topics that the model has learned, we will get slightly different predictions each time we call this function on a document - we can see this below, where we predict the topic distribution for the article on Barack Obama:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "obama = gl.SArray([wiki_docs[int(np.where(wiki['name']=='Barack Obama')[0])]])\n",
+ "pred1 = topic_model.predict(obama, output_type='probability')\n",
+ "pred2 = topic_model.predict(obama, output_type='probability')\n",
+ "print(gl.SFrame({'topics':themes, 'predictions (first draw)':pred1[0], 'predictions (second draw)':pred2[0]}))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To get a more robust estimate of the topics for each document, we can average a large number of predictions for the same document:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "def average_predictions(model, test_document, num_trials=100):\n",
+ " avg_preds = np.zeros((model.num_topics))\n",
+ " for i in range(num_trials):\n",
+ " avg_preds += model.predict(test_document, output_type='probability')[0]\n",
+ " avg_preds = avg_preds/num_trials\n",
+ " result = gl.SFrame({'topics':themes, 'average predictions':avg_preds})\n",
+ " result = result.sort('average predictions', ascending=False)\n",
+ " return result"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "print average_predictions(topic_model, obama, 100)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ What is the topic most closely associated with the article about former US President George W. Bush? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ What are the top 3 topics corresponding to the article about English football (soccer) player Steven Gerrard? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Comparing LDA to nearest neighbors for document retrieval\n",
+ "\n",
+ "So far we have found that our topic model has learned some coherent topics, we have explored these topics as probability distributions over a vocabulary, and we have seen how individual documents in our Wikipedia data set are assigned to these topics in a way that corresponds with our expectations. \n",
+ "\n",
+ "In this section, we will use the predicted topic distribution as a representation of each document, similar to how we have previously represented documents by word count or TF-IDF. This gives us a way of computing distances between documents, so that we can run a nearest neighbors search for a given document based on its membership in the topics that we learned from LDA. We can contrast the results with those obtained by running nearest neighbors under the usual TF-IDF representation, an approach that we explored in a previous assignment. \n",
+ "\n",
+ "We'll start by creating the LDA topic distribution representation for each document:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "wiki['lda'] = topic_model.predict(wiki_docs, output_type='probability')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next we add the TF-IDF document representations:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "wiki['word_count'] = gl.text_analytics.count_words(wiki['text'])\n",
+ "wiki['tf_idf'] = gl.text_analytics.tf_idf(wiki['word_count'])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For each of our two different document representations, we can use GraphLab Create to compute a brute-force nearest neighbors model:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "model_tf_idf = gl.nearest_neighbors.create(wiki, label='name', features=['tf_idf'],\n",
+ " method='brute_force', distance='cosine')\n",
+ "model_lda_rep = gl.nearest_neighbors.create(wiki, label='name', features=['lda'],\n",
+ " method='brute_force', distance='cosine')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's compare these nearest neighbor models by finding the nearest neighbors under each representation on an example document. For this example we'll use Paul Krugman, an American economist:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "model_tf_idf.query(wiki[wiki['name'] == 'Paul Krugman'], label='name', k=10)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "model_lda_rep.query(wiki[wiki['name'] == 'Paul Krugman'], label='name', k=10)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Notice that that there is no overlap between the two sets of top 10 nearest neighbors. This doesn't necessarily mean that one representation is better or worse than the other, but rather that they are picking out different features of the documents. \n",
+ "\n",
+ "With TF-IDF, documents are distinguished by the frequency of uncommon words. Since similarity is defined based on the specific words used in the document, documents that are \"close\" under TF-IDF tend to be similar in terms of specific details. This is what we see in the example: the top 10 nearest neighbors are all economists from the US, UK, or Canada. \n",
+ "\n",
+ "Our LDA representation, on the other hand, defines similarity between documents in terms of their topic distributions. This means that documents can be \"close\" if they share similar themes, even though they may not share many of the same keywords. For the article on Paul Krugman, we expect the most important topics to be 'American college and politics' and 'science and research'. As a result, we see that the top 10 nearest neighbors are academics from a wide variety of fields, including literature, anthropology, and religious studies.\n",
+ "\n",
+ "\n",
+ "__Quiz Question:__ Using the TF-IDF representation, compute the 5000 nearest neighbors for American baseball player Alex Rodriguez. For what value of k is Mariano Rivera the k-th nearest neighbor to Alex Rodriguez? (Hint: Once you have a list of the nearest neighbors, you can use `mylist.index(value)` to find the index of the first instance of `value` in `mylist`.)\n",
+ "\n",
+ "__Quiz Question:__ Using the LDA representation, compute the 5000 nearest neighbors for American baseball player Alex Rodriguez. For what value of k is Mariano Rivera the k-th nearest neighbor to Alex Rodriguez? (Hint: Once you have a list of the nearest neighbors, you can use `mylist.index(value)` to find the index of the first instance of `value` in `mylist`.)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Understanding the role of LDA model hyperparameters\n",
+ "\n",
+ "Finally, we'll take a look at the effect of the LDA model hyperparameters alpha and gamma on the characteristics of our fitted model. Recall that alpha is a parameter of the prior distribution over topic weights in each document, while gamma is a parameter of the prior distribution over word weights in each topic. \n",
+ "\n",
+ "In the video lectures, we saw that alpha and gamma can be thought of as smoothing parameters when we compute how much each document \"likes\" a topic (in the case of alpha) or how much each topic \"likes\" a word (in the case of gamma). In both cases, these parameters serve to reduce the differences across topics or words in terms of these calculated preferences; alpha makes the document preferences \"smoother\" over topics, and gamma makes the topic preferences \"smoother\" over words.\n",
+ "\n",
+ "Our goal in this section will be to understand how changing these parameter values affects the characteristics of the resulting topic model.\n",
+ "\n",
+ "__Quiz Question:__ What was the value of alpha used to fit our original topic model? "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ What was the value of gamma used to fit our original topic model? Remember that GraphLab Create uses \"beta\" instead of \"gamma\" to refer to the hyperparameter that influences topic distributions over words."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We'll start by loading some topic models that have been trained using different settings of alpha and gamma. Specifically, we will start by comparing the following two models to our original topic model:\n",
+ " - tpm_low_alpha, a model trained with alpha = 1 and default gamma\n",
+ " - tpm_high_alpha, a model trained with alpha = 50 and default gamma"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "tpm_low_alpha = gl.load_model('lda_low_alpha')\n",
+ "tpm_high_alpha = gl.load_model('lda_high_alpha')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Changing the hyperparameter alpha\n",
+ "\n",
+ "Since alpha is responsible for smoothing document preferences over topics, the impact of changing its value should be visible when we plot the distribution of topic weights for the same document under models fit with different alpha values. In the code below, we plot the (sorted) topic weights for the Wikipedia article on Barack Obama under models fit with high, original, and low settings of alpha."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "a = np.sort(tpm_low_alpha.predict(obama,output_type='probability')[0])[::-1]\n",
+ "b = np.sort(topic_model.predict(obama,output_type='probability')[0])[::-1]\n",
+ "c = np.sort(tpm_high_alpha.predict(obama,output_type='probability')[0])[::-1]\n",
+ "ind = np.arange(len(a))\n",
+ "width = 0.3\n",
+ "\n",
+ "def param_bar_plot(a,b,c,ind,width,ylim,param,xlab,ylab):\n",
+ " fig = plt.figure()\n",
+ " ax = fig.add_subplot(111)\n",
+ "\n",
+ " b1 = ax.bar(ind, a, width, color='lightskyblue')\n",
+ " b2 = ax.bar(ind+width, b, width, color='lightcoral')\n",
+ " b3 = ax.bar(ind+(2*width), c, width, color='gold')\n",
+ "\n",
+ " ax.set_xticks(ind+width)\n",
+ " ax.set_xticklabels(range(10))\n",
+ " ax.set_ylabel(ylab)\n",
+ " ax.set_xlabel(xlab)\n",
+ " ax.set_ylim(0,ylim)\n",
+ " ax.legend(handles = [b1,b2,b3],labels=['low '+param,'original model','high '+param])\n",
+ "\n",
+ " plt.tight_layout()\n",
+ " \n",
+ "param_bar_plot(a,b,c,ind,width,ylim=1.0,param='alpha',\n",
+ " xlab='Topics (sorted by weight of top 100 words)',ylab='Topic Probability for Obama Article')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here we can clearly see the smoothing enforced by the alpha parameter - notice that when alpha is low most of the weight in the topic distribution for this article goes to a single topic, but when alpha is high the weight is much more evenly distributed across the topics.\n",
+ "\n",
+ "__Quiz Question:__ How many topics are assigned a weight greater than 0.3 or less than 0.05 for the article on Paul Krugman in the **low alpha** model? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ How many topics are assigned a weight greater than 0.3 or less than 0.05 for the article on Paul Krugman in the **high alpha** model? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Changing the hyperparameter gamma\n",
+ "\n",
+ "Just as we were able to see the effect of alpha by plotting topic weights for a document, we expect to be able to visualize the impact of changing gamma by plotting word weights for each topic. In this case, however, there are far too many words in our vocabulary to do this effectively. Instead, we'll plot the total weight of the top 100 words and bottom 1000 words for each topic. Below, we plot the (sorted) total weights of the top 100 words and bottom 1000 from each topic in the high, original, and low gamma models."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now we will consider the following two models:\n",
+ " - tpm_low_gamma, a model trained with gamma = 0.02 and default alpha\n",
+ " - tpm_high_gamma, a model trained with gamma = 0.5 and default alpha"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "del tpm_low_alpha\n",
+ "del tpm_high_alpha\n",
+ "tpm_low_gamma = gl.load_model('lda_low_gamma')\n",
+ "tpm_high_gamma = gl.load_model('lda_high_gamma')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "a_top = np.sort([sum(tpm_low_gamma.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "b_top = np.sort([sum(topic_model.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "c_top = np.sort([sum(tpm_high_gamma.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "\n",
+ "a_bot = np.sort([sum(tpm_low_gamma.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "b_bot = np.sort([sum(topic_model.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "c_bot = np.sort([sum(tpm_high_gamma.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "\n",
+ "ind = np.arange(len(a))\n",
+ "width = 0.3\n",
+ " \n",
+ "param_bar_plot(a_top, b_top, c_top, ind, width, ylim=0.6, param='gamma',\n",
+ " xlab='Topics (sorted by weight of top 100 words)', \n",
+ " ylab='Total Probability of Top 100 Words')\n",
+ "\n",
+ "param_bar_plot(a_bot, b_bot, c_bot, ind, width, ylim=0.0002, param='gamma',\n",
+ " xlab='Topics (sorted by weight of bottom 1000 words)',\n",
+ " ylab='Total Probability of Bottom 1000 Words')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "From these two plots we can see that the low gamma model results in higher weight placed on the top words and lower weight placed on the bottom words for each topic, while the high gamma model places relatively less weight on the top words and more weight on the bottom words. Thus increasing gamma results in topics that have a smoother distribution of weight across all the words in the vocabulary."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ For each topic of the **low gamma model**, compute the number of words required to make a list with total probability 0.5. What is the average number of words required across all topics? (HINT: use the get_topics() function from GraphLab Create with the _cdf_\\__cutoff_ argument)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ For each topic of the **high gamma model**, compute the number of words required to make a list with total probability 0.5. What is the average number of words required across all topics? (HINT: use the get_topics() function from GraphLab Create with the _cdf_\\__cutoff_ argument)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We have now seen how the hyperparameters alpha and gamma influence the characteristics of our LDA topic model, but we haven't said anything about what settings of alpha or gamma are best. We know that these parameters are responsible for controlling the smoothness of the topic distributions for documents and word distributions for topics, but there's no simple conversion between smoothness of these distributions and quality of the topic model. In reality, there is no universally \"best\" choice for these parameters. Instead, finding a good topic model requires that we be able to both explore the output (as we did by looking at the topics and checking some topic predictions for documents) and understand the impact of hyperparameter settings (as we have in this section)."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 2",
+ "language": "python",
+ "name": "python2"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 2
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython2",
+ "version": "2.7.11"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/5_lda_graphlab.ipynb b/machine_learning/4_clustering_and_retrieval/assigment/week5/5_lda_graphlab.ipynb
new file mode 100644
index 0000000..14273eb
--- /dev/null
+++ b/machine_learning/4_clustering_and_retrieval/assigment/week5/5_lda_graphlab.ipynb
@@ -0,0 +1,2406 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Latent Dirichlet Allocation for Text Data\n",
+ "\n",
+ "In this assignment you will\n",
+ "\n",
+ "* apply standard preprocessing techniques on Wikipedia text data\n",
+ "* use GraphLab Create to fit a Latent Dirichlet allocation (LDA) model\n",
+ "* explore and interpret the results, including topic keywords and topic assignments for documents\n",
+ "\n",
+ "Recall that a major feature distinguishing the LDA model from our previously explored methods is the notion of *mixed membership*. Throughout the course so far, our models have assumed that each data point belongs to a single cluster. k-means determines membership simply by shortest distance to the cluster center, and Gaussian mixture models suppose that each data point is drawn from one of their component mixture distributions. In many cases, though, it is more realistic to think of data as genuinely belonging to more than one cluster or category - for example, if we have a model for text data that includes both \"Politics\" and \"World News\" categories, then an article about a recent meeting of the United Nations should have membership in both categories rather than being forced into just one.\n",
+ "\n",
+ "With this in mind, we will use GraphLab Create tools to fit an LDA model to a corpus of Wikipedia articles and examine the results to analyze the impact of a mixed membership approach. In particular, we want to identify the topics discovered by the model in terms of their most important words, and we want to use the model to predict the topic membership distribution for a given document. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Note to Amazon EC2 users**: To conserve memory, make sure to stop all the other notebooks before running this notebook."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Text Data Preprocessing\n",
+ "We'll start by importing our familiar Wikipedia dataset.\n",
+ "\n",
+ "The following code block will check if you have the correct version of GraphLab Create. Any version later than 1.8.5 will do. To upgrade, read [this page](https://turi.com/download/upgrade-graphlab-create.html)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import graphlab as gl\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt \n",
+ "\n",
+ "%matplotlib inline\n",
+ "\n",
+ "'''Check GraphLab Create version'''\n",
+ "from distutils.version import StrictVersion\n",
+ "assert (StrictVersion(gl.version) >= StrictVersion('1.8.5')), 'GraphLab Create must be version 1.8.5 or later.'"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " | URI | \n",
+ " name | \n",
+ " text | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Digby_Morrell> ... | \n",
+ " Digby Morrell | \n",
+ " digby morrell born 10
october 1979 is a former ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Alfred_J._Lewy> ... | \n",
+ " Alfred J. Lewy | \n",
+ " alfred j lewy aka sandy
lewy graduated from ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Harpdog_Brown> ... | \n",
+ " Harpdog Brown | \n",
+ " harpdog brown is a singer
and harmonica player who ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Franz_Rottensteiner> ... | \n",
+ " Franz Rottensteiner | \n",
+ " franz rottensteiner born
in waidmannsfeld lower ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/G-Enka> ... | \n",
+ " G-Enka | \n",
+ " henry krvits born 30
december 1974 in tallinn ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Sam_Henderson> ... | \n",
+ " Sam Henderson | \n",
+ " sam henderson born
october 18 1969 is an ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Aaron_LaCrate> ... | \n",
+ " Aaron LaCrate | \n",
+ " aaron lacrate is an
american music producer ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Trevor_Ferguson> ... | \n",
+ " Trevor Ferguson | \n",
+ " trevor ferguson aka john
farrow born 11 november ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Grant_Nelson> ... | \n",
+ " Grant Nelson | \n",
+ " grant nelson born 27
april 1971 in london ... | \n",
+ "
\n",
+ " \n",
+ " <http://dbpedia.org/resou
rce/Cathy_Caruth> ... | \n",
+ " Cathy Caruth | \n",
+ " cathy caruth born 1955 is
frank h t rhodes ... | \n",
+ "
\n",
+ "
\n",
+ "[59071 rows x 3 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.\n",
+ "
Number of documents 59071
"
+ ],
+ "text/plain": [
+ " Number of documents 59071"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ " Vocabulary size 547462
"
+ ],
+ "text/plain": [
+ " Vocabulary size 547462"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ " Running collapsed Gibbs sampling
"
+ ],
+ "text/plain": [
+ " Running collapsed Gibbs sampling"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+-----------+---------------+----------------+-----------------+
"
+ ],
+ "text/plain": [
+ "+-----------+---------------+----------------+-----------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Iteration | Elapsed Time | Tokens/Second | Est. Perplexity |
"
+ ],
+ "text/plain": [
+ "| Iteration | Elapsed Time | Tokens/Second | Est. Perplexity |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+-----------+---------------+----------------+-----------------+
"
+ ],
+ "text/plain": [
+ "+-----------+---------------+----------------+-----------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 10 | 6.24s | 1.4602e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 10 | 6.24s | 1.4602e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 20 | 11.64s | 1.47799e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 20 | 11.64s | 1.47799e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 30 | 16.98s | 1.50132e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 30 | 16.98s | 1.50132e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 40 | 22.37s | 1.53911e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 40 | 22.37s | 1.53911e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 50 | 27.87s | 1.49653e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 50 | 27.87s | 1.49653e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 60 | 34.00s | 1.38525e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 60 | 34.00s | 1.38525e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 70 | 39.98s | 1.46159e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 70 | 39.98s | 1.46159e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 80 | 45.59s | 1.33147e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 80 | 45.59s | 1.33147e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 90 | 51.41s | 1.1672e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 90 | 51.41s | 1.1672e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 100 | 57.79s | 1.27921e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 100 | 57.79s | 1.27921e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 110 | 1m 4s | 1.36326e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 110 | 1m 4s | 1.36326e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 120 | 1m 10s | 1.29229e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 120 | 1m 10s | 1.29229e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 130 | 1m 17s | 1.16539e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 130 | 1m 17s | 1.16539e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 140 | 1m 23s | 1.16374e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 140 | 1m 23s | 1.16374e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 150 | 1m 29s | 1.37665e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 150 | 1m 29s | 1.37665e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 160 | 1m 35s | 1.41662e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 160 | 1m 35s | 1.41662e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 170 | 1m 41s | 1.08636e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 170 | 1m 41s | 1.08636e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 180 | 1m 49s | 1.142e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 180 | 1m 49s | 1.142e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 190 | 1m 55s | 1.24898e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 190 | 1m 55s | 1.24898e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 200 | 2m 2s | 1.34335e+07 | 0 |
"
+ ],
+ "text/plain": [
+ "| 200 | 2m 2s | 1.34335e+07 | 0 |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+-----------+---------------+----------------+-----------------+
"
+ ],
+ "text/plain": [
+ "+-----------+---------------+----------------+-----------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "topic_model = gl.topic_model.create(wiki_docs, num_topics=10, num_iterations=200)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "GraphLab provides a useful summary of the model we have fitted, including the hyperparameter settings for alpha, gamma (note that GraphLab Create calls this parameter beta), and K (the number of topics); the structure of the output data; and some useful methods for understanding the results."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "Class : TopicModel\n",
+ "\n",
+ "Schema\n",
+ "------\n",
+ "Vocabulary Size : 547462\n",
+ "\n",
+ "Settings\n",
+ "--------\n",
+ "Number of Topics : 10\n",
+ "alpha : 5.0\n",
+ "beta : 0.1\n",
+ "Iterations : 200\n",
+ "Training time : 123.0197\n",
+ "Verbose : False\n",
+ "\n",
+ "Accessible fields : \n",
+ "m['topics'] : An SFrame containing the topics.\n",
+ "m['vocabulary'] : An SArray containing the words in the vocabulary.\n",
+ "Useful methods : \n",
+ "m.get_topics() : Get the most probable words per topic.\n",
+ "m.predict(new_docs) : Make predictions for new documents."
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "topic_model"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It is certainly useful to have pre-implemented methods available for LDA, but as with our previous methods for clustering and retrieval, implementing and fitting the model gets us only halfway towards our objective. We now need to analyze the fitted model to understand what it has done with our data and whether it will be useful as a document classification system. This can be a challenging task in itself, particularly when the model that we use is complex. We will begin by outlining a sequence of objectives that will help us understand our model in detail. In particular, we will\n",
+ "\n",
+ "* get the top words in each topic and use these to identify topic themes\n",
+ "* predict topic distributions for some example documents\n",
+ "* compare the quality of LDA \"nearest neighbors\" to the NN output from the first assignment\n",
+ "* understand the role of model hyperparameters alpha and gamma"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Load a fitted topic model\n",
+ "The method used to fit the LDA model is a _randomized algorithm_, which means that it involves steps that are random; in this case, the randomness comes from Gibbs sampling, as discussed in the LDA video lectures. Because of these random steps, the algorithm will be expected to yield slighty different output for different runs on the same data - note that this is different from previously seen algorithms such as k-means or EM, which will always produce the same results given the same input and initialization.\n",
+ "\n",
+ "It is important to understand that variation in the results is a fundamental feature of randomized methods. However, in the context of this assignment this variation makes it difficult to evaluate the correctness of your analysis, so we will load and analyze a pre-trained model. \n",
+ "\n",
+ "We recommend that you spend some time exploring your own fitted topic model and compare our analysis of the pre-trained model to the same analysis applied to the model you trained above."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "collapsed": false,
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "topic_model = gl.load_model('topic_models/lda_assignment_topic_model')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Identifying topic themes by top words\n",
+ "\n",
+ "We'll start by trying to identify the topics learned by our model with some major themes. As a preliminary check on the results of applying this method, it is reasonable to hope that the model has been able to learn topics that correspond to recognizable categories. In order to do this, we must first recall what exactly a 'topic' is in the context of LDA. \n",
+ "\n",
+ "In the video lectures on LDA we learned that a topic is a probability distribution over words in the vocabulary; that is, each topic assigns a particular probability to every one of the unique words that appears in our data. Different topics will assign different probabilities to the same word: for instance, a topic that ends up describing science and technology articles might place more probability on the word 'university' than a topic that describes sports or politics. Looking at the highest probability words in each topic will thus give us a sense of its major themes. Ideally we would find that each topic is identifiable with some clear theme _and_ that all the topics are relatively distinct.\n",
+ "\n",
+ "We can use the GraphLab Create function get_topics() to view the top words (along with their associated probabilities) from each topic.\n",
+ "\n",
+ "__Quiz Question:__ Identify the top 3 most probable words for the first topic. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " | topic | \n",
+ " word | \n",
+ " score | \n",
+ "
\n",
+ " \n",
+ " | 0 | \n",
+ " university | \n",
+ " 0.0337723780773 | \n",
+ "
\n",
+ " \n",
+ " | 0 | \n",
+ " research | \n",
+ " 0.0120334992502 | \n",
+ "
\n",
+ " \n",
+ " | 0 | \n",
+ " professor | \n",
+ " 0.0118011432268 | \n",
+ "
\n",
+ "
\n",
+ "[3 rows x 3 columns]
\n",
+ "
Starting brute force nearest neighbors model training.
"
+ ],
+ "text/plain": [
+ "Starting brute force nearest neighbors model training."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "Starting brute force nearest neighbors model training.
"
+ ],
+ "text/plain": [
+ "Starting brute force nearest neighbors model training."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "model_tf_idf = gl.nearest_neighbors.create(wiki, label='name', features=['tf_idf'],\n",
+ " method='brute_force', distance='cosine')\n",
+ "model_lda_rep = gl.nearest_neighbors.create(wiki, label='name', features=['lda'],\n",
+ " method='brute_force', distance='cosine')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's compare these nearest neighbor models by finding the nearest neighbors under each representation on an example document. For this example we'll use Paul Krugman, an American economist:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "Starting pairwise querying.
"
+ ],
+ "text/plain": [
+ "Starting pairwise querying."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |
"
+ ],
+ "text/plain": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 0 | 1 | 0.00169288 | 16.669ms |
"
+ ],
+ "text/plain": [
+ "| 0 | 1 | 0.00169288 | 16.669ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Done | | 100 | 324.125ms |
"
+ ],
+ "text/plain": [
+ "| Done | | 100 | 324.125ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " | query_label | \n",
+ " reference_label | \n",
+ " distance | \n",
+ " rank | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Paul Krugman | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Elise Brezis | \n",
+ " 0.744498017262 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Maitreesh Ghatak | \n",
+ " 0.81564984831 | \n",
+ " 3 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Kai A. Konrad | \n",
+ " 0.823700564406 | \n",
+ " 4 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " David Colander | \n",
+ " 0.834625927759 | \n",
+ " 5 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Richard Blundell | \n",
+ " 0.837934267874 | \n",
+ " 6 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Gordon Rausser | \n",
+ " 0.83941534706 | \n",
+ " 7 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Edward J. Nell | \n",
+ " 0.842178500015 | \n",
+ " 8 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Robin Boadway | \n",
+ " 0.842374260596 | \n",
+ " 9 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Tim Besley | \n",
+ " 0.843088109253 | \n",
+ " 10 | \n",
+ "
\n",
+ "
\n",
+ "[10 rows x 4 columns]
\n",
+ "
Starting pairwise querying.
"
+ ],
+ "text/plain": [
+ "Starting pairwise querying."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |
"
+ ],
+ "text/plain": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 0 | 1 | 0.00169288 | 2.839ms |
"
+ ],
+ "text/plain": [
+ "| 0 | 1 | 0.00169288 | 2.839ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Done | | 100 | 24.178ms |
"
+ ],
+ "text/plain": [
+ "| Done | | 100 | 24.178ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " | query_label | \n",
+ " reference_label | \n",
+ " distance | \n",
+ " rank | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Paul Krugman | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Allen Feldman | \n",
+ " 0.00264054322093 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Cleo Paskal | \n",
+ " 0.00503243838456 | \n",
+ " 3 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Lene Auestad | \n",
+ " 0.00725301227925 | \n",
+ " 4 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Ben Klemens | \n",
+ " 0.00878201993564 | \n",
+ " 5 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Steve Lawrence (computer
scientist) ... | \n",
+ " 0.0109690477594 | \n",
+ " 6 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Randall Hansen | \n",
+ " 0.0110336501005 | \n",
+ " 7 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Metta Spencer | \n",
+ " 0.0110450122822 | \n",
+ " 8 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Steven Skiena | \n",
+ " 0.0114691009306 | \n",
+ " 9 | \n",
+ "
\n",
+ " \n",
+ " | Paul Krugman | \n",
+ " Mohamed Osman Elkhosht | \n",
+ " 0.0115525590812 | \n",
+ " 10 | \n",
+ "
\n",
+ "
\n",
+ "[10 rows x 4 columns]
\n",
+ "
Starting pairwise querying.
"
+ ],
+ "text/plain": [
+ "Starting pairwise querying."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |
"
+ ],
+ "text/plain": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 0 | 1 | 0.00169288 | 20.877ms |
"
+ ],
+ "text/plain": [
+ "| 0 | 1 | 0.00169288 | 20.877ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Done | | 100 | 302.418ms |
"
+ ],
+ "text/plain": [
+ "| Done | | 100 | 302.418ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Get a list of 'reference_label' based on knn\n",
+ "alex_rodriguez_tfidf = list(model_tf_idf.query(wiki[wiki['name'] == 'Alex Rodriguez'], label='name', k=5000)['reference_label'])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 40,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "value of k: 52\n"
+ ]
+ }
+ ],
+ "source": [
+ "print 'value of k:', alex_rodriguez_tfidf.index('Mariano Rivera')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ Using the LDA representation, compute the 5000 nearest neighbors for American baseball player Alex Rodriguez. For what value of k is Mariano Rivera the k-th nearest neighbor to Alex Rodriguez? (Hint: Once you have a list of the nearest neighbors, you can use `mylist.index(value)` to find the index of the first instance of `value` in `mylist`.)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 41,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "Starting pairwise querying.
"
+ ],
+ "text/plain": [
+ "Starting pairwise querying."
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |
"
+ ],
+ "text/plain": [
+ "| Query points | # Pairs | % Complete. | Elapsed Time |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| 0 | 1 | 0.00169288 | 3.993ms |
"
+ ],
+ "text/plain": [
+ "| 0 | 1 | 0.00169288 | 3.993ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "| Done | | 100 | 23.926ms |
"
+ ],
+ "text/plain": [
+ "| Done | | 100 | 23.926ms |"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "text/html": [
+ "+--------------+---------+-------------+--------------+
"
+ ],
+ "text/plain": [
+ "+--------------+---------+-------------+--------------+"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "value of k: 729\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Get a list of 'reference_label' based on knn\n",
+ "alex_rodriguez_lda = list(model_lda_rep.query(wiki[wiki['name'] == 'Alex Rodriguez'], label='name', k=5000)['reference_label'])\n",
+ "print 'value of k:', alex_rodriguez_lda.index('Mariano Rivera')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Understanding the role of LDA model hyperparameters\n",
+ "\n",
+ "Finally, we'll take a look at the effect of the LDA model hyperparameters alpha and gamma on the characteristics of our fitted model. Recall that alpha is a parameter of the prior distribution over topic weights in each document, while gamma is a parameter of the prior distribution over word weights in each topic. \n",
+ "\n",
+ "In the video lectures, we saw that alpha and gamma can be thought of as smoothing parameters when we compute how much each document \"likes\" a topic (in the case of alpha) or how much each topic \"likes\" a word (in the case of gamma). In both cases, these parameters serve to reduce the differences across topics or words in terms of these calculated preferences; alpha makes the document preferences \"smoother\" over topics, and gamma makes the topic preferences \"smoother\" over words.\n",
+ "\n",
+ "Our goal in this section will be to understand how changing these parameter values affects the characteristics of the resulting topic model.\n",
+ "\n",
+ "__Quiz Question:__ What was the value of alpha used to fit our original topic model? "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 43,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "5.0"
+ ]
+ },
+ "execution_count": 43,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "topic_model['alpha']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ What was the value of gamma used to fit our original topic model? Remember that GraphLab Create uses \"beta\" instead of \"gamma\" to refer to the hyperparameter that influences topic distributions over words."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 44,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.1"
+ ]
+ },
+ "execution_count": 44,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "topic_model['beta']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We'll start by loading some topic models that have been trained using different settings of alpha and gamma. Specifically, we will start by comparing the following two models to our original topic model:\n",
+ " - tpm_low_alpha, a model trained with alpha = 1 and default gamma\n",
+ " - tpm_high_alpha, a model trained with alpha = 50 and default gamma"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 45,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "tpm_low_alpha = gl.load_model('topic_models/lda_low_alpha')\n",
+ "tpm_high_alpha = gl.load_model('topic_models/lda_high_alpha')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Changing the hyperparameter alpha\n",
+ "\n",
+ "Since alpha is responsible for smoothing document preferences over topics, the impact of changing its value should be visible when we plot the distribution of topic weights for the same document under models fit with different alpha values. In the code below, we plot the (sorted) topic weights for the Wikipedia article on Barack Obama under models fit with high, original, and low settings of alpha."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 46,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNXV//HPYVgU2UFRGBhQwR1/IiK4seUhoEE0Piig\nxqBRNEET8+SJQUIgRqMxxhiiPK7BhaBx30CDCY47YgRxCZsQBhhwQUABlfX8/qjbQ88w01Mj1HQz\n832/Xv2arqrbVaebYU7XrVv3mLsjIiKSi+pkOwAREZGKKEmJiEjOUpISEZGcpSQlIiI5S0lKRERy\nlpKUiIjkrESTlJndY2Yfm9m7GdpMMLNFZvaOmf2/JOMREZE9S9JnUpOAb1e00cwGAge5eydgJHB7\nwvGIiMgeJNEk5e6vAmszNBkM3B/avgk0NbPWScYkIiJ7jmxfk2oLLE9bLg7rREREsp6kREREKlQ3\ny8cvBtqlLeeHdTsxM00yKCJSg7m7lV1XHWdSFh7leRr4HoCZ9QDWufvHFe3I3WM9xo0bF7ttdT4U\n154dk+La82NSXLkbU0USPZMysylAb6ClmS0DxgH1o3zjd7r7NDM71cw+BDYCI5KMR0RE9iyJJil3\nHx6jzagkYxARkT1XjRw40bt372yHUC7FFV8uxgSKqypyMSZQXFWRCzFZpr7AXGJmvqfEKiIiVWNm\neDkDJ7I9uk9EaokOHTpQVFSU7TAkywoKCli6dGns9jqTEpFqEb4pZzsMybKKfg8qOpOqkdekRESk\nZlCSEhGRnKUkJSIiOUtJSkRqvY4dOzJjxoysHb9OnTosWbJkt7etCZSkRCRr8gs6YGaJPfILOmT7\nLcZiVtHMcbvWtibQEHQRyZriZUVcP3tzYvsf3bV+Yvvenaoy6rG2jZDUmZSISJrNmzfzk5/8hLZt\n25Kfn8+VV17Jli1bgGgGhieeeAKA1157jTp16vDcc88BMGPGDI455phy9/nWW29xwgkn0Lx5c9q2\nbcvll1/O1q1by207YsQILrvsMvr370+TJk3o06cPy5YtK9XmhRdeoHPnzrRo0YJRo3bMLLdkyRL6\n9etHq1at2G+//TjvvPP44osvdvkzySYlKRGRNNdeey2zZs3i3XffZe7cucyaNYtrr70WgF69elFY\nWAjAyy+/zEEHHcTLL78MwEsvvVThNEJ5eXnccsstrFmzhjfeeIMZM2YwceLECmOYMmUK48aN47PP\nPuPoo4/m3HPPLbV96tSpvP3228ydO5eHH36Y6dOnA9FZ1tVXX81HH33EvHnzWLFiBePHj9+1DyTL\nlKRERNKkEkTLli1p2bIl48aN44EHHgCiJPXSSy8BUZIaPXp0yfJLL71Er169yt1n165d6d69O2ZG\n+/btueSSS0peV57TTjuNE088kXr16nHdddfxxhtvUFy8o9Te6NGjady4Me3ataNPnz688847ABx0\n0EH069ePunXr0rJlS6688sqMx9kTKEmJiKRZuXIl7du3L1kuKChg5cqVAPTs2ZOFCxfyySefMHfu\nXL73ve+xfPlyPvvsM2bNmsUpp5xS7j4XLVrEoEGDOOCAA2jWrBljxoxh9erVFcbQrt2OWrD77LMP\nLVq0KIkBoHXr1iXPGzZsyIYNGwD45JNPGDZsGPn5+TRr1ozzzjsv43H2BEpSIiJp2rRpU2qOwaKi\nItq0aQPA3nvvzbHHHsuf/vQnjjzySOrWrUvPnj25+eabOfjgg2nRokW5+7zssss47LDDWLx4MevW\nreO6667LOABi+fLlJc83bNjAmjVraNu2baWxX3311dSpU4cPPviAdevWMXny5D1+oIWSlIhImmHD\nhnHttdeyevVqVq9ezW9+8xvOP//8ku2nnHIKt956a0nXXu/evUstl2f9+vU0adKEhg0bMn/+fP7v\n//4vYwzTpk3j9ddfZ/PmzYwdO5aePXuWJMpM1q9fT6NGjWjcuDHFxcX8/ve/j/muc5eSlIjUeun3\nHv3yl7+kW7dudOnShaOPPppu3boxZsyYku29evViw4YNJV17qeVMSeqmm27ir3/9K02aNGHkyJEM\nHTq0wuMDDB8+nPHjx9OyZUvmzJnD5MmTK2ybbty4cbz99ts0a9aMQYMGcdZZZ8X7AHKYZkEXkWpR\n3uzX+QUdKF6WXPmOtu0LWFG0NLH9J2HEiBG0a9eOa665JtuhJKKqs6DrZl4RyZo9LYFI9VN3n4hI\nDqlt0x5VRt19IlItVPRQQEUPRUSkBlGSEhGRnKUkJSIiOUtJSkREcpaSlIiI5CwlKRGRKrrsssu4\n7rrrdnvbTIqKiqhTpw7bt2/f5X1V1YgRI/jVr34Vq23Hjh2ZMWPGbju2buYVkazpkJ9PUVoJit2t\noG1blq5Ysdv3W9nce9+0bWVq4z1UsZKUmQ0FDnL368ysHbCfu7+dbGgiUtMVFRezLsGifM0S2Pf2\n7dupU0edUNWl0k/azG4F+gDnhVUbgduTDEpEpDrNnz+fPn360Lx5c4466iieeeaZkm0jRozghz/8\nIaeddhqNGzemsLBwp+6vG2+8kTZt2pCfn88999xDnTp1WLJkScnrU21feukl2rVrx80330zr1q1p\n27Yt9957b8l+pk2bRteuXWnatCkFBQX8+te/jv0eOnbsyE033cTRRx9N48aNufjii/nkk0849dRT\nadKkCf379+fzzz8vaf/0009z5JFH0qJFC/r27cv8+fNLts2ZM4djjz2Wpk2bMnToUL7++utSx3r2\n2Wc55phjaN68OSeddBLvvfde7DirKs7XgRPcfSTwNYC7rwHqJxaRiEg12rp1K4MGDWLAgAF8+umn\nTJgwgXPPPZdFixaVtHnwwQcZO3Ys69ev58QTTyz1+ueff55bbrmFGTNm8OGHH1JYWJixW+6jjz5i\n/fr1rFy5krvvvpsf/ehHJcmjUaNGPPDAA3z++edMnTqV22+/naeffjr2e3n88cf55z//ycKFC3n6\n6ac59dRTueGGG1i9ejXbtm1jwoQJACxcuJDhw4czYcIEPv30UwYOHMigQYPYunUrW7Zs4cwzz+SC\nCy5gzZo1DBkyhMcee6zkGHPmzOGiiy7irrvuYs2aNYwcOZLTTz+dLVu2xI6zKuIkqS1mVgdwADNr\nCVT/lTsRkQTMnDmTjRs3ctVVV1G3bl369OnDd77zHR588MGSNoMHD6ZHjx4ANGjQoNTrH3nkEUaM\nGMGhhx7KXnvtxfhKuhjr16/P2LFjycvLY+DAgTRq1IgFCxYAUa2qI444AoAjjzySoUOHVqn8++WX\nX06rVq044IADOPnkkzn++OPp0qUL9evX58wzz2TOnDkAPPzww3znO9+hb9++5OXl8bOf/Yyvv/6a\n119/nZkzZ7J161auuOIK8vLyOOusszjuuONKjnHXXXdx6aWX0q1bN8yM888/nwYNGjBz5szYcVZF\nnCR1G/AYsK+Z/Rp4FfhdItGIiFSzlStXlirXDlHJ+OK0AR1lt2d6fbt27TLOUdiyZctS17TSy7+/\n+eab9O3bl/32249mzZpxxx13VKn8e3pZ+b333nun5dRxVq5cSUFBQck2MyM/P5/i4mJWrly5UxXg\n9LZFRUX84Q9/oEWLFrRo0YLmzZuzYsWKUuXtd6dKk5S73w/8ErgJWAsMcfeHEolGRKSatWnTplS5\ndoBly5aV+kOdqfvugAMOYEXaCMJly5Z941F45557LmeccQbFxcWsW7eOkSNHJjIpb5s2bSgqKl3H\na/ny5bRt23an9wPRe0pp164dY8aMYc2aNaxZs4a1a9eyYcMGzjnnnN0eJ2RIUmbWJPUAlgOTgL8A\ny8I6EZE93vHHH0/Dhg258cYb2bp1K4WFhTz77LMMGzYs1uvPPvtsJk2axPz58/nyyy+59tprv3Es\nGzZsoHnz5tSrV49Zs2YxZcqUUtt3V8I6++yzmTp1Ki+++CJbt27lpptuYq+99uKEE06gZ8+e1KtX\njz//+c9s3bqVxx9/nFmzZpW89uKLL+b2228vWbdx40amTZvGxo0bd0tsZWU6k/oAeD/8TD1/P+25\niMger169ejzzzDNMmzaNVq1aMWrUKB544AE6deoElH8Wlb5uwIABXHHFFfTp04fOnTvTs2dPYOdr\nVxVJ39fEiRMZO3YsTZs25dprr93p7CTTGVrZbZnadu7cmcmTJzNq1Cj23Xdfpk6dyjPPPEPdunWp\nV68ejz/+OJMmTaJly5Y88sgjpcrQH3vssdx1112MGjWKFi1a0LlzZ+67775Yx/0mVE9KRKpFeXWE\n9tSbeTOZP38+Rx11FJs2bdL9VOXY7fWkzOx0M2uattzMzL6zy5GKSK23dMUK3D2xR3UlqCeffJLN\nmzezdu1arrrqKk4//XQlqN0kzqd4jbuX3AHm7uuA38Q9gJkNMLP5ZrbQzK4qZ3sTM3vazN4xs/fM\n7Ptx9y0ikgvuuOMO9ttvPzp16kS9evWYOHFitkOqMSrt7jOzue5+dJl177n7UZXuPLq/aiHQD1gJ\nvAUMdff5aW1GA03cfbSZtQIWAK3dfWuZfam7T2QPpvLxAsmUj59jZjeaWUF4/B6YEzOe7sAidy9y\n9y3AQ8DgMm0caByeNwY+K5ugRESkdoqTpEaFdk+FB8APY+6/LdHw9ZQVYV26W4HDzWwlMBf4ccx9\ni4hIDVfpLOjuvgH4WYIxfBuY4+59zewg4AUz6xKOKyIitViFScrM/uDu/2NmTxDm7Uvn7t+Nsf9i\noH3acn5Yl24EcH3Y52Iz+w9wKPCvsjtLnxOrd+/e9O7dO0YIIiKSawoLCyksLKy0XYUDJ8ysu7vP\nMrN+5W13939WunOzPKKBEP2AVcAsYJi7z0trcxvwibv/2sxaEyWno8Ns6+n70sAJkT2YBk4I7MaB\nE+6emgfjMHf/Z/oDOCxOMO6+jeia1nSimSoecvd5ZjbSzC4Jza4FTjCzd4EXgJ+XTVAiIknKVPL8\n1Vdf5bDDYv3JK6kXtTtUZV+787i5Jk5l3guJBjeku6icdeVy9+eBQ8qsuyPt+Sqi61IiUst0KNif\nomUfJ7b/gvatWVr00S7t46STTmLevHmVNwx257RAVdlXTS0tn+ma1DnAUOBAM3s8bVNjYF3SgYlI\nzVe07GM8/t//KrPDkkuAUj0yDUGfRVRLalH4mXqMAfonH5qISPWZM2cORx99NM2bN2fYsGFs3rwZ\n2Lkrbfbs2SUl3s8++2yGDh1aqpS8u1dYHr6se++9l8MPP5wmTZpw8MEHc+edd1bYtmPHjtxwww0c\nccQRtGzZkosuuqgkxsqOuytl6bMt0zWp/wAvAuvKXJOaFW7MFRGpMR555BGmT5/Of/7zH+bOnVvq\nj3yqK23Lli1897vf5cILL2TNmjUMGzaMJ554otR+MpWHL6t169ZMmzaNL774gkmTJnHllVfyzjvv\nVBjjlClTeOGFF1i8eDELFiwoVRYkybL02ZTxZt4w8CFP9aNEpKb78Y9/TOvWrWnWrBmDBg0qN1m8\n8cYbbNu2jVGjRpGXl8eZZ55J9+7dS7XJVB6+rIEDB9KhQwcATj75ZPr3788rr7xSYYyXX345bdq0\noVmzZowZM6ZUifsky9JnU5yBE58Dc81sOlBS1crdf5pYVCIi1Sy91HrDhg1ZtWrVTm1WrVq1U2n1\nsqPqMpWHL+u5557jmmuuYeHChWzfvp2vvvqKLl26VBhjfn5+yfOCgoJSJdsrK0s/evRo3n//fTZv\n3szmzZsZMmRIhcfJJXGmRXqWaJj4LEoXQBQRqVUOOOAAisvUvypbej6uzZs389///d/8/Oc/59NP\nP2Xt2rUMHDgw471k6ccqKiqiTZs2sY5VXWXpk1BpknL3e9IfwHNA08peJyJS0/Ts2ZO8vDxuu+02\ntm3bxlNPPVWqtHpVpM5oWrVqRZ06dXjuueeYPn16xtfcdtttFBcXs2bNGn77298ydOjQWMeqrCx9\nLotVlcvMWpjZJWb2IvA6UJBsWCIi1SfuPUap0up33303zZs3Z8qUKQwaNChjqfiK9t2oUSMmTJjA\nkCFDaNGiBQ899BCDB5ctElHa8OHD6d+/PwcffDCdOnVizJgxsY5bWVn6XJZpWqR9gDOA4cARRDOg\n/7e7l53FvFpoWiSRPVu55eP3gJt5K9OjRw8uu+wyLrjggkSP07FjR+655x769u2b6HGSVtVpkTIN\nnPiEaB698cBL7r7dzE7fXYF+E3G/7bRtX8CKoqXJBiMiuyzpBJKEl19+mUMOOYRWrVoxefJk3nvv\nPQYMGJDtsGqsTElqHNGMEzcDD5rZ3yhnNvTqdP3szZU3AkZ3rZ9wJCJSWy1YsICzzz6bL7/8kgMP\nPJDHHnus1MjApNTUaY8qE6d8fCdgGGGKJKIZJ55w9yXJh1cqDq9KklLXoEhu0SzoAgmUj3f3Re5+\njbsfDvQA9gMqLdMhIiKyq2KN7ktx93fc/Sp375hUQCIiIilVSlIiIiLVSUlKRERyVpy5+0REdllB\nQUGtHaEmOxQUVG0uiEqTlJkdBFwHHA7slVrv7p2rGpyI1F5Lly7NdgiyB4rT3XcvMAkwYCDwMPC3\nBGMSEREB4iWphu7+dwB3X+zuvyRKViIiIomKc01qk5nVARab2aVAMdA42bBERETiJakrgX2AK4iu\nTTUFLkwyKBEREYiRpNz9zfB0PXB+suGIiIjsEGd0X1dgNFENqZL27t41wbhERERidfc9SJSk3gO2\nJxuOiIjIDnGS1Gp3fzzxSERERMqIk6R+bWa3E818vim10t2fTiwqERER4iWpc4EuRMPOU919DihJ\niYhIouIkqR7ufkjikYiIiJQRZ8aJN81MSUpERKpdnDOpY4B3zexDomtSBriGoIuISNLiJKkzEo9C\nRESkHHFmnFgMYGYtSCvVISIikrRKr0mZ2WlmthBYAbwJLAdmJB2YiIhInIET1wEnAgvcvR0wAHgl\n0ahERESIl6S2uvunQB0zM3d/AeiecFwiIiKxBk58bmaNgNeA+83sE+CrZMMSERGJdyZ1BvA18GOg\nkKjo4aAEY9plDfLyMLPYjw75+dkOWUREyhFndN96M9sXOA5YCTwduv9iMbMBwC1ECfEed/9dOW16\nA38E6gGfunufuPsvz6Zt21g3fnzs9s2q0FZERKpPnNF9I4DZwHDgPOBfZnZBnJ2HsvO3At8GjgCG\nmdmhZdo0BW4DvuPuRwJDqvQORESkxopzTeoXQNfU2VM4q3oVuC/Ga7sDi9y9KLz2IWAwMD+tzXDg\nMXcvBnD31fHDFxGRmizONak1wLq05XVhXRxtie6rSlkR1qXrDLQwsxfN7C0zU4l6EREBMpxJmdkV\n4ekC4A0ze5KoRMcZwPu7OYauQF9gn3CsN9z9w914DBER2QNl6u7bN/xcHh4NwvLzVdh/MdA+bTk/\nrEu3gqj679fA12b2MnA0sFOS+sft15Q8P7BbLw7s1qsKoYiISK4oLCyksLCw0nbm7pU3MtsLICSS\n2Mwsj+hMrB+wCpgFDHP3eWltDgX+TDSTRQOiqZfOcfd/l9mXXz97c6zjju5av8qj++J8DiIikgwz\nw92t7PqM16TM7GIzWwJ8BHxkZovN7JK4B3X3bcAoYDrwAfCQu88zs5Gp/bj7fODvwLvATODOsglK\nRERqp0zXpEYDvYEB7r4wrOsM/MnMWrr79XEO4O7PA4eUWXdHmeWbgJuqFrqIiNR0mc6kvg+ckUpQ\nAOH5WcCIhOMSERHJmKTc3Xeao8/dvwS2JxeSiIhIJFOSWhWmKyrFzHoRXaMSERFJVKYh6FcAT5rZ\ni8DbYV03outUKikvIiKJq/BMyt3fA44kGjZ+aHjMAo4K20RERBKVce6+cE3qzmqKRUREpJQ4c/eJ\niIhkhZKUiIjkrMpmnMgzs/urKxgREZF0GZNUmNboQDOrV03xiIiIlIhT9HAx8IqZPQVsTK109wmJ\nRSUiIkK8JLUsPBqGh4iISLWoNEm5+1gAM9s7LO80VZKIiEgSKh3dZ2aHm9lbwCJgkZm9aWaHJR+a\niIjUdnGGoN8JXO3u+e6eD4wB7ko2LBERkXhJqrG7v5BacPd/AI2TC0lERCQSJ0ktNbPRZpYfHr8A\nliYcl4iISKwkdSHQDpgGTAXywzoREZFEZSoff6+7fx8Y5u4/rL6QREREIpnOpLqb2X7AxWbW2Mya\npD+qK0AREam9Mt0ndTfwGtAe+ACwtG0e1ouIiCQmU9HDm929E3C/u7d393ZpDyUoERFJXKUDJ9z9\n4uoIREREpCzVkxIRkZylJCUiIjkrztx9l5lZ0+oIJlsa1Aczi/XoULB/tsMVEak14pTqKABmm9mb\nwF/CtEg1yqbN4PPitbXDPk42GBERKRFn4MQvgE7AX4FLzWyRmV1jZh0Sjk1ERGq5WNek3H070Xx9\nS4HtwAHAU2Z2fWKRiYhIrVdpd5+Z/Qi4APgCuAcY4+6bzKwO8CEwOtkQRUSktopzTaoN0fx9i9NX\nuvt2Mzs9mbBERETidfe1LZugzOxeAHd/P4mgREREIF6S6pK+ELr5jksmHBERkR0qTFJmdpWZrQW6\nmNma8FgLrCaqLSUiIpKoTGdSNwL7An8MP/cFWrl7C3f/3+oITkREardMAycOdvdFZvYAcERqpVlU\nscPd3004NhERqeUyJanRRGXibytnmwOnJBKRiIhIUGGScvcLw8+Tqy8cERGRHSpMUpXdA+XuT8c5\ngJkNAG4huv51j7v/roJ2xwGvA+e4++Nx9i0iIjVbpu6+IRm2OVBpkgrD1W8F+gErgbfM7Cl3n19O\nuxuAv1casYiI1BqZuvvO3w377w4scvciADN7CBgMzC/T7nLgUXT/lYiIpMnU3TfM3R80syvK2+7u\nE2Lsvy2wPG15BVHiSj9OG+AMd+9jZqW2iYhI7Zapu695+LlvwjHcAlyVtmwJH09ERPYQmbr7Joaf\nY3dh/8VA+7Tl/LAuXTfgIYtuwGoFDDSzLeUNzPjH7deUPD+wWy8O7NZrF0ITEZFsKSwspLCwsNJ2\n5u6ZG0TFDf8I9AyrXgP+x92XVrpzszxgAdHAiVXALKIZ1cutg2tmk4BnyhvdZ2Z+/ezNlR0SgNFd\n67Nu/PhYbQGajR9fhcq8UNlnJiIiVWNmuPtOPWlxJph9kGgkX/vweCasq5S7bwNGAdOBD4CH3H2e\nmY00s0vKe0mc/YqISO0Qp57UPu4+KW35XjO7Mu4B3P154JAy6+6ooO2FcfcrIiI1X6bRfU3C02lm\n9jPgIaIznXOAqdUQm4iI1HKZzqQ+IEpKqT7CH6dtc+DqpIISERGBzKP72lVnICIiImXFuSaFmR0K\nHA7slVrn7lOSCkpERARiJCkz+yXQHziUaG69bwOvAkpSIiKSqDhD0M8B+gCrwnx+RwP7JBqViIgI\n8ZLUV+F+p61m1hj4CChINiwREZF416TmmFkz4C/Av4AviGaOEBERSVSlScrdR4ant5nZ34Em7j47\n2bBERETij+47HTiJ6P6oVwElKRERSVyl16TM7M9EN/IuAj4ErjCzOLWkREREdkmcM6lvAYd7mPrb\nzP4CvJ9oVCIiIsQb3fcfojpQKQcAi5MJR0REZIdME8w+QXQNai9gnpnNDJt6AG9WQ2wiIlLLZeru\nu7XaohARESlHpglm/5l6bmatiMq8A/zL3VcnHZiIiEic0X1nEQ05Px/4HvAvMzsz6cBERETijO77\nFXCcu38MYGaticrBP5FkYCIiInFG99VJJajgk5ivExER2SVxzqReMLOpwINheShRyQ4REZFExUlS\n/wMMIZoWCeA+4NHEIhIREQkyJikzywOed/f/Ah6unpBEREQiGa8thTpSeWbWpJriERERKRGnu+9z\nYK6ZTQc2pla6+08Ti0pERIR4SerZ8BAREalWlV2TOgr4DPjA3RdVT0giIiKRCq9JmdnVwJPAuUTD\n0C+stqhERETIfCZ1LtDF3Tea2b7ANOAv1ROWiIhI5tF9m9x9I4C7f1pJWxERkd0u05nUgWb2eHhu\nwEFpy7j7dxONTEREar1MSeqsMsuqLyUiItUqVj0pERGRbNB1JhERyVlKUiIikrOUpEREJGfFKR//\nvJk1S1tuHupLiYiIJCrOmVRrd1+XWnD3tUCb5EISERGJxElS280sP7VgZu0TjEdERKREnCT1K+A1\nM5tkZvcCLwNXxz2AmQ0ws/lmttDMripn+3Azmxser4ZJbUVERCov1eHuU82sO9AzrPq5u38SZ+dm\nVofoJuB+wErgLTN7yt3npzVbApzi7p+b2QDgLqBHVd6EiIjUTJlmQe8UfnYBWhMlkyXA/mFdHN2B\nRe5e5O5bgIeAwekN3H2mu38eFmcCbav2FkREpKbKdCb1C+Ai4LZytjlwSoz9twWWpy2vIEpcFfkB\n8FyM/YqISC2QaVqki8LPk6sjEDPrA4wATqqO44mISO6r9JqUmTUARhIlDwdeAe5y900x9l8MpI8G\nzA/ryh6jC3AnMCAMcS/XP26/puT5gd16cWC3XjFCEBGRXFNYWEhhYWGl7czdMzcwewjYBEwOq4YD\ne7v70Ep3bpYHLCAaOLEKmAUMc/d5aW3aA/8Eznf3mRn25dfP3lzZIQEY3bU+68aPj9UWoNn48eyI\nKDM7DCr7zEREpGrMDHe3susrPZMiqs57eNryC2b27zgHdfdtZjYKmE40SOMed59nZiOjzX4nMBZo\nAUw0MwO2uHum61YiIlJLxElSc83sOHd/C8DMjgXmxD2Auz8PHFJm3R1pzy8GLo67PxERqT3iJKmj\ngDfNbElY7gjMM7M5RGdDXROLTkREarU4SWpw5U1ERER2vzgzTiw2syOA1FD0V9z9g2TDEhERiVeq\nYxTwCNFQ8vbAw2b2w6QDExERidPddwnQ3d03AJjZb4HXgYlJBiYiIhJnFnQD0m9Q2hLWiYiIJKrC\nMykzq+vuW4EHiEb3PRY2nQncVx3BiYhI7Zapu28W0NXdbzSzQnbMqXdp6p4pERGRJGVKUiVdeu4+\niyhpiYiIVJtMSWpfM/tpRRvd/eYE4hERESmRKUnlAY3QIAkREcmSTElqlbtfk2G7VFGH/HyKineq\nVFKugrYBcrr/AAAPlElEQVRtWbpiRcIRiYjktljXpGT3KCoujl1CpPVvxxNNCh9PQfvWLC366BtG\nJiKSmzIlqX7VFoXsZNNmYte4ArDDPk4uGBGRLKnwZl53X1OdgYiIiJQVZ8YJySC/oANmFushIiJV\nE2fuPsmgeFkRVSlrLyIi8elMSkREcpaSlIiI5CwlKRERyVlKUiIikrOUpEREJGcpSYmISM5SkhIR\nkZylJCUiIjlLSUpERHKWkpSIiOQsJakaqCrzCZoZ+QUdYu+7Q8H+sffboWD/5N7kHhBXVWKq7s9L\nZE+huftqoKrMJwhVm1OwaNnHsUuIVGf5kFyMqyoxgcqtiJRHZ1IiIpKzlKREaplc7BoVqYi6+4QG\neXmqd1WL5GLXqEhFlKSETdu2sW78+Fhtm8VsJyKyO6i7T3JSh/z8Ko2ME5GaSWdSkpOKiotjn92B\nzvBqgg4F+1O0LF73YkH71iwt+ijhiHIzJsjduJKgJCVSBR3y8ykqLs52GDvJ1biqIhevleViTJCb\ncVUlcUL85KkkJdUmv6ADxcuKsh3GLqnKGV51nt3lalxSeyR1X6CSlFSbqtxkXJUbjCU31YSzO8m+\nxJOUmQ0AbiEapHGPu/+unDYTgIHARuD77v5O0nGJSLJy9bpiribPXI0r2xJNUmZWB7gV6AesBN4y\ns6fcfX5am4HAQe7eycyOB24HeiQZl0i6mtANKfHlatdoLsa1/7778vHq1dVyrIokfSbVHVjk7kUA\nZvYQMBiYn9ZmMHA/gLu/aWZNzay1u+suQqkW6oYUKd/Hq1dnPXEmfZ9UW2B52vKKsC5Tm+Jy2ojU\nKlWdyT4X4xLZHTRwQiQHJTmT/a7I1bPOXOyyzcWYIHfjqoi5e3I7N+sBjHf3AWH5F4CnD54ws9uB\nF939b2F5PtCrbHefmSUXqIiIZJ2773QKnvSZ1FvAwWZWAKwChgLDyrR5GvgR8LeQ1NaVdz2qvOBF\nRKRmSzRJufs2MxsFTGfHEPR5ZjYy2ux3uvs0MzvVzD4kGoI+IsmYRERkz5Fod5+IiMiuqHGzoJvZ\nADObb2YLzeyqbMcDYGb3mNnHZvZutmNJMbN8M5thZh+Y2XtmdkW2YwIwswZm9qaZzQlxjct2TClm\nVsfMZpvZ09mOJcXMlprZ3PB5zcp2PCnhVpJHzGxe+B07Pgdi6hw+p9nh5+e58HtvZlea2ftm9q6Z\n/dXMcuI+BzP7cfg/mNW/DzXqTCrcPLyQtJuHgaHpNw9nKa6TgA3A/e7eJZuxpJjZ/sD+7v6OmTUC\n3gYGZ/uzAjCzhu7+pZnlAa8BV7h71v8Am9mVwLFAE3c/PdvxAJjZEuBYd1+b7VjSmdm9wEvuPsnM\n6gIN3f2LLIdVIvytWAEc7+7LK2ufYBxtgFeBQ919s5n9DZjq7vdnK6YQ1xHAg8BxwFbgOeBSd19S\n3bHUtDOpkpuH3X0LkLp5OKvc/VUgp/6IuPtHqemn3H0DMI8cuT/N3b8MTxsQXTfN+jcpM8sHTgXu\nznYsZRg59v/YzJoAJ7v7JAB335pLCSr4FrA4mwkqTR6wTyqZE33BzrbDgDfdfZO7bwNeBr6bjUBy\n6pd7N4hz87CUYWYdgP8HvJndSCKhW20O8BHwgru/le2YgD8C/0sOJMwyHHjBzN4ys4uzHUzQEVht\nZpNC19qdZrZ3toMq4xyiM4WscveVwB+AZUQTGaxz939kNyoA3gdONrPmZtaQ6Atau2wEUtOSlFRR\n6Op7FPhxOKPKOnff7u7HAPnA8WZ2eDbjMbPTgI/DmaeFR6440d27Ev0R+VHoWs62ukBX4LYQ25fA\nL7Ib0g5mVg84HXgkB2JpRtTbUwC0ARqZ2fDsRgWh2/93wAvANGAOsC0bsdS0JFUMtE9bzg/rpByh\ne+FR4AF3fyrb8ZQVuoheBAZkOZQTgdPD9Z8HgT5mltVrBinuvir8/BR4gqjLO9tWAMvd/V9h+VGi\npJUrBgJvh88s274FLHH3NaFb7XHghCzHBIC7T3L3bu7eG1hHdL2/2tW0JFVy83AYITOU6GbhXJBr\n38AB/gL8293/lO1AUsyslZk1Dc/3Bv6L0hMSVzt3v9rd27v7gUS/UzPc/XvZjAmiASbhTBgz2wfo\nT9RNk1XhZvzlZtY5rOoH/DuLIZU1jBzo6guWAT3MbC+LJjzsR3R9OOvMbN/wsz1wJjAlG3HUqLn7\nKrp5OMthYWZTgN5ASzNbBoxLXVTOYkwnAucC74XrPw5c7e7PZzMu4ADgvjD6qg7wN3efluWYclVr\n4IkwZVhd4K/uPj3LMaVcAfw1dK0tIUdu0g/XV74FXJLtWADcfZaZPUrUnbYl/Lwzu1GVeMzMWhDF\n9cNsDX6pUUPQRUSkZqlp3X0iIlKDKEmJiEjOUpISEZGcpSQlIiI5S0lKRERylpKUiIjkLCWpWsTM\nWqSVKlhlZivSlqt0z1woP9JpF+NpaGYv7so+0vZ1ZVVLHJhZPzN7opz1F5nZH3dHXFWMp9LP1Mwe\nMLOdZmA3s45mds43OObNoRTDb8us72Nmu2X2Cos8b2ZrzezxMtsOtKg0y0Izmxxmvk9tm2hmi8zs\nHTOrtuoBZnZQuHewou0NzOylcPOtJExJqhYJU68cE+ZT+z/g5tSyu2+t4r4ucvdFuxjSD4CHd3Ef\nhD9sPwX2+gYvr+hGwWq/gXAXP9ODiGbDiC38kR3h7ke5+9VlNvcFen7DWErx6GbM3wEXlLP598AN\n7t4Z+Ar4fohtENDW3TsBPwIm7o5YypOeGNNU+O/v7puAQmBIUjHJDkpStVepb4Fm9vPwjfrdMGtH\n6hvl+2b2oJn928weMrMGYdsrqW+3Znaamb0dzsqeD+v6hm/As83sXxXMgn0u8FRo3ybsc3aIoUdY\nf15YftfMrgvr8sK38j+a2TtEs5PvB7xiZtNDm4Fm9no49oOp44dY55vZv8hcxqWDmRWa2QIzuzq8\n9joz+1HaZ3aDmV1W5nP8hZldGp7/2cz+Hp7/l0U1llKFOcuLLf0zHRmO/YaZ3WVmN6cdpq+ZvWZm\nH5pZ6j1cD/QOn9+oMjGZmf0h/PvONbNUyYVniSY0nZ22DjM7kOgLxM/Cth5m1sGiIpnvmNnfLaqD\nlDqzmxjey3wzK3eeRXd/EdhYJq46wCnAk2HVfcAZ4flg4P7w2teA1mbWsszrh5rZ78Lz/zGzBeF5\nJzMrDM/7h9/LuWZ2h4UeAzNbbmbXm9nbwBlm1i20mQ1cmnaMo8xsVvgc3rGoYgBEv7fnlvdeZTdz\ndz1q4QMYB/w0PO9ONB1LfaAR0TxrRxB9O98OHBfa3UdUgBDgFaAL0dQ8RUB+WN8s/JyW9rqGhNlN\n0o7fAFiRtvxz4H/DcwuvaQv8B2hOVHOnkGi277wQ1+C01y8DGofn+4a2e4Xlq4lm4d6bqJRLh7D+\nUeDxcj6bi0K7JiGOD8J7PQiYFdrUARYDTcu89kSi6YkgKmY3M7yfa4imBio3tjKfaT7RVEJNiKY7\neo3orBfggbT9HwXMC8/7lfdewraziQrpEf69lgGtwue4poLX/Cb1b5327zk0PL8YeCQtnqfD885h\n3/Uq2GepGEMs/05b7gDMDs+fA7qnbSsEupTZX1vgtfD8CaJSM/sCFwK/LuffezLR9D6E9T9J29f7\nQI/w/Oa0OCYCQ8LzekD98DyPaGb8rP9frukPnUkJwEnAY+6+2aNyHU8CJ4dtS3xHPafJoW26nkQT\nrq4AcPd1Yf1rwITwrb6ph//ZafYD1qQtvwX8wMzGAkd5VPjweOCf7r7WoxmipxB98wbY5KVnbk+f\nwPcE4HDgdYuuLQwn+gN4OLDA3ZeGdn/N8Jn83d2/CHE8CZzk7ouBLyyqWjqQqCjc52Ve9xZwnEWT\n5G4Iy8cSfZ6vVBBbQZl9pN73Fx51wz5aZvuTAO7+HlF5h8qcRJhQ1aPJX18BuoVtca+rHA/8LTy/\nn9K/Bw+HfS8kSlK7dK0yLncvBlpYNB/f/iGOXuz4rA+j9L/3/ez4/YHwfsIZ2l7uPjOsfyCtzevA\nWDP7X6C9u28Ox94GbE/1LEhylKSkqsrrq9/pD527X0f0jbsRMNPMDirT5CvSriF51B3UG1hFNMHs\nsIr2nfb6ihjwnEfX2o5x9yPd/dK0bXGUfZ+p5XuIzohGEM0iX7pR9EdsJfA9ojOpV4jOINq7+4cV\nxHZZ2f1UEuemmO0qkv6auNfeMrVL32ZV2OenQCuzkgEI6aV1iildZK+isjszic58PyD6rE8mSqiv\np8VTkY0ZtgHg7pOJuiA3Ac9b6Xpd9T26PiUJUpISiP5zn2nRqKVGRNcDXgnbOprZseH58LT1Ka8T\nXQtpD2BmzcPPA939fXe/AZgNHJL+IndfDeyddo2gPVH3yd3AvcAxRN03vS2qDlqXaGBAYdhF2T8+\nXxB1j6Vi6mVmHcO+G5rZwUTdmKlSLkZUsqEi/c2sSfiWPpjozBCiej+DgKO94gqqrwA/Iyq5/SrR\nhf9UbaWKYks3K7zvJhbNIp6pbHfqc1gPNM4Qz9Bwbao10dlcKp6K/oivZ8fnCVEyODs8Pz+8t5Qh\n4b10JkomFQ3+KFWuxt23h9hS7+8CwjVKohI73wv7PQn4yN0/K2efrxJ91i8R/Z59G1gfzoDnEf17\ndwhtz2PH70+JsN+vzOz4sKrkWpOZdXT3Je4+gegaXuqa4X6oVl21UJISQnfeg0R/uF4nqqj6Qdg8\nD/ipmf2bqI//rtTLwms/AS4DngrdV5PD9p+FC/XvEP3BK6+ExD/YUeCtH5C6cH0m8OfQnTOWHX+A\nXvcdpUTKflu/C/iHmU0PMf0A+Fs4/mtAJ3f/KsT6PFEiWJnhY3mL6A/lHKJrQO+G97uJ6A90pnpE\nrxB1Z870qDz45vCa1Od1UdnY0t+Tuy8nGvX2VnjdYuDz9DZpUstzgLphkMCoMm0eJarJ9S7Rv8OV\n4UtCeftLeQo426IBMT2IEu3IEPMQ4Mq0tsUWDUR5CrjYyxkpamavE3Wv9jezZWbWJ2z6OXCVmS0E\n9iH6ggLwDLDSzD4EbgvHL88rRInx5XDcFez4rL8i+qyfMLO5wNfA3RW87wuBO8PvX3oF2uEWDR6a\nQ/TvlPr97gNMrSAm2Y1UqkMqFLroHvWolHsS++8GXObuFyWx/ySEEWlziAZtLE3wOPu4+8ZwBvkU\nMNHdc+6Popk9QDSIIleKi1YLM3uSKNn/J9ux1HQ6k5LKJPYtxqPy4q8mtf/dzcyOBD4EpiWZoILf\nhG/1c4H5uZigglr3Ldeim8YfUYKqHjqTEhGRnKUzKRERyVlKUiIikrOUpEREJGcpSYmISM5SkhIR\nkZylJCUiIjnr/wNRBmUj3gTArwAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "a = np.sort(tpm_low_alpha.predict(obama,output_type='probability')[0])[::-1]\n",
+ "b = np.sort(topic_model.predict(obama,output_type='probability')[0])[::-1]\n",
+ "c = np.sort(tpm_high_alpha.predict(obama,output_type='probability')[0])[::-1]\n",
+ "ind = np.arange(len(a))\n",
+ "width = 0.3\n",
+ "\n",
+ "def param_bar_plot(a,b,c,ind,width,ylim,param,xlab,ylab):\n",
+ " fig = plt.figure()\n",
+ " ax = fig.add_subplot(111)\n",
+ "\n",
+ " b1 = ax.bar(ind, a, width, color='lightskyblue')\n",
+ " b2 = ax.bar(ind+width, b, width, color='lightcoral')\n",
+ " b3 = ax.bar(ind+(2*width), c, width, color='gold')\n",
+ "\n",
+ " ax.set_xticks(ind+width)\n",
+ " ax.set_xticklabels(range(10))\n",
+ " ax.set_ylabel(ylab)\n",
+ " ax.set_xlabel(xlab)\n",
+ " ax.set_ylim(0,ylim)\n",
+ " ax.legend(handles = [b1,b2,b3],labels=['low '+param,'original model','high '+param])\n",
+ "\n",
+ " plt.tight_layout()\n",
+ " \n",
+ "param_bar_plot(a,b,c,ind,width,ylim=1.0,param='alpha',\n",
+ " xlab='Topics (sorted by weight of top 100 words)',ylab='Topic Probability for Obama Article')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here we can clearly see the smoothing enforced by the alpha parameter - notice that when alpha is low most of the weight in the topic distribution for this article goes to a single topic, but when alpha is high the weight is much more evenly distributed across the topics.\n",
+ "\n",
+ "__Quiz Question:__ How many topics are assigned a weight greater than 0.3 or less than 0.05 for the article on Paul Krugman in the **low alpha** model? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 52,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "+---------------------+-------------------------------+\n",
+ "| average predictions | topics |\n",
+ "+---------------------+-------------------------------+\n",
+ "| 0.629444444444 | art and publishing |\n",
+ "| 0.0176543209877 | Great Britain and Australia |\n",
+ "| 0.0161111111111 | international athletics |\n",
+ "| 0.0146913580247 | American college and politics |\n",
+ "| 0.0136419753086 | science and research |\n",
+ "| 0.0131481481481 | team sports |\n",
+ "| 0.0129012345679 | general politics |\n",
+ "| 0.010987654321 | Business |\n",
+ "+---------------------+-------------------------------+\n",
+ "[? rows x 2 columns]\n",
+ "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
+ "You can use sf.materialize() to force materialization.\n",
+ "8\n"
+ ]
+ }
+ ],
+ "source": [
+ "paul_krugman = gl.SArray([wiki_docs[int(np.where(wiki['name']=='Paul Krugman')[0])]])\n",
+ "paul_krugman_low_alpha = average_predictions(tpm_low_alpha, paul_krugman, 100)\n",
+ "print paul_krugman_low_alpha[(paul_krugman_low_alpha['average predictions'] > 0.3) |\n",
+ " (paul_krugman_low_alpha['average predictions'] < 0.05)\n",
+ " ]\n",
+ "print len(paul_krugman_low_alpha[(paul_krugman_low_alpha['average predictions'] > 0.3) |\n",
+ " (paul_krugman_low_alpha['average predictions'] < 0.05)\n",
+ " ])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ How many topics are assigned a weight greater than 0.3 or less than 0.05 for the article on Paul Krugman in the **high alpha** model? Use the average results from 100 topic predictions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 53,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "+---------------------+--------------------+\n",
+ "| average predictions | topics |\n",
+ "+---------------------+--------------------+\n",
+ "| 0.629444444444 | art and publishing |\n",
+ "| 0.010987654321 | Business |\n",
+ "+---------------------+--------------------+\n",
+ "[? rows x 2 columns]\n",
+ "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
+ "You can use sf.materialize() to force materialization.\n",
+ "2\n"
+ ]
+ }
+ ],
+ "source": [
+ "paul_krugman_high_alpha = average_predictions(tpm_high_alpha, paul_krugman, 100)\n",
+ "print paul_krugman_low_alpha[(paul_krugman_high_alpha['average predictions'] > 0.3) |\n",
+ " (paul_krugman_high_alpha['average predictions'] < 0.05)\n",
+ " ]\n",
+ "print len(paul_krugman_low_alpha[(paul_krugman_high_alpha['average predictions'] > 0.3) |\n",
+ " (paul_krugman_high_alpha['average predictions'] < 0.05)\n",
+ " ])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Changing the hyperparameter gamma\n",
+ "\n",
+ "Just as we were able to see the effect of alpha by plotting topic weights for a document, we expect to be able to visualize the impact of changing gamma by plotting word weights for each topic. In this case, however, there are far too many words in our vocabulary to do this effectively. Instead, we'll plot the total weight of the top 100 words and bottom 1000 words for each topic. Below, we plot the (sorted) total weights of the top 100 words and bottom 1000 from each topic in the high, original, and low gamma models."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now we will consider the following two models:\n",
+ " - tpm_low_gamma, a model trained with gamma = 0.02 and default alpha\n",
+ " - tpm_high_gamma, a model trained with gamma = 0.5 and default alpha"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 54,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "del tpm_low_alpha\n",
+ "del tpm_high_alpha\n",
+ "tpm_low_gamma = gl.load_model('topic_models/lda_low_gamma')\n",
+ "tpm_high_gamma = gl.load_model('topic_models/lda_high_gamma')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 55,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmclWX9//HXm1XJDaRUZmRAUVNxQ1JQFNQyxFzQHwiS\nW2VIkoUtZqZImaaZlanh9nXBfV/SzFzGDRQMxDQQl9hdQVQwZfv8/rjvGc6MM4d7YM6cw/B+Ph7n\nMfdynev+nDMz53Pu677u61JEYGZmVopaFDsAMzOz+jhJmZlZyXKSMjOzkuUkZWZmJctJyszMSpaT\nlJmZlayCJylJ/SVNlzRD0hn1lOknaYqkVyQ9WeiYzMxs3aBC3iclqQUwAzgImA9MAoZExPScMpsC\n44GDI2KepI4R8UHBgjIzs3VGoc+k9gJej4hZEbEMuA04olaZY4G7I2IegBOUmZlVKXSSKgPm5KzP\nTbfl2h7oIOlJSZMkHVfgmMzMbB3RqtgBkMTQAzgQ+BIwQdKEiHijuGGZmVmxFTpJzQM656yXp9ty\nzQU+iIjPgM8kPQ3sBtRIUpI8yKCZWTMWEaq9rdDNfZOAbpIqJLUBhgAP1CpzP9BHUktJ7YC9gWl1\nVRYRmR6jR4/OXLYpH45r3Y7Jca37MTmu0o2pPgU9k4qIFZJGAo+SJMRrI2KapOHJ7rgqIqZL+gfw\nMrACuCoi/lPIuMzMbN1Q8GtSEfEIsEOtbVfWWr8YuLjQsZiZ2bqlWY440a9fv2KHUCfHlV0pxgSO\nqyFKMSZwXA1RCjEV9GbexiQp1pVYzcysYSQRdXScKIUu6Ga2HujSpQuzZs0qdhhWZBUVFcycOTNz\neZ9JmVmTSL8pFzsMK7L6/g7qO5NqltekzMyseXCSMjOzkuUkZWZmJctJyszWe127duWJJ54odhhW\nBycpMyua8oouSCrYo7yiS7Ffoq0ld0E3s6KZN3sWF0xeWrD6z+zRpmB1W9PwmZSZWY6lS5fy4x//\nmLKyMsrLyxk1ahTLli0DkhEY7r33XgCee+45WrRowd///ncAnnjiCfbYY4866/zss8844YQT6NCh\nAzvvvDO///3v2Xrrrav3X3jhhXTr1o1NNtmE7t27c99991Xvu+GGG+jTpw+nn3467du3p1u3bkyY\nMIEbbriBzp07s+WWW3LjjTdWlz/ppJM49dRTGTBgABtvvDH77bcf7777LqNGjaJDhw7stNNOTJ06\nNdOxS4GTlJlZjvPOO4+JEyfy8ssvM3XqVCZOnMh5550HQN++famsrATg6aefZtttt+Xpp58G4Kmn\nnqp3GKFzzz2X2bNnM3PmTP75z39y0003Ia26Jahbt24899xzfPzxx4wePZpvf/vbvPvuu9X7J06c\nyO67787ChQsZOnQoQ4YM4cUXX+TNN99k3LhxjBw5kk8//bS6/J133sn555/PggULaNOmDb1796Zn\nz54sWLCAo48+mlGjRmU+drE5SZmZ5bjlllsYPXo0m2++OZtvvjmjR49m3LhxQJKknnrqKSBJUmee\neWb1+lNPPUXfvn3rrPPOO+/krLPOYpNNNqFTp06cdtppNfYfffTRbLHFFgAMGjSI7bbbjokTJ1bv\n79q1K8cffzySOOaYY5g7dy6jR4+mdevWfOMb36BNmza88caqKfgGDhzI7rvvTps2bRg4cCAbbrgh\nw4YNq37+Sy+9lPnYxeYkZWaWY/78+XTuvGqu1oqKCubPnw9A7969mTFjBu+99x5Tp07l+OOPZ86c\nOSxYsICJEyey//7711tneXl59XpuUx/AjTfeyB577EH79u1p3749r776Kh988EH1/qokArDhhhsC\n0LFjxxrbFi9eXG/52uu5ZVd37GJzkjIzy9GpU6caYwzOmjWLTp06AckH/J577smf//xnunfvTqtW\nrejduzeXXHIJ3bp1o0OHDvXWOXfu3Or12bNn11j+/ve/zxVXXMGHH37Ihx9+yM4779wkQ0gV89hZ\nOUmZmeUYOnQo5513Hh988AEffPABv/nNbzjuuOOq9++///5cdtll1U17/fr1q7Fel0GDBnHBBRew\naNEi5s2bx+WXX169b8mSJbRo0YKOHTuycuVKrrvuOl555ZW8Ma5tEql6/pocu6k5SZnZei+3E8Ov\nfvUrevbsya677spuu+1Gz549Oeuss6r39+3bl8WLF1c37VWt50tS55xzDmVlZXTt2pWDDz6YQYMG\n0bZtWwB23HFHfvKTn9CrVy+23HJLXn31Vfr06ZM53rrWs77eNTl2U/Mo6GbWJOoa/bq8ogvzZhdu\n+o6yzhXMnTWzYPWvqbFjx3L77bfz5JNPFjuUJudR0M1snTF31kwiomCPUklQ77zzDuPHjycieO21\n1/jDH/7AUUcdVeyw1gkeccLMrMCWLl3K8OHDmTlzJpttthlDhw5lxIgRxQ5rneDmPjNrEp700MDN\nfWZm1ow4SZmZWclykjIzs5LlJGVmZiXLScrMzEqWk5SZWQONGDGC3/72t41eNp9Zs2bRokULVq5c\nudZ1NdRJJ53EOeeck6ls165deeKJJxrt2L5PysyKpkt5ObPmzStY/RVlZczMGdi1sfz1r38tSNnV\naejwR82Bk5SZFc2sefNYdO65Bat/swLUvXLlSlq0cCNUU1ntOy3pcEkbp8s/lXSLpF0KH5qZWdOY\nPn06BxxwAO3bt2eXXXbhwQcfrN530kkn8YMf/IBDDz2UjTfemMrKyi80f1100UV06tSJ8vJyrr32\nWlq0aMFbb71V/fyqsk899RRbb701l1xyCVtssQVlZWVcf/311fU8/PDD9OjRg0033ZSKigrGjBmT\n+TV07dqViy++mN12242NN96Yk08+mffee48BAwawySabcPDBB/PRRx9Vl3/ggQfo3r07HTp04MAD\nD2T69OnV+6ZMmcKee+7JpptuypAhQ/jss89qHOtvf/tb9RxUffr04d///nfmOBsqy9eB8yLiE0m9\ngCOBu4Grsh5AUn9J0yXNkHRGHfv7SlokaXL6+FX28M3M1s7y5cs57LDD6N+/P++//z6XXnopw4YN\n4/XXX68uc+utt3L22WfzySefsO+++9Z4/iOPPMKf/vQnnnjiCd544w0qKyvzNsu98847fPLJJ8yf\nP59rrrmGU089tTp5bLTRRowbN46PPvqIhx56iLFjx/LAAw9kfi333HMPjz/+ODNmzOCBBx5gwIAB\n/O53v+ODDz5gxYoVXHrppQDMmDGDY489lksvvZT333+fQw45hMMOO4zly5ezbNkyBg4cyAknnMDC\nhQsZNGgQd999d/UxpkyZwne/+12uvvpqFi5cyPDhwzn88MNZtmxZ5jgbIkuSWpH+PAy4MiLuBtpm\nqVxSC+Ay4JvAzsBQSV+to+jTEdEjfZyXpW4zs8bw/PPPs2TJEs444wxatWrFAQccwLe+9S1uvfXW\n6jJHHHEEvXr1AqieYqPKnXfeyUknncRXv/pVNthgA85dTRNjmzZtOPvss2nZsiWHHHIIG220Ea+9\n9hqQzFW18847A9C9e3eGDBlSPT19Fj/84Q/p2LEjW221Ffvttx977703u+66a/U08lOmTAHgjjvu\n4Fvf+hYHHnggLVu25Kc//SmfffYZ48eP5/nnn2f58uWcdtpptGzZkqOPPpqvfe1r1ce4+uqrOeWU\nU+jZsyeSOO6442jbti3PP/985jgbIkuSelfSn4EhwEOS2gAtM9a/F/B6RMyKiGXAbcARdZRb/64G\nmllJmD9//hemc6+oqGBeToeO2vvzPX/rrbfOO0bh5ptvXuOaVrt27aqnc3/hhRc48MAD+cpXvsJm\nm23GlVde2aCp3LNOGz9//nwqKiqq90mivLycefPmMX/+fMrKymrUm1t21qxZ/OEPf6BDhw506NCB\n9u3bM3fuXObPn585zobIkqQGAS8Ah0fEQuDLwFn5n1KtDJiTsz433VZbb0kvSXpI0k4Z6zYzW2ud\nOnVizpw5NbbNnj27xgd1vua7rbba6gtTw69pL7xhw4Zx5JFHMm/ePBYtWsTw4cMLMihvp06dmDWr\n5jxec+bMoays7AuvB2pOd7/11ltz1llnsXDhQhYuXMiHH37I4sWLOeaYYxo9TsiTpCS1k9SOpLnv\nPuC/6fqHQON1god/AZ0jYneSpsH7GrFuM7O89t57b9q1a8dFF13E8uXLqays5G9/+xtDhw7N9PzB\ngwdz3XXXMX36dD799FPOO2/Nr1gsXryY9u3b07p1ayZOnMgtt9xSY39jJazBgwfz0EMP8eSTT7J8\n+XIuvvhiNthgA/bZZx969+5N69at+ctf/sLy5cu55557mDhxYvVzTz75ZMaOHVu9bcmSJTz88MMs\nWbKkUWKrLV8X9DeBqndkC+BTkma5DYF3gU4Z6p8HdM5ZL0+3VYuIxTnLf5d0haQO6VlbDbltvf36\n9aNfv34ZQjAzq1/r1q158MEHGTFiBOeffz7l5eWMGzeO7bbbDqj7LCp3W//+/TnttNM44IADaNmy\nJWeffTbjxo37wrWr+uTWdcUVV3D66aczcuRI+vbtyzHHHMOiRYvqLJuvntWV3X777bnpppsYOXIk\n8+fPZ/fdd+fBBx+kVaskJdxzzz1873vf41e/+hUDBgzg6KOPrn7unnvuydVXX83IkSN544032HDD\nDenTpw99+/Zd7XFzVVZWUllZudpyq51PStIVwGMRcU+6PhA4KCJGrrZyqSXwGnAQ8DYwERgaEdNy\nymwREe+my3sBd0RElzrq8nxSZuuwuuYRWldv5s1n+vTp7LLLLnz++ee+n6oODZ1PKkuS+ndE7FJr\n28sRsWvGgPoDfyZpWrw2In4naTgQEXGVpFOBEcAy4H/AqIh4oY56nKTM1mHNedLD++67jwEDBrBk\nyRJOPPFEWrVqVaPbtq1SiCT1GPAIcFO6aRgwICIOWvtws3OSMlu3NeckdcghhzBhwgRatWpFv379\nuPzyy2v0rLNVCpGkvgycB+xPco3qaeCciHivUSLOyEnKbN3WnJOUZdfQJJV37L70mtKPI2J444Vo\nZmaWTd6rehGxAvh6E8ViZmZWQ5ZR0F+UdCdwB1DdET4iHi5YVGZmZmRLUh2A5cBROdsCcJIyM7OC\nWm3HiVLhjhNm6zZ3nDBoeMeJLPNJbSnpVklz08fNkrZspHjNzIou35Tnzz77LDvuuGOmeqrmi7LG\nk+V26OtJup1vnz6eSbeZma2VLhVbIqlgjy4Va/99uk+fPkybNm31BVPr4xTvhZTlmtSWEfHXnPWx\nkk4pVEBmtv6YNftdIvvnf4Npx3cLV7k1iSxnUosk/b+qFUlHA4vylDczW+dMmTKF3Xbbjfbt2zN0\n6FCWLl0KfLEJb/LkydVTvA8ePJghQ4bUmEo+IuqdHr62mTNn0rdvXzbddFMOPvhgRo4cyXHHHVe9\nf/DgwWy11Va0b9+efv368Z///Kd630knncSpp57KgAED2Hjjjdlvv/149913GTVqFB06dGCnnXZi\n6tSp1eUbOr18vmM3pSxJ6jvA9yUtlLQAODndZmbWbNx55508+uij/Pe//2Xq1Kk1kktVE96yZcs4\n6qij+M53vsPChQsZOnQo9957b4168k0PX9uxxx5Lr169WLBgAaNHj2bcuHE1mgsHDBjAm2++yXvv\nvUePHj0YNmzYF2I+//zzWbBgAW3atKF379707NmTBQsWcPTRRzNq1Kga5bNOL5/l2E0l33xSXwKI\niLci4uCI6BARm0dE/4h4q+lCNDMrvB/96EdsscUWbLbZZhx22GG89NJLXygzYcIEVqxYwciRI2nZ\nsiUDBw5kr732qlEm3/TwuebMmcOLL77ImDFjaNWqFfvuuy+HH354jTInnngi7dq1o3Xr1pxzzjlM\nnTqVTz75pHr/wIED2X333aunh99www0ZNmwYkjjmmGO+8BqyTi+f5dhNJd+Z1BxJkyX9RdJQSe6y\nYmbNVu6AsLlTuud6++23vzC1eu3efPmmh881f/58OnTowAYbbFBnXStXruQXv/gF3bp1Y7PNNqNr\n165IqjGdfNbp4htaPsuxm0q+JLU5cCLwKnAI8KSkOZJul3RaUwRnZlZKttpqK+bVmv+q9tTzDalr\n4cKFfPbZZ3XWdfPNN/Pggw/yxBNPsGjRImbOnElENMm9ZsU8dm31JqlIvBwRYyPieJJR0C8E9gQu\nbqoAzcxKRe/evWnZsiWXX345K1as4P77768xtXpDdO7cmZ49e3LuueeybNkyJkyYwIMPPli9f/Hi\nxbRt25b27duzZMkSzjzzzAZ3b1/TpNIYx24s+a5J9ZA0UtItkiYCVwAbAd8D2jdVgGZmhZb1A7h1\n69bcc889XHPNNbRv355bbrmFww47LO9U8fnqvvnmmxk/fjwdO3bknHPOYciQIdV1HX/88XTu3Jmy\nsjK6d+/OPvvs07AXVevYDZlevjGO3VjqHRZJ0kpgMvBHkindlzVlYHXE42GRzNZhdU4fX7Els2YX\n7l6mis5bMHPWOwWrH6BXr16MGDGCE044Ya3rGjJkCDvuuCOjR49uhMhKU6NNeiipC7BP+tiTZHr3\nicAEYEJEzG+0qDNwkjJbtzWXsfuefvppdthhBzp27MhNN93ED37wA9566601mon3xRdfpEOHDnTt\n2pV//OMfHHXUUUyYMIHddtutAJGXhkab9DAiZgIzgVvSCjYi6UhxIdAVaNkYAZuZrUtee+01Bg8e\nzKeffso222zD3XffvcZTxb/zzjscddRRLFy4kPLycsaOHdusE9SayHcmtSGwN6vOpvYC3gbGA89F\nxE1NFWQaj8+kzNZhzeVMytZOYzb3fQhMImneew54PiI+btxws3OSMlu3OUkZNGJzH7B5RKxszODW\nVtYeOGWdK5g7a2ZhgzEzs4LLd02qpBIUwAWTl2Yqd2aPNgWOxMzMmkKWAWbNzMyKIst8UmZma62i\nosITAhoVFRUNKp83SUnqCxwJVI2oOA+4PyIq1yQ4M1t/zZw5s9gh2Doo37BIFwHnAFOBq9LHVOBs\nSb9vmvBKX3lFl8xTWZdXdCl2uGZm65R8Z1JHRsT2tTdKugGYAfysYFGtQ+bNnlVyHTrKK7owb/as\nzOXdG9LMSlW+JLVU0m4RMbXW9l2BzwsYk62lhiROcG9IMytd+ZLU94AbJAVQNclJZyDSfWZmZgWV\n7z6p54Hd04FmqztOpGP6mZmZFVyW+6Q2rfVoEEn9JU2XNEPSGXnKfU3SMklHNfQYZmbWPNV7JiXp\nAGAsMJ+k6zlAuaStgFMi4snVVS6pBXAZcFBazyRJ90fE9DrK/Q74xxq9CjMza5byXZO6DDg0It7I\n3ShpO+B+YKcM9e8FvB4Rs9Ln3gYcAUyvVe6HwF3A1zLGbeughvQ6dI9DM4P8SaoN8FYd2/+b7sui\njFWdLgDmkiSuapI6kXR3P0BSjX3WvJRid30zK235ktRNwARJt7Aq0WwNHJvuayx/AnKvVdU7bspj\nY39dvbxNz75s07NvI4Zh6yuf4Zk1vcrKSiorK1dbLl/vvjGSHiBpnuuebp4HjIiIyRnjmEfSbb1K\nOauub1XpCdymZFCvjsAhkpZFxAO1K/v6KedkPKxZdj7DM2t6/fr1o1+/ftXrY8aMqbNc3rH7ImIK\nMGUt4pgEdJNUQTKr7xBgaK1jbFO1LOk64MG6EpSZma1/1miqDkn3ZSkXESuAkcCjwKvAbRExTdJw\nSd+v6ylrEo+ZmTVP+bqg19d7T9Tq/JBPRDwC7FBr25X1lP1O1nrNzKz5y9fc92/gBeruyNC+MOGY\nmZmtki9JvQYcFxFv1t4haU4d5c2skXgke7NEviT1G+q/H8rTdJgVkEeyN0vk64J+a559txUmHDMz\ns1XWqHefmZlZU3CSMjOzkuUkZWZmJSvviBMAkloDJwN9SG62fRa4JiKWFTg2MzNbz602SQHXkySn\ncen6UGA/koFmzczMCiZLkuoRETvmrP9d0rRCBWRmZlYlyzWplyXtUbUiaXfgpcKFZGZmlshyJvVV\nkmnfq2bo3Q54RdIkICKi5CYqbNuyJcnMH9lUlJUxc+7cAkZkZmZrIkuSGlzwKBrZ5ytWsOjcczOX\n36wBZddGQ5KnE6eZWYYkFRGvSfoqSe8+gGcjYnphw2qeGpI8mypxmpmVstVek5I0AriPpJlve+Be\nScMLHZiZmVmW5r4RwNci4hMASb8BngPqnBPKzMyssWTp3Sfg85z1z6l7jikzM7NGleVM6hZgvKS7\n0vWjgZsKF5I1NXfoMLNSlW/6eEXiAklPsarjxGkRMaFpwrOm4A4dZlaq8p1J/QvoARAR44HxTRKR\nGb7XzcwS+ZKUrztZ0ZTqvW5m1rTyJakvSzqtvp0RcWkB4jEzM6uWr3dfS6Aj8OV6Hma2nimv6IKk\nTI/yii7FDteagXxnUm9HxDlNFomZlbx5s2dxweSlmcqe2aNNgaOx9UG+MylfkzIzs6LKl6QObrIo\nzMzM6lBvkoqI95syELN1QVXX+CyPLuXlxQ7XbJ2X72beVhGxvCmDMSt1pXrjs0cNseYqX8eJiUAP\nSddHxIlNFI+ZrYFSTZ5maytfkmojaTCwn6TDa++MiAcKF5aZWXblFV2YN3tWprJlnSuYO2tmYQOy\nRpMvSZ0KfBvYDBhUa18AmZKUpP7An0iuf10bERfW2n848BtgJbAMGBURz2WKvpG0bUP2ppLOWzBz\n1jsFjsjMGsJd45uvepNURDwFPCXpxYhYo7mjJLUALgMOAuYDkyTdX2tm38eqzsok7QLcAey4Jsdb\nU58vhZiWrax2fLewwZg1Ex5/0RpDlqk6/k/SD4D90/WngKszdqrYC3g9ImYBSLoNOAKoTlIR8WlO\n+Y1IzqjMbB3n8RetMWRJUpcBXwL+L13/NrAH8P0Mzy0D5uSszyVJXDVIOhK4gGS4pUMz1GtmZuuB\nLEmqV0TslrP+qKSpjRlERNwH3CepD3Ae8I26yj029tfVy9v07Ms2Pfs2Zhhmth5wd/3SUFlZSWVl\n5WrLZUlSKyV1iYiZAJK6kL1Jbh7QOWe9PN1Wp4h4VtI2kjpExMLa+79+iocSNLO14+76paFfv370\n69even3MmDF1lss3LFKVM4BnJD0m6XGSa1I/yxjHJKCbpApJbYAh1OoVKGnbnOUeQJu6EpSZ2bqk\nISPGe9T4+q32TCoiHpW0Pat63E2LiP9lqTwiVkgaCTzKqi7o0yQNT3bHVcDRko4HlgL/AwavyQsx\nMyslDekWD+4aX58szX2kSWnymhwgIh4Bdqi17cqc5YuAi9ak7uasIfduge/fMrPmKVOSsqbXkHu3\nwPdvmVnzlOWalJmZWVGsNklJul3SN9WQtidrtqqaITNNVVGx5Xofl5mtnSzNfdcB3wEuk3Q7cH1E\nvFHYsKxUleoQUqUal5mtndWeSUXEIxFxDMlIEe8AT0p6WtJxknxNy8zMCibTNSlJ7YFjgeOAl4Er\ngX2ARwoXmpmZre9WeyYk6U5gF+Bm4OiIqBoj5GZJUwoZnJmZrd+yNNddRTKdRlRtqJpaPiL2KFxo\nZma2vsvS3HdhboJKTSxEMGZmZrnqPZOS9BVgK2DDdDLCqi7omwDtmiA2MzNbz+Vr7juUpOt5OXBF\nzvZPgLMLGZSZmRnknz7+OuA6SYMj4o4mjMmsWfD4i2ZrL19z39CIuBXYStJptfdHxKUFjcxsHefx\nF60hPBlj3fI197VPf3ZsikDMzNZnTTUZY3lFF+bNnpWpbFnnCubOmrnGx2oM+Zr7rkh/+vqTWTPS\nkGZIN0E2Pw2Z56oU5rjK19x3Sb4nRsTpjR+OmRWaxzm0dUm+5r5XmywKM1vv+QzP6pKvue/apgzE\nzNZvPsOzuuRr7vtDRPxE0r1A7REniIijChqZmZmt9/I1992e/rysKQIxMzOrLV9z38T05+OSWgPb\nkZxRvR4Ry5soPjOzovF1suLLMlVHf5KR0GeTjN9XLunkiHi00MGZmRWTr5MVX5apOv4EfD0iZgBI\n2h64H9ixkIGZmZllmapjcVWCAkiXlxQuJDMzy6eqGTLLo0vFlsUOd63k6913eLo4UdIDwB0k16QG\nAS80QWxmZlaH9akZMl9z36Cc5Y+Ab6bLnwAbFywiMzOzVL7efcc1ZSBmZma1Zend1xY4EdgZ2KBq\ne0R8v3BhmZmZZes4cSPQBfgWybWobYHPChiTmZkZkC1JbR8RZ5L08rsW6A/sVdiwzMzMsiWpZenP\nRZJ2JOk08ZWsB5DUX9J0STMknVHH/mMlTU0fz0raJWvdZmbWvGW5mfdaSe2B0cA/gHbAOVkql9SC\nZOy/g4D5wCRJ90fE9JxibwH7R8RH6egWVwO9GvAazMysmVptkoqIK9PFJ4HODax/L5Kx/mYBSLoN\nOAKoTlIR8XxO+eeBsgYew8zMmqnVNvdJai/pj5ImSnpB0sXpmVUWZcCcnPW55E9C3wP+nrFuMzNr\n5rJck7oN+BgYBnyb5Gbe2/M+Yw1IOgA4CfjCdSszM1s/ZbkmVRYRo3PWx0h6JWP986jZRFiebqtB\n0q4kI633j4gP66vssbG/rl7epmdftunZN2MYZmbWUG1btsw+VUlZGTPnzs1cd2VlJZWVlastlyVJ\nPS7p/0XEXQCSjgL+mTGOSUA3SRXA28AQYGhuAUmdgbuB4yLizXyVff2UTP01zMysEXy+YgWLzj03\nU9ktzj83c0KDL86/NWbMmDrL5Rtg9kOSAWUF/FDS8pznLAJGrS6IiFghaSTwKEnT4rURMU3S8GR3\nXAWcDXQArlDyCpdFhO/DMjNbhzRk0FvIPvBtvjOpjtkPV7+IeATYoda2K3OWTwZOboxjmZlZ85Jv\ngNkVVcuSBgD7p6uVaeIxMzMrqCxd0H8L/Jzkptu3gJ9LOq/QgZmZmWXpOHEYsEfVmZWk/wMmA78q\nZGBmZmZZ7pMC2CRn2RMemplZk8hyJnURMFnS4yQ9/fqR9MgzMzMrqLxJKu0S/jjJuH17p5vPiYgv\n3JBrZmbW2PImqYgISf+MiO7APU0Uk5mZGZDtmtRLkvYoeCRmZma1ZLkmtQfJPFBvAktIrktFRPQo\naGRmZraCEh0KAAARKElEQVTey5KkDi94FGZmZnXIN3ZfW5LhiroB/wauzx2FwszMrNDyXZO6HugD\nvA4cCVzcFAGZmZlVydfc1z0idgGQdBXwQtOEZGZmlsh3JrWsaiEiluUpZ2ZmVhD5zqR2k7QwXRaw\ncbpe1buvQ8GjMzOz9Vq+JNWmyaIwMzOrQ6b5pMzMzIoh6yjoZmZmTc5JyszMSpaTlJmZlax8I058\nCERdu3DvPjMzawL5evd1bLIozMzM6pC5d5+kDsAGOZvmFyooMzMzyHBNStKhkmYAc0mGRpoLPFHo\nwMzMzLJ0nPgtsC/wWkRsDXwTeKagUZmZmZEtSS2PiPeBFpIUEf8E9ipwXGZmZpkmPfxI0kbAs8CN\nkt4D/lfYsMzMzLKdSR1JkpR+DFQC84BvFTAmMzMzIFuSOjMiVkTEsoi4NiIuAU4vdGBmZmZZklT/\nOrYd2tiBmJmZ1VZvkpI0XNIUYAdJk3MerwPTsh5AUn9J0yXNkHRGHft3kDRe0meSfIZmZmbV8nWc\nuAN4HLgA+EXO9k8i4r0slUtqAVwGHERy8+8kSfdHxPScYguAH5Jc+zIzM6tW75lURHwYEW9ExCCS\nkSa+kT6+3ID69wJej4hZ6RT0twFH1DrOBxHxL2B5g6M3M7NmLcuIE6cCdwKd08cdkn6Qsf4yYE7O\n+tx0m5mZ2WpluU9qOLBXRCwGkHQ+MB64opCBmZmZZUlSApbmrC9Lt2Uxj+Tsq0p5um2NPDb219XL\n2/TsyzY9+65pVWZmVmTnnnvuasvkm0+qVUQsB8YBL0i6O901ELghYwyTgG6SKoC3gSHA0Dzl8ya/\nr59yTsbDmplZqctNUmPGjKmzTL4zqYlAj4i4SFIl0CfdfkpETMoSQESskDQSeJTk+te1ETFN0vBk\nd1wlaQvgRWBjYKWkHwE7VTUvmpnZ+itfkqo+q4mIiSRJq8Ei4hFgh1rbrsxZfhfYek3qNjOz5i1f\nkvpyvptr0+GRzMzMCiZfkmoJbET2ThJmZmaNKl+Sejsifp1nv5mZWUHlu5nXZ1BmZlZU+ZLUQU0W\nhZmZWR3yjd23sCkDMTMzqy3LfFJmZmZF4SRlZmYly0nKzMxKlpOUmZmVLCcpMzMrWU5SZmZWspyk\nzMysZDlJmZlZyXKSMjOzkuUkZWZmJctJyszMSpaTlJmZlSwnKTMzK1lOUmZmVrKcpMzMrGQ5SZmZ\nWclykjIzs5LlJGVmZiXLScrMzEqWk5SZmZUsJykzMytZTlJmZlaynKTMzKxkOUmZmVnJKniSktRf\n0nRJMySdUU+ZSyW9LuklSbsXOiYzM1s3FDRJSWoBXAZ8E9gZGCrpq7XKHAJsGxHbAcOBsYWMyczM\n1h2FPpPaC3g9ImZFxDLgNuCIWmWOAG4EiIgXgE0lbVHguMzMbB1Q6CRVBszJWZ+bbstXZl4dZczM\nbD3kjhNmZlayFBGFq1zqBZwbEf3T9V8AEREX5pQZCzwZEben69OBvhHxbq26CheomZkVXUSo9rZW\nBT7mJKCbpArgbWAIMLRWmQeAU4Hb06S2qHaCgrqDNzOz5q2gSSoiVkgaCTxK0rR4bURMkzQ82R1X\nRcTDkgZIegNYApxUyJjMzGzdUdDmPjMzs7XR7DpOZLl5uKlJulbSu5JeLnYsVSSVS3pC0quS/i3p\ntGLHBCCpraQXJE1J4xpd7JiqSGohabKkB4odSxVJMyVNTd+vicWOp4qkTSXdKWla+je2dwnEtH36\nPk1Of35UCn/3kkZJekXSy5JultSm2DEBSPpR+j9Y1M+HZnUmld48PAM4CJhPck1sSERML3JcfYDF\nwI0RsWsxY6kiaUtgy4h4SdJGwL+AI4r9XgFIahcRn0pqCTwHnBYRRf8AljQK2BPYJCIOL3Y8AJLe\nAvaMiA+LHUsuSdcDT0XEdZJaAe0i4uMih1Ut/ayYC+wdEXNWV76AcXQCngW+GhFLJd0OPBQRNxYr\npjSunYFbga8By4G/A6dExFtNHUtzO5PKcvNwk4uIZ4GS+hCJiHci4qV0eTEwjRK5Py0iPk0X25Jc\nNy36NylJ5cAA4Jpix1KLKLH/Y0mbAPtFxHUAEbG8lBJU6uvAm8VMUDlaAl+qSuYkX7CLbUfghYj4\nPCJWAE8DRxUjkJL6424EWW4etlokdQF2B14obiSJtFltCvAO8M+ImFTsmIA/Aj+jBBJmLQH8U9Ik\nSScXO5hUV+ADSdelTWtXSdqw2EHVcgzJmUJRRcR84A/AbJKBDBZFxGPFjQqAV4D9JLWX1I7kC9rW\nxQikuSUpa6C0qe8u4EfpGVXRRcTKiNgDKAf2lrRTMeORdCjwbnrmqfRRKvaNiB4kHyKnpk3LxdYK\n6AFcnsb2KfCL4oa0iqTWwOHAnSUQy2YkrT0VQCdgI0nHFjcqSJv9LwT+CTwMTAFWFCOW5pak5gGd\nc9bL021Wh7R54S5gXETcX+x4akubiJ4E+hc5lH2Bw9PrP7cCB0gq6jWDKhHxdvrzfeBekibvYpsL\nzImIF9P1u0iSVqk4BPhX+p4V29eBtyJiYdqsdg+wT5FjAiAirouInhHRD1hEcr2/yTW3JFV983Da\nQ2YIyc3CpaDUvoED/B/wn4j4c7EDqSKpo6RN0+UNgW8ARe3MERG/jIjOEbENyd/UExFxfDFjgqSD\nSXomjKQvAQeTNNMUVXoz/hxJ26ebDgL+U8SQahtKCTT1pWYDvSRtIEkk79W0IscEgKQvpz87AwOB\nW4oRR6FHnGhS9d08XOSwkHQL0A/YXNJsYHTVReUixrQvMAz4d3r9J4BfRsQjxYwL2Aq4Ie191QK4\nPSIeLnJMpWoL4N50yLBWwM0R8WiRY6pyGnBz2rT2FiVyk356feXrwPeLHQtAREyUdBdJc9qy9OdV\nxY2q2t2SOpDE9YNidX5pVl3QzcyseWluzX1mZtaMOEmZmVnJcpIyM7OS5SRlZmYly0nKzMxKlpOU\nmZmVLCep9YikDjlTFbwtaW7OeoPumUunH9luLeNpJ+nJtakjp65RDZ3iQNJBku6tY/t3Jf2xMeJq\nYDyrfU8ljZP0hRHYJXWVdMwaHPOSdCqG82ttP0BSo4xeocQjkj6UdE+tfdsomZplhqSb0pHvq/Zd\nIel1SS9JarLZAyRtm947WN/+tpKeSm++tQJzklqPpEOv7JGOp/ZX4JKq9YhY3sC6vhsRr69lSN8D\n7ljLOkg/2E4HNliDp9d3o2CT30C4lu/ptiSjYWSWfsieFBG7RMQva+0+EOi9hrHUEMnNmBcCJ9Sx\n+/fA7yJie+B/wIlpbIcBZRGxHXAqcEVjxFKX3MSYo97ff0R8DlQCgwoVk63iJLX+qvEtUNLP02/U\nL6ejdlR9o3xF0q2S/iPpNklt033PVH27lXSopH+lZ2WPpNsOTL8BT5b0Yj2jYA8D7k/Ld0rrnJzG\n0Cvd/u10/WVJv023tUy/lf9R0ksko5N/BXhG0qNpmUMkjU+PfWvV8dNYp0t6kfzTuHSRVCnpNUm/\nTJ/7W0mn5rxnv5M0otb7+AtJp6TLf5H0j3T5G0rmWKqamLOu2HLf0+HpsSdIulrSJTmHOVDSc5Le\nkFT1Gi4A+qXv38haMUnSH9Lf71RJVVMu/I1kQNPJOduQtA3JF4ifpvt6SeqiZJLMlyT9Q8k8SFVn\ndlekr2W6pDrHWYyIJ4ElteJqAewP3JduugE4Ml0+Argxfe5zwBaSNq/1/CGSLkyXfyLptXR5O0mV\n6fLB6d/lVElXKm0xkDRH0gWS/gUcKalnWmYycErOMXaRNDF9H15SMmMAJH+3w+p6rdbIIsKP9fAB\njAZOT5f3IhmOpQ2wEck4azuTfDtfCXwtLXcDyQSEAM8Au5IMzTMLKE+3b5b+fDjnee1IRzfJOX5b\nYG7O+s+Bn6XLSp9TBvwXaE8y504lyWjfLdO4jsh5/mxg43T5y2nZDdL1X5KMwr0hyVQuXdLtdwH3\n1PHefDctt0kax6vpa90WmJiWaQG8CWxa67n7kgxPBMlkds+nr+fXJEMD1Rlbrfe0nGQooU1Ihjt6\njuSsF2BcTv27ANPS5YPqei3pvsEkE+mR/r5mAx3T93FhPc/5TdXvOuf3OSRdPhm4MyeeB9Ll7dO6\nW9dTZ40Y01j+k7PeBZicLv8d2CtnXyWwa636yoDn0uV7Saaa+TLwHWBMHb/vm0iG9yHd/uOcul4B\neqXLl+TEcQUwKF1uDbRJl1uSjIxf9P/l5v7wmZQB9AHujoilkUzXcR+wX7rvrVg1n9NNadlcvUkG\nXJ0LEBGL0u3PAZem3+o3jfQ/O8dXgIU565OA70k6G9glkokP9wYej4gPIxkh+haSb94An0fNkdtz\nB/DdB9gJGK/k2sKxJB+AOwGvRcTMtNzNed6Tf0TEx2kc9wF9IuJN4GMls5YeQjIp3Ee1njcJ+JqS\nQXIXp+t7kryfz9QTW0WtOqpe98eRNMPeVWv/fQAR8W+S6R1Wpw/pgKqRDP76DNAz3Zf1usrewO3p\n8o3U/Du4I617BkmSWqtrlVlFxDygg5Lx+LZM4+jLqvd6R2r+vm9k1d8PpK8nPUPbICKeT7ePyykz\nHjhb0s+AzhGxND32CmBlVcuCFY6TlDVUXW31X/igi4jfknzj3gh4XtK2tYr8j5xrSJE0B/UD3iYZ\nYHZofXXnPL8+Av4eybW2PSKie0SckrMvi9qvs2r9WpIzopNIRpGvWSj5EJsPHE9yJvUMyRlE54h4\no57YRtSuZzVxfp6xXH1yn5P12lu+crn71IA63wc6StUdEHKn1plHzUn26pt253mSM99XSd7r/UgS\n6viceOqzJM8+ACLiJpImyM+BR1Rzvq42kVyfsgJykjJI/rkHKum1tBHJ9YBn0n1dJe2ZLh+bs73K\neJJrIZ0BJLVPf24TEa9ExO+AycAOuU+KiA+ADXOuEXQmaT65Brge2IOk+aafktlBW5F0DKhMq6j9\n4fMxSfNYVUx9JXVN624nqRtJM2bVVC4imbKhPgdL2iT9ln4EyZkhJPP9HAbsFvXPoPoM8FOSKbef\nJbnwXzW3Un2x5ZqYvu5NlIwinm/a7qr34RNg4zzxDEmvTW1BcjZXFU99H+KfsOr9hCQZDE6Xj0tf\nW5VB6WvZniSZ1Nf5o8Z0NRGxMo2t6vWdQHqNkmSKnePTevsA70TEgjrqfJbkvX6K5O/sm8An6Rnw\nNJLfd5e07LdZ9fdTLa33f5L2TjdVX2uS1DUi3oqIS0mu4VVdM/wKnquuSThJGWlz3q0kH1zjSWZU\nfTXdPQ04XdJ/SNr4r656Wvrc94ARwP1p89VN6f6fphfqXyL5wKtrConHWDXB20FA1YXrgcBf0uac\ns1n1ATQ+Vk0lUvvb+tXAY5IeTWP6HnB7evzngO0i4n9prI+QJIL5ed6WSSQflFNIrgG9nL7ez0k+\noPPNR/QMSXPm85FMD740fU7V+/Xd2rHlvqaImEPS621S+rw3gY9yy+SoWp8CtEo7CYysVeYukjm5\nXib5PYxKvyTUVV+V+4HBSjrE9CJJtMPTmAcBo3LKzlPSEeV+4OSoo6eopPEkzasHS5ot6YB018+B\nMyTNAL5E8gUF4EFgvqQ3gMvT49flGZLE+HR63Lmseq//R/Je3ytpKvAZcE09r/s7wFXp31/uDLTH\nKuk8NIXk91T1930A8FA9MVkj8lQdVq+0ie6uSKZyL0T9PYEREfHdQtRfCGmPtCkknTZmFvA4X4qI\nJekZ5P3AFRFRch+KksaRdKIolclFm4Sk+0iS/X+LHUtz5zMpW52CfYuJZHrxZwtVf2OT1B14A3i4\nkAkq9Zv0W/1UYHopJqjUevctV8lN43c6QTUNn0mZmVnJ8pmUmZmVLCcpMzMrWU5SZmZWspykzMys\nZDlJmZlZyXKSMjOzkvX/AbawIRe6AqQgAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNW5//HPw66RZRAFYWQgIorghqigGEdNVHBBNCKI\nG/ozuHBNSHKvGhfwRmM0anKNWzReRVzirqBEURE1bqMBcUVwYdgEBEQFr7I9vz/q9NDTzPTUDFPd\nDfN9v179mq6qc6qe7pnpp+vUqXPM3RERESlEjfIdgIiISHWUpEREpGApSYmISMFSkhIRkYKlJCUi\nIgVLSUpERApW4knKzI40s5lmNsvMLqymzI1mNtvM3jGzvWqqa2ZFZjbZzD42s2fNrHVY/1Mze9vM\nZpjZW2Z2SFqd3mb2btjXX5J8zSIiUj8STVJm1gi4CTgC6AkMM7NdM8oMAHZy952BkcBtMepeBDzv\n7rsAU4CLw/ovgaPdfU/gDGB82qFuBc5y9+5AdzM7op5froiI1LOkz6T2A2a7e7m7rwH+AQzKKDMI\nuAfA3d8EWptZ+xrqDgLGhefjgONC/Rnuvig8/wBoYWZNzawD0NLd3wp17knVERGRwpV0kuoEzEtb\nnh/WxSmTrW57d18MEJLS9pkHNrOfA9NCgusU6meLQ0RECkyTfAdQBatDnUpjO5lZT+Bq4Gf1EpGI\niORF0klqAdA5bbk4rMsss2MVZZplqbvIzNq7++LQlLckVcjMioHHgFPdfU4Nx9iImWkwQxGRHHP3\nKk9Qkm7uewvoZmYlZtYMGApMyCgzATgNwMz6AitCU162uhOIOkYAnA48Geq3AZ4CLnT3N1IHCE2C\nX5vZfmZm4XhPVhe0u9f4GDNmTKxyuXgoFsWyOcejWBRLNomeSbn7OjMbBUwmSoh3uvtHZjYy2uy3\nu/skMxtoZp8Aq4AR2eqGXV8DPGRmZwLlwJCw/nxgJ+ByMxtD1Ax4uLsvDdvuBloAk9z9mSRfu4iI\nbLrEr0mFZLBLxrq/ZSyPils3rF8O/LSK9VcBV1Wzr38Du8cOXERE8k4jTtRRaWlpvkOooFiqpliq\nV0jxKJaqKZaI1dQe2NCYmes9ERHJHTPDq+k4UYhd0EVkC9SlSxfKy8vzHYbkUUlJCXPmzKlVHZ1J\nZdCZlEgywrflfIcheVTd30C2MyldkxIRkYKlJCUiIgVLSUpERAqWkpSINHhdu3ZlypQp+Q5DqqAk\nJSJ5U1zSBTNL7FFc0iXfL1E2kbqgi0jeLJhbztXTVie2/4t7N0ts35IbOpMSEUmzevVqfvWrX9Gp\nUyeKi4sZPXo0a9asAaKRFx5//HEAXn31VRo1asQ///lPAKZMmcLee+9d5T6///57Tj/9dNq2bUvP\nnj3505/+xI47bpiY4ZprrqFbt260atWKXr168cQTT1RsGzduHP379+fXv/41RUVFdOvWjddff51x\n48bRuXNnOnTowD333FNRfsSIEZx//vkMHDiQli1bctBBB7F48WJGjx5N27Zt2W233ZgxY0asYxcC\nJSkRkTRXXnklZWVlvPvuu8yYMYOysjKuvPJKAA4++GCmTp0KwMsvv8xOO+3Eyy+/DMBLL71U7fBB\nY8eOZe7cucyZM4fnnnuOe++9l2hChki3bt149dVX+eabbxgzZgynnHIKixcvrtheVlbGXnvtxfLl\nyxk2bBhDhw7l7bff5tNPP2X8+PGMGjWK7777rqL8ww8/zB/+8AeWLVtGs2bN6NevH3369GHZsmWc\ncMIJjB49Ovax801JSkQkzf3338+YMWPYdttt2XbbbRkzZgzjx48HoiT10ksvAVGSuvjiiyuWX3rp\nJQ4++OAq9/nwww9zySWX0KpVKzp27MgFF1xQafsJJ5xA+/btATjxxBPZeeedKSsrq9jetWtXTjvt\nNMyMk046ifnz5zNmzBiaNm3Kz372M5o1a8Ynn3xSUX7w4MHstddeNGvWjMGDB7PVVlsxfPjwivrv\nvPNO7GPnm5KUiEiahQsX0rnzhvlWS0pKWLhwIQD9+vVj1qxZLFmyhBkzZnDaaacxb948li1bRllZ\nGT/5yU+q3WdxcXHFcnpTH8A999zD3nvvTVFREUVFRXzwwQcsXbq0YnsqiQBstdVWALRr167SupUr\nV1ZbPnM5vWxNx843Jakc6FLSIVZPpC4lHfIdqkiD17Fjx0pjDJaXl9OxY0cg+oDfZ599+J//+R96\n9epFkyZN6NevHzfccAPdunWjbdu21e5z/vz5Fctz586t9PwXv/gFt9xyC1999RVfffUVPXv2zMkQ\nUvk8dlxKUjlQPncx/hE1PsrnFk47sEhDNWzYMK688kqWLl3K0qVL+f3vf8+pp55asf0nP/kJN910\nU0XTXmlpaaXlqpx44olcffXVrFixggULFnDzzTdXbFu1ahWNGjWiXbt2rF+/nrvuuov3338/a4yb\nmkRS9ety7FyrVZKyyI+SCkZEJB/SOzFceuml9OnThz322IM999yTPn36cMkll1RsP/jgg1m5cmVF\n015qOVuSuvzyy+nUqRNdu3bl8MMP58QTT6R58+YA9OjRg9/85jf07duXDh068MEHH9C/f//Y8Va1\nHPf11uXYuVbjKOhmdg8wClgLlAHbAn9y9xuSDy/3khgF3cyomPg+W7kem/4NSaRQVTUCdnFJFxbM\nTW76jk6dS5hfPiex/dfVbbfdxoMPPsiLL76Y71ByKqlR0Pdw92+A44DngBLgjE2IU0QEgPnlc3D3\nxB6FkqAWLVrEa6+9hrvz8ccfc/3113P88cfnO6zNQpwRJ5qaWRNgEHCru682s/UJxyUissVYvXo1\nI0eOZM6cObRp04Zhw4Zx7rnn5juszUKcJPV3YC7wPvCSmXUGVmavIiIiKZ07d+a9997LdxibpVrP\nzGvRFbem7p7cgFt5pGtSIsnQzLxSl2tS1Z5JmdkF1W0LbqxdeCIiIrWTrblvu/BzZ2A/YGJYPhp4\nEyUpERFJWJwu6C8DR4cefphZK2Ciu1d/U8BmTM19IslQc58k1QW9PfB92vIPgMbvERGRxMVJUvcB\nb5rZpWZ2KfAaMD7ZsERECte5557LVVddVe9lsykvL6dRo0asX5/7O4BGjBjB5ZdfHqts165dmTJl\nSr0du8Yu6O7+32b2TyA1vO857v5WvUUgIg1Wl+JiyhcsSGz/JZ06MSdtYNf6cuuttyZStia1Hf5o\nS5A1SZlZY+Bdd+8JKDGJSL0qX7CAFWPHJrb/Ngnse/369TRqpLG5cyXrO+3u64DPzKxTjuIREcm5\nmTNncsghh1BUVMTuu+/OxIkTK7aNGDGC8847j6OOOoqWLVsyderUjZq/rr32Wjp27EhxcTF33nkn\njRo14rPPPquonyr70ksvseOOO3LDDTfQvn17OnXqxN13312xn0mTJtG7d29at25NSUkJV1xxRezX\n0LVrV6677jr23HNPWrZsydlnn82SJUsYOHAgrVq14vDDD+frr7+uKD9hwgR69epF27ZtOfTQQ5k5\nc2bFtunTp7PPPvvQunVrhg4dyvfff1/pWE899VTFHFT9+/dP9EblOF8HtgE+MrNnzeyx1COxiERE\ncmjt2rUcc8wxHHnkkXz55ZfceOONDB8+nNmzZ1eUeeCBB7jsssv49ttvOfDAAyvVf+aZZ/jLX/7C\nlClT+OSTT5g6dWrWZrlFixbx7bffsnDhQv7+979z/vnnVySPbbbZhvHjx/P111/z9NNPc9tttzFh\nwoTYr+Wxxx7jhRdeYNasWUyYMIGBAwfyxz/+kaVLl7Ju3TpuvDG6c2jWrFmcfPLJ3HjjjXz55ZcM\nGDCAY445hrVr17JmzRoGDx7M6aefzvLlyznxxBN59NFHK44xffp0zjrrLO644w6WL1/OyJEjOfbY\nY1mzZk3sOGsjTpK6EhgMXAvcnPYQEdnsvfHGG6xatYoLL7yQJk2acMghh3D00UfzwAMPVJQZNGgQ\nffv2BaiYYiPl4YcfZsSIEey66660aNGCsTU0MTZr1ozLLruMxo0bM2DAALbZZhs+/vhjIJqrqmfP\nngD06tWLoUOHVkxPH8d//Md/0K5dO3bYYQcOOugg9t9/f/bYY4+KaeSnT58OwEMPPcTRRx/NoYce\nSuPGjfntb3/L999/z2uvvcYbb7zB2rVrueCCC2jcuDEnnHAC++67b8Ux7rjjDs455xz69OmDmXHq\nqafSvHlz3njjjdhx1kaNScrdXwBmAE3DY0ZYJyKy2Vu4cOFG07mXlJSwIK1DR+b2bPV33HHHrPeD\nbbvttpWuaW299dYV07m/+eabHHrooWy//fa0adOGv/3tb7Wayj3utPELFy6kpKSkYpuZUVxczIIF\nC1i4cCGdOlW+wpNetry8nOuvv562bdvStm1bioqKmD9/PgsXLowdZ23UmKTM7ARgGnAqcBrwtpkN\nTiQaEZEc69ixI/Pmzau0bu7cuZU+qLM13+2www4bTQ1f1154w4cP57jjjmPBggWsWLGCkSNHJnID\ndMeOHSkvrzyP17x58+jUqdNGrwcqT3e/4447cskll7B8+XKWL1/OV199xcqVKznppJPqPU6I19x3\nObCvuw9395OB/YGxiUQjIpJj+++/P1tvvTXXXnsta9euZerUqTz11FMMGzYsVv0hQ4Zw1113MXPm\nTL777juuvPLKOseycuVKioqKaNq0KWVlZdx///2VttdXwhoyZAhPP/00L774ImvXruW6666jRYsW\nHHDAAfTr14+mTZvy17/+lbVr1/LYY49RVlZWUffss8/mtttuq1i3atUqJk2axKpVq+oltkxxklQj\nd1+ctrwkZj0RkYLXtGlTJk6cyKRJk2jXrh2jRo1i/Pjx7LzzzkDVZ1Hp64488kguuOACDjnkELp3\n706/fv2Aja9dVSd9X7fccguXXXYZrVu35sorr9zo7CTbGVptppTv3r079957L6NGjWK77bbj6aef\nZuLEiTRp0oSmTZvy2GOPcdddd7Htttvy8MMPc8IJJ1TU3WeffbjjjjsYNWoUbdu2pXv37owbNy7W\ncesizth91wO7AqmriEOBme7+23qNpEBo7D6RZFQ1btvmejNvNjNnzmT33Xfnhx9+0P1UGeoydl+c\nJGXAECDV7/IV4JF6/yQvEEpSIsnYkgeYfeKJJxg4cCCrVq3ijDPOoEmTJpW6bUukXpOUmY0iGqfv\nHXdvMNPFK0mJJGNLTlIDBgzg9ddfp0mTJpSWlnLzzTdX6lknkfpOUn8BDgC6AdOBV4mS1uvu/nWV\nlbYASlIiydiSk5TEk1RzXwuiSQ8PAPoR9e5b4u57bHLEBUhJSiQZSlJSr9PHZ5RpBjQPj4XAB5sQ\np4iISCzVJikzuwXYA/iOaAT0N4Bb3P3LHMUmIiINXLb+kd2BFsBc4FPgEyUoERHJpazXpMysEdHZ\n1AHh0YPoZt7X3P33OYkwx3RNSiQZuiYldbkmVdN8Uuvd/R3gUeBxYCrRGdZ/bXK0IiIFItuU5//6\n17/o0aNHrP2k5ouS+lNtkjKz88zsXjP7HCgDfg7MIbqxt01uwhORLVmXkg6YWWKPLiUdNjnG/v37\n89FHMZpCgoY4xXuSsvXu2xWYCFzs7vOylBMRqZPyuYtjNYXXlfVYXHMhKWjVnkm5+wXu/qASlIg0\nBNOnT2fPPfekqKiIYcOGsXr1amDjJrxp06ZVTPE+ZMgQhg4dWmkqeXevdnr4THPmzOHggw+mdevW\nHH744YwaNYpTTz21YvuQIUPYYYcdKCoqorS0lA8//LBi24gRIzj//PMZOHAgLVu25KCDDmLx4sWM\nHj2atm3bsttuuzFjxoyK8rWdXj7bsXNJox+KiBDNsDt58mQ+//xzZsyYUSm5pJrw1qxZw/HHH8+Z\nZ57J8uXLGTZsGI8//nil/WSbHj7TySefTN++fVm2bBljxoxh/PjxlZoLBw4cyKeffsqSJUvo3bs3\nw4cP3yjmP/zhDyxbtoxmzZrRr18/+vTpw7JlyzjhhBMYPXp0pfJxp5ePc+xcSTxJmdmRZjbTzGaZ\n2YXVlLnRzGab2TtmtldNdc2syMwmm9nHZvasmbUO69ua2RQz+9bMbsw4xothX9PNbJqZtUvqNYvI\n5ueXv/wl7du3p02bNhxzzDG88847G5V5/fXXWbduHaNGjaJx48YMHjyY/fbbr1KZbNPDp5s3bx5v\nv/02V1xxBU2aNOHAAw/k2GOPrVTmjDPOYOutt6Zp06ZcfvnlzJgxg2+//bZi++DBg9lrr70qpoff\naqutGD58OGbGSSedtNFriDu9fJxj50qsJGVmrcysVW13Hrqw3wQcAfQEhpnZrhllBgA7ufvOwEjg\nthh1LwKed/ddgCnAxWH998ClwG+qCWmYu+/t7r3dPf6czCKyxUsfEDZ9Svd0X3zxxUZTq2f25ss2\nPXy6hQsX0rZtW1q0aFHlvtavX89FF11Et27daNOmDV27dsXMKk0nH3e6+NqWj3PsXMnWu6849O77\nEpgBvGtmS8K6zjH3vx8w293L3X0N8A9gUEaZQcA9AO7+JtDazNrXUHcQkJplaxxwXKj/nbu/BvxQ\n29crIlKTHXbYgQUZ819lTj1fm30tX76c77//vsp93XfffUycOJEpU6awYsUK5syZg7vn5F6zfB47\nU7YP7QeBfwId3b2ru3cBOgHPECWMODoB6b/B+WFdnDLZ6rZPzRbs7ouA7WPGc3do6rs0ZnkRkQr9\n+vWjcePG3Hzzzaxbt44nn3yy0tTqtdG5c2f69OnD2LFjWbNmDa+//joTJ06s2L5y5UqaN29OUVER\nq1at4uKLL6519/a6JpX6OHZ9yZaktnf3+8JZDADuvsbd7wW2SzCmurwTcX4TJ7v77sBBwEFmdkod\njiMiW6C4H8CpqdX//ve/U1RUxP33388xxxyTdar4bPu+7777eO2112jXrh2XX345Q4cOrdjXaaed\nRufOnenUqRO9evXigAMOqN2Lyjh2baaXr49j15ds80k9DHxB1JyWOqPZETgD2MHdf17jzs36AmPd\n/ciwfBHg7n5NWpnbgBfd/cGwPBM4GOhaXV0z+wgodffFZtYh1O+Rts/TgX3c/YJq4qp2u5n5mDFj\nKpZLS0spLS2t6aXW9D5oWCRp8KoaEqdLSQfK5yZ3L1NJ5/bMKV+U2P4B+vbty7nnnsvpp5++yfsa\nOnQoPXr0IP0zaEuS+huYOnUqU6dOrVh/xRVX1GnSw+bAL4iu/6Sa2RYAE4Db3f37KitW3kdj4GPg\nMKKEV0bUeeGjtDIDgfPd/aiQ1P7i7n2z1TWza4DlIWFdCBS5+0Vp+zwd6OPu/5EWRxt3X2ZmTYH7\ngefc/fYqYtbYfSIJ2FLG7nv55ZfZZZddaNeuHffeey/nnXcen332WZ1m4n377bdp27YtXbt25dln\nn+X444/n9ddfZ88990wg8vyr1/mk3P0H4K/hUSfuvs6iaegnEzUt3hmSzMhos9/u7pPMbKCZfQKs\nAkZkqxt2fQ3wkJmdCZQTDdWUerGfAy2BZmY2CDicaCT3Z82sCdAYeB64o66vS0Qaro8//pghQ4bw\n3Xff8eMf/5hHH320zlPFL1q0iOOPP57ly5dTXFzMbbfdtsUmqLqqaRT0w4h6zqWfST3p7s/nILa8\n0JmUSDK2lDMpqbt6PZMys+uBXsB4op51AMXAf5rZQHf/9aaHLCIiUr1s16RmuXv3KtYbMCvcfLvF\n0ZmUSDJ0JiV1OZPK1gX9BzPrXcX63lR/s6yIiEi9yTZVx5nA7aGXX6oLemeioYfOTDowERGRbL37\n3gL6mFkxaR0n3H1+dXVERKpTUlKiCQEbuJKSklrXyXYmldI+PADWsqEThYhIbHPmzMl3CLIZyta7\n7zDgVqL7kFIjKhaHwWXPdfcXchCfiIg0YNnOpP4KHOnun6WvNLOdgKeAHlXWEhERqSfZevc1JTqL\nyjQ3bBMREUlUtjOpccCbZvYAlQeYHQbcnXBcIiIiNQ6LtDtVDDDr7u/mILa80M28IiK5VadhkQDc\n/T3gvUSiEhERqUGdplM3s4k1lxIREdk02bqg71HdJqBPMuGIiIhskK25bzrwKlVP594mmXBEREQ2\nyJakZgJnuvsnmRvMbF4V5UVEROpVtmtSV1B9EhudQCwiIiKVZO2C3hCpC7qISG7VdT4pERGRvFKS\nEhGRgqUkJSIiBavG+aTMrBFwJNAlvby735hcWCIiIvEmPXwScKLhkdYnG46IiMgGcZJUF3ffPfFI\nREREMsS5JvWsmR2aeCQiIiIZ4iSpV4CJZrbSzJab2VdmtjzpwCQZXUo6YGY1PrqUdMh3qCIiNd/M\na2afAyeQcU3K3dclG1p+bOk38xZSLCIisAnzSQXzgen1/sktIiJSgzhJ6hNgiplNAn5IrVQXdBER\nSVrcM6n5QKuEYxEREamkxiTl7pcBmNlWYfn/kg5KREQEYvTuM7PdzOwtYDYw28zeNLMeyYcmIiIN\nXZwu6LcDv3P3YncvBi4B7kg2LBERkXhJqqW7P5dacPfngZbJhSQiIhKJk6TmmNnFZlYcHhcBcxKO\nS0REJFaSOhPYEZgEPA0UAyOSDEoaBo1+ISI1idMF/WB3Py99hZkdDzyWTEjSUJTPXRxz9IvFyQcj\nIgUpzpnUpVWsu6S+AxHJJ53ViRSmas+kzOwIoskOO5nZDWmbWqF5pWQLo7M6kcKUrblvCfA+8D3w\nQdr6b4GLkgxKREQEsiQpd58OTDezH7n7nenbzGwUcFPSwYmISMMW55rUGVWsO6ue4xAREdlItmtS\nJwFDga5mlt6TrxWwIunAREREsl2TKgOWEd0XdXPa+m+B6UkGJSIiAtmvSX0OfA48b2btgD5h02fu\nviYXwYmISMMWZxT044FpwKnAacDbZjY46cBERETijDgxBtjX3RcDmFl7YDLweJKBiYiIxOnd1yiV\noIIlMeuJiIhskjhnUpPN7GnggbA8FHg2uZBEREQicZLUb4ETgf5heRzwSGIRbSa6FBdTvmBBvsMQ\nEdmi1Zik3N2Bh4CHzKyNu9fqHikzOxL4C1ET4Z3ufk0VZW4EBgCrgDPc/Z1sdc2sCHgQKCGa22qI\nu39tZm2JEui+wF3ufkHaMXoDdwMtgEnu/qvavI5M5QsWsGLs2Fhl28QsJyIilVV7bcnM9jOz583s\nITPb08zeBT4xs8VmdnicnZtZI6Lhk44AegLDzGzXjDIDgJ3cfWdgJHBbjLoXAc+7+y7AFODisP57\nolHbf1NFOLcCZ7l7d6B7GEB3i9CluDjWCN5mlu9QRURqJduZ1M1EPftaAy8Cx7j7q2bWExhP1MOv\nJvsBs929HMDM/gEMAmamlRkE3APg7m+aWevQg7BrlrqDgIND/XHAVOAid/8OeM3Mdk4Pwsw6AC3d\n/a2w6h7gOLaQa2uFdFanZlARqU/ZklQTd58EYGaXu/urAO7+gcX/St4JmJe2PJ8ocdVUplMNddun\nehy6+yIz2z5GHPOrOIbUs0JKmCKy+cuWpDzt+f9l2Vbf6tImVa/xjE378CwtLaW0tLQ+dy8i0qBN\nnTqVqVOnxiqbLUntaWbLiZJGy/CcsLxNzFgWAJ3TlovDuswyO1ZRplmWuovMrL27Lw5NeUtixFHV\nMao0Vt/wRUQSk/nl/4orrqi2bLabcpsB2wHtgObheWq5RcxY3gK6mVmJmTUjusdqQkaZCUTDLWFm\nfYEVoSkvW90JbJhC5HTgySqOXXFG5u6LgK9DZxALx6uqjmxB1KFEZPOXbYDZdZu6c3dfFyZInMyG\nbuQfmdnIaLPf7u6TzGygmX1C1AV9RLa6YdfXEHWJPxMoB4akjmlmnwMtgWZmNgg43N1nAudTuQv6\nM5v6+qSw6fqYyOYvzs28myQkg10y1v0tY3lU3Lph/XLgp9XU6VrN+n8Du8eLWkRECkG2+6QST2Ai\nIiLZZLsmVQZgZnfnJhQREZHKsp0tNTOzIcBBZnZs5kZ3z+wAISIiUq+yJanzgVOANkQDzKZzNu6l\nJyIiUq+y9e57CXjJzN7O7OggIiKSC3EmL/xfMzvPzP4RHueqU4VI7dTmnq0uxcX5DlekYMRJNjcB\nPwL+NyyfAuwN/CKpoES2NLpnS6Ru4iSpvu6+Z9ryZDObkVRAIiIiKXGa+9abWZfUQni+PplwRERE\nNoiTpC4EXgkTIL4AvAT8Z7JhiUiS4l4j0/Uxybc408dPNrPuQI+w6iN3z5y6Q0Q2I3GvkeXi+ljc\niTJLOnVizvz5NZaTLUusXnohKU1LOBYRaYAKKWFK4YnT3CciIpIXSlIiIoGu1RWeGpv7zOxBonuk\nJrt7ktPGiwjQvBmxJ2Is6dyeOeWLEo6o4VDTY+GJc03qLuBM4KaQsO5290+SDUuk4fphNVRM71kD\n67E40ViUMCXf4vTuewZ4xsyKgOHAi2H22zuAB9x9bcIxikieFFLClIYp1jWpkKBOBk4F3gX+BhwA\naAp2ERFJTJxrUg8TTbt+H3CCu6duVLjPzKYnGZyIiDRsca5J3Q48n95pwsyauPtad987udBERKSh\ni9Pcd00VvfrKkghGREQi6g4fqfZMysy2B3YAtjKz3YFUF59WwNY5iE1EpMFSd/hItua+o4i6nhcD\nt6St/xa4LMmgREREIPv08XcBd5nZEHd/KIcxiYiIANmb+4a5+wPADmZ2QeZ2d78x0chERKTBy9Zx\noij8bAdsV8VDRCRnUqNfxOpMUNKhwcSypcvW3HdL+KnrTyKSd4U0+kUhxbKly9bcd0O2iu7+6/oP\nR0REamNLH18xW+++D3IWhYiI1MmWflaXrbnvzlwGIiIihalLcTHlCxbUWK6kUyfmzJ9fY7nayNbc\nd727/8bMHgc2mkfK3Y+v10hERKQg5fPG4mzNfQ+GnzfV+1FFRERiyNbcVxZ+vmBmTYGdic6oZmsO\nKRERyYU4U3UcSTQS+lyi8fuKzexsd5+cdHAiItKwxZmq4y/AT919FoCZdQeeBHokGZiIiEicqTpW\nphIUQHi+KrmQREREItl69x0bnpaZ2QTgIaJrUicCb+YgNhERaeCyNfedmPb8a+CI8PxboGViEYmI\niATZevdkHn/QAAAUxUlEQVSdmstAREREMsXp3dccOAPoCbRIrXf3XyQXloiISLyOE/cAXYCjia5F\n7QR8n2BMIiKyGUpiCpM4XdC7u/tJZnaUu99pZvcAr2zSKxERkS1OEoPdxjmTWhN+rjCzHkSdJraP\nF4aIiEjdxTmTutPMioAxwLPA1sDliUYlIiJCjCTl7n8LT18EOicbjoiIyAY1NveZWZGZ/dnMyszs\nTTO7LpxZiYiIJCrONal/AN8Aw4FTiG7mfTBrDRERkXoQ55pUJ3cfk7Z8hZm9n1RAIiIiKXHOpF4w\ns5+nFszseOC55EISERGJVJukzOwrM1sOnAY8ZGarzWw18AhwetwDmNmRZjbTzGaZ2YXVlLnRzGab\n2TtmtldNdcN1sslm9rGZPWtmrdO2XRz29ZGZHZ62/sWwr+lmNs3M2sV9DSIikh/ZzqTaAduFn02B\nrcKjaVhfIzNrRDT9/BFEwyoNM7NdM8oMAHZy952BkcBtMepeBDzv7rsAU4CLQ53dgCFEc10NAG4x\nM0s73DB339vde7v70jivQURE8qfaJOXu61IPokRxVXj8LKyLYz+i6ebL3X0NUSeMQRllBhENvYS7\nvwm0NrP2NdQdBIwLz8cBx4XnxwL/cPe17j4HmB32U+PrFRGRwhOnC/pVwH8Bn4XHf5nZlTH33wmY\nl7Y8P6yLUyZb3fbuvhjA3RexYQSMzDoLMo53d2jquzRm/CIikkdxevcdA+ydOnsys/8FpgFJfdBb\nzUU24jHKnOzuX5jZj4DHzOwUd7+3DscSEZEciZOkAFoBX4XntZnwcAGVR6koDusyy+xYRZlmWeou\nMrP27r7YzDoAS2rYF+7+Rfi5yszuJ2oGrDJJjR07tuJ5aWkppaWl2V6jiIjUwtSy6Gf6Z2114iSp\na4FpZvYC0VlOKXBZzFjeArqZWQnwBTAUGJZRZgJwPvCgmfUFVoTkszRL3QlEc1xdQ9TT8Mm09feZ\n2Z+Jmvm6AWVm1hho4+7LzKwp0bQj1Xajj/PGiYhI3ZSGngKpz9orrrii2rJZk1ToGfcC0bh9+4fV\nl7t75tlQldx9nZmNAiYTXf+6090/MrOR0Wa/3d0nmdlAM/sEWAWMyFY37Poaom7xZwLlRD36cPcP\nzewh4EOi0dvPc3cPEzc+a2ZNgMbA88AdcV6DiIjkT9YkFT7gn3P3XsBjdTmAuz8D7JKx7m8Zy6Pi\n1g3rlwM/rabO1cDVGeu+A/rUKnAREcm7OF2y3zGzvROPREREJEOca1J7A2+Z2adEzXFGdJLVO9HI\nRESkwYuTpI5NPAoREZEqVJukQmeDs4l6yL0H3F2LkSZEREQ2WbZrUncD/YmGFjoOuC4XAYmIiKRk\na+7r5e67A5jZ7cCbuQlJREQkku1Mak3qSRjgVUREJKeynUntGeaTgqhHX8uwnOrd1zbx6EREpEHL\nlqSa5SwKERGRKlSbpNSTT0RE8k2TAIqISMFSkhIRkYKlJCUiIgUr24gTX1H1jLfq3SciIjmRrXdf\nu5xFISIiUoXYvfvMrC3QIm3VwqSCEhERgRjXpMzsKDObBcwnGhppPjAl6cBERETidJy4CjgQ+Njd\ndwSOAF5JNCoRERHiJam17v4l0MjMzN2fA/ZLOC4REZFYkx5+bWbbAP8C7jGzJcD/JRuWiIhIvDOp\n44iS0q+AqcAC4OgEYxIREQHiJamL3X2du69x9zvd/Qbg10kHJiIiEidJHVnFuqPqOxAREZFM2Uac\nGAmcA3Q3s2lpm1oC/046MBERkWwdJx4CXgCuBi5KW/+tuy9JNCoRERGyjzjxFfAVcKKZ9QQOCpte\nAZSkREQkcXFGnDgfeBjoHB4Pmdl5SQcmIiIS5z6pkcB+7r4SwMz+ALwG3JJkYCIiInF69xmwOm15\nTVgnIiKSqGy9+5q4+1pgPPCmmT0aNg0GxuUiOBERadiynUmVAbj7tURNft+Fxznufl0OYssbM6vx\nkQvFJV0KJhYRkXzIdk2q4tPP3csISashuHra6hrLXNy7WeJxLJhbXjCxiIjkQ7YktZ2ZVTv8URge\nSRqI4pIuLJhbnu8wRKSByZakGgPboE4SQmGd1RVSwlQsVSukWGTzli1JfeHu/52zSERiKqSEuTnG\nAsnHU0ixKGFu3mJdkxIR2VwpYVatkGLJJluSOixnUYiINACFlDALKZZsqu2C7u7LcxmIiIhIpjgj\nToiIiOSFkpSIiBQsJSkRESlYSlIiIlKwlKRERKRgKUmJiEjBUpISEZGCpSQlIiIFS0lKREQKlpKU\niIgULCUpEREpWEpSIiJSsBJPUmZ2pJnNNLNZZnZhNWVuNLPZZvaOme1VU10zKzKzyWb2sZk9a2at\n07ZdHPb1kZkdnra+t5m9G/b1l6Rer4iI1J9Ek5SZNQJuAo4AegLDzGzXjDIDgJ3cfWdgJHBbjLoX\nAc+7+y7AFODiUGc3YAjQAxgA3GJmqXmxbgXOcvfuQHczOyKZVy0iIvUl6TOp/YDZ7l7u7muAfwCD\nMsoMAu4BcPc3gdZm1r6GuoOAceH5OOC48PxY4B/uvtbd5wCzgf3MrAPQ0t3fCuXuSasjIiIFKukk\n1QmYl7Y8P6yLUyZb3fbuvhjA3RcB21ezrwVp+5pfQxwiIlJgCrHjRF2mrfd6j0JERPLP3RN7AH2B\nZ9KWLwIuzChzG3BS2vJMoH22usBHRGdTAB2Aj6raP/AMsH96mbB+KHBrNTG7HnrooYceuX1Ul0ea\nkKy3gG5mVgJ8QZQchmWUmQCcDzxoZn2BFe6+2MyWZqk7ATgDuAY4HXgybf19ZvZnoua8bkCZu7uZ\nfW1m+4WYTgNurCpgd6/LmZyIiCQg0STl7uvMbBQwmahp8U53/8jMRkab/XZ3n2RmA83sE2AVMCJb\n3bDra4CHzOxMoJyoRx/u/qGZPQR8CKwBzvNwekSUCO8GWgCT3P2ZJF+7iIhsOtvwGS4iIlJYCrHj\nRMGLc4NyjuK408wWm9m7+YohLZZiM5tiZh+Y2XtmdkEeY2luZm+a2fQQy5h8xZIWUyMzm2ZmE/Ic\nxxwzmxHem7I8x9LazB4ON95/YGb75ymO7uH9mBZ+fp3nv9/RZvZ+GHzgPjNrlsdYfhn+h/L2P60z\nqVoKNxnPAg4DFhJd4xrq7jPzEEt/YCVwj7vvkevjZ8TSAejg7u+Y2TbAv4FB+XhfQjxbu/t3ZtYY\neBW4wN3z9qFsZqOBfYBW7n5sHuP4DNjH3b/KVwxpsdwNvOTud5lZE2Brd/8mzzE1IrpFZX93n1dT\n+QSO3xH4F7Cru682sweBp939njzE0hN4ANgXWAv8EzjH3T/LZRw6k6q9ODco54S7/wvI+4cNgLsv\ncvd3wvOVRD0w83Yvmrt/F542J7r2mrdvY2ZWDAwE/p6vGNIYBfB/b2atgIPc/S6AcAN+XhNU8FPg\n03wkqDSNgR+lEjfRl+F86AG86e4/uPs64GXg+FwHkfc/1s1QnBuUGzQz6wLsBbyZxxgamdl0YBHw\nXNpoI/nwZ+A/yWOiTOPAc2b2lpmdncc4ugJLzeyu0Mx2u5ltlcd4Uk4iOnvIC3dfCFwPzCUajGCF\nuz+fp3DeBw4KY6VuTfRFa8dcB6EkJfUqNPU9AvwynFHlhbuvd/e9gWJg/zCuY86Z2VHA4nCWadTt\nZvX6dKC79yb6wDk/NBnnQxOgN3BziOc7ovsc88bMmhINrfZwHmNoQ9QyUwJ0BLYxs5PzEUtoqr8G\neA6YBEwH1uU6DiWp2lsAdE5bLg7rGrzQPPEIMN7dn6ypfC6EJqQXgSPzFMKBwLHhWtADwCFmlvPr\nCynu/kX4+SXwOFHzdT7MB+a5+9th+RGipJVPA4B/h/cmX34KfObuy0MT22PAAfkKxt3vcvc+7l4K\nrCC6Hp9TSlK1V3GDcuh1M5ToJuJ8KYRv5yn/C3zo7v+TzyDMrF1q+pbQhPQzopFMcs7df+fund39\nx0R/K1Pc/bR8xGJmW4czXczsR8DhRE06ORfG3pxnZt3DqsOI7m/Mp2HksakvmAv0NbMWYQaHw4iu\n7+aFmW0XfnYGBgP35zqGpEec2OLUcJNxTpnZ/UApsK2ZzQXGpC5E5yGWA4HhwHvhWpADv8vTTdM7\nAONCT61GwIPuPikPcRSa9sDjZuZE//v3ufvkPMZzAdEIMU2Bzwg38udDuObyU+AX+YoBwN3LzOwR\noqa1NeHn7XkM6VEza8uGwRFy3rlFXdBFRKRgqblPREQKlpKUiIgULCUpEREpWEpSIiJSsJSkRESk\nYClJiYhIwVKSakDMrG3alARfmNn8tOVa3TMXpgnZeRPj2drMXtyUfaTta3RtpzQws8PM7PEq1p9l\n0ezOORXnPTWz8Wa20SjqZtbVzE6qwzFvCNMw/CFj/e9rMzVD5vHNbG8zO6K28cQ81pAwtcc6M9sj\nY9ulZjbbzD40s8PS1u8bXucsM7s+bX1zi6YLmW1mr4bBgHOiur+/tO3tzezpXMVTqJSkGpAw1Mre\nYay0W4EbUsvuvraW+zrL3WdvYkj/D3hoE/dBmI7j10SzLtdWdTcK5vwGwk18T3ciGtEitjCiwQh3\n393df1fH41Z3/N4kNxTVu0Tj272avtLMdgeOA3YFjib6G0+5FTjd3bsDvdIS2C+AL9x9Z+AW4I8J\nxZyaBiRTtX9nYVSOpWa2b1IxbQ6UpBquSkMpmdl/hW+a74YRNTCznSyafO2B8M30H2bWPGx7JfUt\n1syOMrN/h7OyZ8K6Q83snXCW9nY1I1wPB54M5TuGfU4LMfQN608Jy++a2VVhXWMz+8rM/mxm7xCN\nML498IqZTQ5lBpjZa+HYD6SOH2KdaWZvk32KlS5mNtXMPjaz34W6V5nZ+Wnv2R/N7NyM9/EiMzsn\nPP+rmT0bnv/MovmTUpNmVhVb+ns6Mhz7dTO7w8xuSDvMoeFb/ydmlnoNVwOl4f0blRGTmdn14fc7\nw8xS0y08RTSA6bS0del6h+N/bGYjathX+vEvAC4HTk7t28y2NbMnQ51/WRjw16IztrvCa//czAaZ\n2XVh/xOr+mB395nu/gkbDwc2CHjA3deFOY/KzWyfcHbU3N2nhXLjiZJZqs648PwhYKOzvxp+p3eF\n53H+Tvet7u8vy//Lk8ApVfxuGg5316MBPoAxwK/D8/2Ihl9pBmxDNIZaT6Jvx+uBfUO5cUSTBwK8\nAuxBNNROOVAc1rcJPyel1duaMLpJ2vGbA/PTlv8L+M/w3EKdTsDnQBHRHDtTiUbvbhziGpRWfy7Q\nMjzfLpRtEZZ/RzTC9lZE06x0CesfAR6r4r05K5RrFeL4ILzWnYCyUKYR8CnQOqPugUTDDUE0ed0b\n4fX8N9GwP1XGlvGeFhMNE9SKaPiiV4nOeiH6gE3tf3fgo/D8sKpeS9g2hGjiPMLvay7QLryPy6up\n83vgbaBpiHle+FndviodP7yHN6Qt3wJcHJ7/DHgr7TgvhveoN7AKODRsmwAMzPI3/AqwR9ryrcCQ\ntOW7iUY13x+YlLa+NBUr0bh426dt+5xoYsra/E5j/Z2S5e+Pav5fiAaznpbvz4t8PnQmJQD9gUfd\nfbVH02s8ARwUtn3mG+ZiujeUTdePaNDU+QDuviKsfxW4MXyrb+3hPy7N9sDytOW3gP9nZpcBu3s0\naeH+wAvu/pVHI0LfD/wklP/BK4+0nj7Q7gHAbsBrFo0jeDLQJaz72N3nhHL3ZXlPnnX3b0IcTwD9\n3f1T4BuLZiwdQDQh3NcZ9d4i+sbcmmjW5LeIZuQ9iOhDtarYSjL2kXrd33jUDPtIxvYnANz9PaLp\nHGrSnzBwqkdNSK8AfcK2bIMTP+HuazwaFfwloi8z2fZVUwzjQ73ngB3SzhYmhb+P96LNPiWsf4/o\n95ZLVb0fNf1O4/6dZvv7q+7/ZQnRWJQNlpKU1FZVbegb/WO7+1XA2URnZm+Y2U4ZRf6PtGtI7v4i\n0TfcL4gGhx1W3b7T6lfHgH96dK1tb3fv5e7n1LC/jV5CNct3En17HkE06nvlQu6riWZSPY3oW/cr\nRGcZnX1DE1VmbOdm7qeGOH+IWa466XWyXXtL32ZEZwXZ9pVNtuOkXs96YHXa+vXUbhDsBVSelC81\njU626XUq6lg00O1GU9jH+J1C/L/TKstl+X9pUcU+GhQlKYHon26wRT2dtiFqK38lbOtqZvuE5yen\nrU95jehaRGcAMysKP3/s7u+7+x+BacAu6ZXcfSmwlYVehaH+Ynf/O1Ezzd5EM/uWWjQzaBOiC/NT\nwy4y/9m/IWoeS8V0sJl1Dfve2sy6ETVjpqZZMaKpGapzuJm1smh07PSL9I8BxwB7evUzpr4C/JZo\nuu1/AecTNZ1liy1dWXjdrcIHZ7Ypu1Pvw7dAyyzxDA3Xk9oTnc2l4smWZI4zs6YWTdfQP9Spbl/f\nsuH9p4rlVwjXVszsp8ACd6/qw7e2STe9/ARgWIh5J6Ik8u9wlv99uD5lwKmEa6Ghzhnh+UlEsxtU\nJdvvNO7fabV/f1n+X7qTp+lUCoWSlBCa8x4g+qd7jWi21A/C5o+AX5vZh0Rt6nekqoW6S4BzgSdD\n89W9Yftvw8Xvd4g+sKr653+eDRO6HQbMMLNpRPPW/NXdFwCXETU1TQNe8w1Tf2R+M78DeN7MJoeY\n/h/wYDj+q8DO4UPxXOAZokSwMMvb8hbRB9h0ousR74bX+wPRB1W2eYdeIWrOfMOj6cBXhzqp9+us\nzNjSX5O7zwP+FGJ4meja19fpZdKklqcDTSzqvDIqo8wjRPNpvUv0exgdviRUtb9077PhQ/ny0OxX\n3b6mA43Tjj8F2NOiDjXHE3Wk6GdmM4CxbEgMmWrsVWlmPzezeUTNjM+Y2USA8Dt6guhv9imi33XK\neUTXVGcBH6R9wbidqOlxdihTXS/HbL/TWH+n4e/vHKr++6vu/+UQoEF3Q9dUHVKt8G30EY+mYU9i\n/32Ac939rCT2n4TQ22w60cXwOQke50fuvip8M38SuMXdG/SHVUNkZi8DR7n7t/mOJV90JiU1Sexb\njEdTh/8rqf3XNzPrBXxCdKF/TsKH+304q5wBzFSCanjMbHvg2oacoEBnUiIiUsB0JiUiIgVLSUpE\nRAqWkpSIiBQsJSkRESlYSlIiIlKwlKRERKRg/X9CuYVdWIuo+QAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "a_top = np.sort([sum(tpm_low_gamma.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "b_top = np.sort([sum(topic_model.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "c_top = np.sort([sum(tpm_high_gamma.get_topics(topic_ids=[i], num_words=100)['score']) for i in range(10)])[::-1]\n",
+ "\n",
+ "a_bot = np.sort([sum(tpm_low_gamma.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "b_bot = np.sort([sum(topic_model.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "c_bot = np.sort([sum(tpm_high_gamma.get_topics(topic_ids=[i], num_words=547462)[-1000:]['score']) for i in range(10)])[::-1]\n",
+ "\n",
+ "ind = np.arange(len(a))\n",
+ "width = 0.3\n",
+ " \n",
+ "param_bar_plot(a_top, b_top, c_top, ind, width, ylim=0.6, param='gamma',\n",
+ " xlab='Topics (sorted by weight of top 100 words)', \n",
+ " ylab='Total Probability of Top 100 Words')\n",
+ "\n",
+ "param_bar_plot(a_bot, b_bot, c_bot, ind, width, ylim=0.0002, param='gamma',\n",
+ " xlab='Topics (sorted by weight of bottom 1000 words)',\n",
+ " ylab='Total Probability of Bottom 1000 Words')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "From these two plots we can see that the low gamma model results in higher weight placed on the top words and lower weight placed on the bottom words for each topic, while the high gamma model places relatively less weight on the top words and more weight on the bottom words. Thus increasing gamma results in topics that have a smoother distribution of weight across all the words in the vocabulary."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ For each topic of the **low gamma model**, compute the number of words required to make a list with total probability 0.5. What is the average number of words required across all topics? (HINT: use the get_topics() function from GraphLab Create with the _cdf_\\__cutoff_ argument)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 72,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def calculate_avg_words(model, num_words=547462, cdf_cutoff=0.5, num_topics=10):\n",
+ " avg_num_of_words = []\n",
+ " for i in range(num_topics):\n",
+ " avg_num_of_words.append(len(model.get_topics(topic_ids=[i], num_words=547462, cdf_cutoff=.5)))\n",
+ " avg_num_of_words = np.mean(avg_num_of_words)\n",
+ " return avg_num_of_words"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 73,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "252.40000000000001"
+ ]
+ },
+ "execution_count": 73,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "calculate_avg_words(tpm_low_gamma)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__Quiz Question:__ For each topic of the **high gamma model**, compute the number of words required to make a list with total probability 0.5. What is the average number of words required across all topics? (HINT: use the get_topics() function from GraphLab Create with the _cdf_\\__cutoff_ argument)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 74,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "576.20000000000005"
+ ]
+ },
+ "execution_count": 74,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "calculate_avg_words(tpm_high_gamma)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We have now seen how the hyperparameters alpha and gamma influence the characteristics of our LDA topic model, but we haven't said anything about what settings of alpha or gamma are best. We know that these parameters are responsible for controlling the smoothness of the topic distributions for documents and word distributions for topics, but there's no simple conversion between smoothness of these distributions and quality of the topic model. In reality, there is no universally \"best\" choice for these parameters. Instead, finding a good topic model requires that we be able to both explore the output (as we did by looking at the topics and checking some topic predictions for documents) and understand the impact of hyperparameter settings (as we have in this section)."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 2",
+ "language": "python",
+ "name": "python2"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 2
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython2",
+ "version": "2.7.12"
+ },
+ "toc": {
+ "toc_cell": false,
+ "toc_number_sections": false,
+ "toc_threshold": "8",
+ "toc_window_display": false
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.04 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.04 PM.png
new file mode 100644
index 0000000..2103c74
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.04 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.11 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.11 PM.png
new file mode 100644
index 0000000..9c3c535
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.11 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.16 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.16 PM.png
new file mode 100644
index 0000000..c6709d3
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.16 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.20 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.20 PM.png
new file mode 100644
index 0000000..66eebdd
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.20 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.26 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.26 PM.png
new file mode 100644
index 0000000..13aca9b
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.26 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.30 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.30 PM.png
new file mode 100644
index 0000000..11fd8ef
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.30 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.34 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.34 PM.png
new file mode 100644
index 0000000..24e4714
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.34 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.39 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.39 PM.png
new file mode 100644
index 0000000..b6fd355
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.39 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.44 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.44 PM.png
new file mode 100644
index 0000000..bf2d8d2
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.44 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.49 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.49 PM.png
new file mode 100644
index 0000000..a265d66
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.49 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.53 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.53 PM.png
new file mode 100644
index 0000000..dab5bbb
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.53 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.57 PM.png b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.57 PM.png
new file mode 100644
index 0000000..f34cc74
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/images/Screen Shot 2016-07-31 at 7.12.57 PM.png differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/quiz-week5-assignment.ipynb b/machine_learning/4_clustering_and_retrieval/assigment/week5/quiz-week5-assignment.ipynb
new file mode 100644
index 0000000..1bead40
--- /dev/null
+++ b/machine_learning/4_clustering_and_retrieval/assigment/week5/quiz-week5-assignment.ipynb
@@ -0,0 +1,194 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Modeling text topics with Latent Dirichlet Allocation"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 1\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 2\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 3\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 4\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 5\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 6\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 7\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 8\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 9\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 10\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 11\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Question 12\n",
+ "\n",
+ "\n",
+ "\n",
+ "*Screenshot taken from [Coursera](https://www.coursera.org/learn/ml-clustering-and-retrieval/exam/LXSnc/modeling-text-topics-with-latent-dirichlet-allocation)*\n",
+ "\n",
+ ""
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 2",
+ "language": "python",
+ "name": "python2"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 2
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython2",
+ "version": "2.7.12"
+ },
+ "toc": {
+ "toc_cell": false,
+ "toc_number_sections": false,
+ "toc_threshold": "8",
+ "toc_window_display": false
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/topic_models.zip b/machine_learning/4_clustering_and_retrieval/assigment/week5/topic_models.zip
new file mode 100644
index 0000000..23d415f
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/topic_models.zip differ
diff --git a/machine_learning/4_clustering_and_retrieval/assigment/week5/week5-assignment1.pdf b/machine_learning/4_clustering_and_retrieval/assigment/week5/week5-assignment1.pdf
new file mode 100644
index 0000000..6ab0c27
Binary files /dev/null and b/machine_learning/4_clustering_and_retrieval/assigment/week5/week5-assignment1.pdf differ