github-bugs-prediction-using-bert-model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.055864,
     "end_time": "2020-12-13T16:45:06.451312",
     "exception": false,
     "start_time": "2020-12-13T16:45:06.395448",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# GITHUB BUGS PREDICTION\n",
    "[Indrajit Singh](https://www.linkedin.com/in/indrajitsinghds/) - December 2020\n",
    "\n",
    "\n",
    "----------\n",
    "## My First Kaggle Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.052055,
     "end_time": "2020-12-13T16:45:06.555689",
     "exception": false,
     "start_time": "2020-12-13T16:45:06.503634",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "This notebook simplifies the steps to solve github bug predition through a typical workflow using BERT base uncased model. Before we get started with the workflow , the first question that comes into our mind is how do we get started. Well we will get there but let's first understand what exactly the problem statement is and what is expected here?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.09005,
     "end_time": "2020-12-13T16:45:06.698639",
     "exception": false,
     "start_time": "2020-12-13T16:45:06.608589",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# BERT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.056315,
     "end_time": "2020-12-13T16:45:06.807352",
     "exception": false,
     "start_time": "2020-12-13T16:45:06.751037",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "* ###  What is BERT?\n",
    "\n",
    "BERT is the first deeply bidirectional, unsupervised language representation, pre-trained using only a plain text corpus (in this case, Wikipedia).\n",
    "\n",
    "* ### What Makes BERT Different?\n",
    "\n",
    "BERT builds upon recent work in pre-training contextual representations — including Semi-supervised Sequence Learning, Generative Pre-Training, ELMo, and ULMFit. However, unlike these previous models, BERT is the first deeply bidirectional, unsupervised language representation, pre-trained using only a plain text corpus (in this case, Wikipedia).\n",
    "\n",
    "* ### Why does pretrained language models makes a difference?\n",
    "\n",
    "Pre-trained representations can either be **context-free** or **contextual**, and contextual representations can further be unidirectional or bidirectional. \n",
    "\n",
    "* Context-free models such as word2vec or GloVe generate a single word embedding representation for each word in the vocabulary. For example, the word “bank” would have the same context-free representation in “bank account” and “bank of the river.” \n",
    "\n",
    "* Contextual models instead generate a representation of each word that is based on the other words in the sentence. For example, in the sentence “I accessed the bank account,” a unidirectional contextual model would represent “bank” based on “I accessed the” but not “account.” \n",
    "\n",
    "However, BERT represents “bank” using both its previous and next context — “I accessed the ... account” — starting from the very bottom of a deep neural network, making it deeply bidirectional.\n",
    "\n",
    "Read more about Basic BERT [here](http://https://arxiv.org/pdf/1810.04805.pdf).\n",
    "\n",
    "In this Notebook we will be using BERT Base Uncased, from [Hugging Face Library](http://https://huggingface.co/bert-base-uncased). \n",
    "________________________________________\n",
    "**Model: bert-base-uncased**\n",
    "\n",
    "*        12-layer, 768-hidden, 12-heads, 110M parameters.\n",
    "\n",
    "*        Trained on lower-cased English text.\n",
    "\n",
    "```\n",
    "@article{DBLP:journals/corr/abs-1810-04805,\n",
    "  author    = {Jacob Devlin and\n",
    "               Ming{-}Wei Chang and\n",
    "               Kenton Lee and\n",
    "               Kristina Toutanova},\n",
    "  title     = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
    "               Understanding},\n",
    "  journal   = {CoRR},\n",
    "  volume    = {abs/1810.04805},\n",
    "  year      = {2018},\n",
    "  url       = {http://arxiv.org/abs/1810.04805},\n",
    "  archivePrefix = {arXiv},\n",
    "  eprint    = {1810.04805},\n",
    "  timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
    "  biburl    = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
    "  bibsource = {dblp computer science bibliography, https://dblp.org}\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.05323,
     "end_time": "2020-12-13T16:45:06.912708",
     "exception": false,
     "start_time": "2020-12-13T16:45:06.859478",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Problem Statement:**\n",
    "\n",
    " [MachineHack](http://https://www.machinehack.com/) a Biggest Machine Learning Community For Data Science & AI Enthusiastshas comeup with one of the popular Hackathon on GitHub bugs prediction. \n",
    " \n",
    "Detecting defects in software systems is an evergreen topic, since there is no real world software without bugs. There are many algorithms that has come up recently to help developers find bugs in software. The challenges of having curated datasets for own domain specific software , or code relates bug finding is rare. You can go through the [research paper](http://https://www.researchgate.net/publication/304664263_A_Public_Bug_Database_of_GitHub_Projects_and_Its_Application_in_Bug_Prediction) for gaining more ideas on Bug Prediction Databases and related algorithms that can be used.\n",
    "\n",
    "In this Hackathon proposed by Machinehack , we need to come up with an algorithm that can predict the bugs, features, and questions based on GitHub titles and the text body. \n",
    "\n",
    "\n",
    "The Dataset has three columns Title, Body and Label: \n",
    "\n",
    "*     **-Title** - the title of the GitHub bug, feature question\n",
    "\n",
    "*     **-Body** - the body of the GitHub bug, feature question\n",
    "\n",
    "*     **-Label** - Represents various classes of Labels\n",
    "\n",
    "Once we clean up the dataset [embold_train.json, embold_test.json] and make it ready for the model we need feed the data to a specifc model (Here we using BERT base uncased as pretrained model to predict the expected outcome )"
   ]
  },
  {
   "attachments": {
    "image.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgAAAADOCAIAAADHZy1AAAAMZmlDQ1BJQ0MgUHJvZmlsZQAASImVVwdck0cbv3dkkrACEZAR9hJFZgAZIawIAjIFUQlJIGHEmBBU3GhRwTpQEcWJVhkWrVZA6kDEOovito7iQKVSi1VcqHwXEtDab/y+5/e79/7vc8/9n5G7vHcA6HTyZbJcVBeAPGm+PC48mDUpJZVFegS0gD7QA97AhC9QyDixsVEAylD/d3l9HSCq/oqLiuuf4/9V9IUihQAAJA3iDKFCkAdxCwB4kUAmzweAGAL11jPzZSoshthADgOEeK4KZ6nxahXOUONdgzYJcVyImwAg0/h8eRYA2m1QzyoQZEEe7UcQu0qFEikAOgYQBwjEfCHECRCPysubrsILIXaA9jKIqyFmZ3zBmfU3/oxhfj4/axir8xoUcohEIcvlz/4/S/O/JS9XOeTDDjaaWB4Rp8of1vBmzvRIFaZB3CPNiI5R1RritxKhuu4AoFSxMiJRbY+aChRcWD/AhNhVyA+JhNgU4jBpbnSURp+RKQnjQQxXCzpLks9L0MxdJlKExms4N8unx8UM4Uw5l6OZW8+XD/pV2bcpcxI5Gv6bYhFviP9VoTghGWIqABi1QJIUDbE2xAaKnPhItQ1mVSjmRg/ZyJVxqvhtIGaLpOHBan4sLVMeFqexl+UphvLFisUSXrQGV+SLEyLU9cFqBfzB+I0gbhBJOYlDPCLFpKihXISikFB17li7SJqoyRe7J8sPjtPM7ZXlxmrscbIoN1ylt4LYRFEQr5mLj8uHi1PNj0fJ8mMT1HHi6dn88bHqePACEAW4IASwgBK2DDAdZANJe09jD3xTj4QBPpCDLCACLhrN0IzkwREpfMaDQvAHRCKgGJ4XPDgqAgVQ/3FYq366gMzB0YLBGTngMcR5IBLkwnfl4CzpsLck8AhqJP/wLoCx5sKmGvunjgM1URqNcoiXpTNkSQwlhhAjiGFER9wED8D98Cj4DILNDWfjPkPRfrYnPCZ0EB4QrhE6CbemSYrkX8UyAXRC/jBNxhlfZozbQU5PPBj3h+yQGWfiJsAF94B+OHgg9OwJtVxN3KrcWf8mz+EMvqi5xo7iSkEpIyhBFIevZ2o7aXsOs6gq+mV91LFmDFeVOzzytX/uF3UWwj7ya0tsGXYQO42dwM5iR7BGwMKOY03YBeyoCg+voUeDa2jIW9xgPDmQR/IPf3yNT1UlFa51rt2uHzRjIF80K1+1wbjTZbPlkixxPosDvwIiFk8qGD2K5ebq5gqA6pui/pt6yRz8ViDMc591RXcB8E8ZGBg48lkXBffpj0/hNu/5rLOvA4B+DIAz3wiU8gK1Dlc9CPDfQAfuKGNgDqyBA8zIDXgBPxAEQsF4EAMSQAqYCusshutZDmaCuWARKAalYDVYDzaBbWAnqAbfgwOgERwBJ8DP4Dy4BK6B23D9dIFnoBe8Bv0IgpAQOsJAjBELxBZxRtwQNhKAhCJRSBySgqQjWYgUUSJzkcVIKVKGbEJ2IDXID8hh5ARyFulAbiH3kW7kL+Q9iqE01AA1Q+3QMSgb5aCRaAI6Bc1CZ6CF6BJ0JVqBVqF70Qb0BHoevYZ2os/QPgxgWhgTs8RcMDbGxWKwVCwTk2PzsRKsHKvC6rFm+EtfwTqxHuwdTsQZOAt3gWs4Ak/EBfgMfD6+At+EV+MNeBt+Bb+P9+KfCHSCKcGZ4EvgESYRsggzCcWEcsJuwiHCKbibugiviUQik2hP9Ia7MYWYTZxDXEHcQtxHbCF2EB8S+0gkkjHJmeRPiiHxSfmkYtJG0l7ScdJlUhfpLVmLbEF2I4eRU8lSchG5nFxLPka+TH5C7qfoUmwpvpQYipAym7KKsovSTLlI6aL0U/Wo9lR/agI1m7qIWkGtp56i3qG+1NLSstLy0ZqoJdFaqFWhtV/rjNZ9rXc0fZoTjUtLoylpK2l7aC20W7SXdDrdjh5ET6Xn01fSa+gn6ffob7UZ2qO1edpC7QXaldoN2pe1n+tQdGx1ODpTdQp1ynUO6lzU6dGl6NrpcnX5uvN1K3UP697Q7dNj6I3Vi9HL01uhV6t3Vu+pPknfTj9UX6i/RH+n/kn9hwyMYc3gMgSMxYxdjFOMLgOigb0BzyDboNTge4N2g15DfUMPwyTDWYaVhkcNO5kY047JY+YyVzEPMK8z348wG8EZIRqxfET9iMsj3hiNNAoyEhmVGO0zumb03phlHGqcY7zGuNH4rglu4mQy0WSmyVaTUyY9Iw1G+o0UjCwZeWDkr6aoqZNpnOkc052mF0z7zMzNws1kZhvNTpr1mDPNg8yzzdeZHzPvtmBYBFhILNZZHLf4nWXI4rByWRWsNlavpallhKXScodlu2W/lb1VolWR1T6ru9ZUa7Z1pvU661brXhsLmwk2c23qbH61pdiybcW2G2xP276xs7dLtltq12j31N7InmdfaF9nf8eB7hDoMMOhyuGqI9GR7ZjjuMXxkhPq5Okkdqp0uuiMOns5S5y3OHeMIozyGSUdVTXqhgvNheNS4FLncn80c3TU6KLRjaOfj7EZkzpmzZjTYz65errmuu5yvT1Wf+z4sUVjm8f+5ebkJnCrdLvqTncPc1/g3uT+wsPZQ+Sx1eOmJ8NzgudSz1bPj17eXnKveq9ubxvvdO/N3jfYBuxY9gr2GR+CT7DPAp8jPu98vXzzfQ/4/unn4pfjV+v3dJz9ONG4XeMe+lv58/13+HcGsALSA7YHdAZaBvIDqwIfBFkHCYN2Bz3hOHKyOXs5z4Ndg+XBh4LfcH2587gtIVhIeEhJSHuofmhi6KbQe2FWYVlhdWG94Z7hc8JbIggRkRFrIm7wzHgCXg2vd7z3+Hnj2yJpkfGRmyIfRDlFyaOaJ6ATxk9YO+FOtG20NLoxBsTwYtbG3I21j50R+9NE4sTYiZUTH8eNjZsbdzqeET8tvjb+dUJwwqqE24kOicrE1iSdpLSkmqQ3ySHJZcmdk8ZMmjfpfIpJiiSlKZWUmpS6O7Vvcujk9ZO70jzTitOuT7GfMmvK2akmU3OnHp2mM40/7WA6IT05vTb9Az+GX8Xvy+BlbM7oFXAFGwTPhEHCdcJukb+oTPQk0z+zLPNpln/W2qxucaC4XNwj4Uo2SV5kR2Rvy36TE5OzJ2cgNzl3Xx45Lz3vsFRfmiNtm24+fdb0DpmzrFjWOcN3xvoZvfJI+W4FopiiaMo3gIf3C0oH5TfK+wUBBZUFb2cmzTw4S2+WdNaF2U6zl89+UhhW+N0cfI5gTutcy7mL5t6fx5m3Yz4yP2N+6wLrBUsWdC0MX1i9iLooZ9EvRa5FZUWvFicvbl5itmThkoffhH9TV6xdLC++sdRv6bZl+DLJsvbl7ss3Lv9UIiw5V+paWl76YYVgxblvx35b8e3AysyV7au8Vm1dTVwtXX19TeCa6jK9ssKyh2snrG1Yx1pXsu7V+mnrz5Z7lG/bQN2g3NBZEVXRtNFm4+qNHzaJN12rDK7ct9l08/LNb7YIt1zeGrS1fpvZttJt77dLtt/cEb6jocquqnwncWfBzse7knad/o79Xc1uk92luz/uke7prI6rbqvxrqmpNa1dVYfWKeu696btvfR9yPdN9S71O/Yx95XuB/uV+3//If2H6wciD7QeZB+s/9H2x82HGIdKGpCG2Q29jeLGzqaUpo7D4w+3Nvs1H/pp9E97jlgeqTxqeHTVMeqxJccGjhce72uRtfScyDrxsHVa6+2Tk05ebZvY1n4q8tSZn8N+Pnmac/r4Gf8zR876nj18jn2u8bzX+YYLnhcO/eL5y6F2r/aGi94Xmy75XGruGNdx7HLg5RNXQq78fJV39fy16Gsd1xOv37yRdqPzpvDm01u5t178WvBr/+2Fdwh3Su7q3i2/Z3qv6jfH3/Z1enUevR9y/8KD+Ae3HwoePnukePSha8lj+uPyJxZPap66PT3SHdZ96ffJv3c9kz3r7yn+Q++Pzc8dnv/4Z9CfF3on9Xa9kL8Y+GvFS+OXe155vGrti+279zrvdf+bkrfGb6vfsd+dfp/8/kn/zA+kDxUfHT82f4r8dGcgb2BAxpfzB48CGGxoZiYAf+2B54QUABiX4PlhsvrONyiI+p46iMB/wup74aB4AVAPO9VxndsCwH7Y7BZCbviuOqonBAHU3X24aUSR6e6m5qLBGw/h7cDASzMASM0AfJQPDPRvGRj4CO+o2C0AWmao75oqIcK7wXbVXRfcWjt5yP+wqO+hX+T4dQ9UEXiAr/t/AaEbihZ5unFjAAAAXGVYSWZNTQAqAAAACAAEAQYAAwAAAAEAAgAAARIAAwAAAAEAAQAAASgAAwAAAAEAAgAAh2kABAAAAAEAAAA+AAAAAAACoAIABAAAAAEAAAIAoAMABAAAAAEAAADOAAAAAMJ77Z0AAAILaVRYdFhNTDpjb20uYWRvYmUueG1wAAAAAAA8eDp4bXBtZXRhIHhtbG5zOng9ImFkb2JlOm5zOm1ldGEvIiB4OnhtcHRrPSJYTVAgQ29yZSA1LjQuMCI+CiAgIDxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+CiAgICAgIDxyZGY6RGVzY3JpcHRpb24gcmRmOmFib3V0PSIiCiAgICAgICAgICAgIHhtbG5zOnRpZmY9Imh0dHA6Ly9ucy5hZG9iZS5jb20vdGlmZi8xLjAvIj4KICAgICAgICAgPHRpZmY6T3JpZW50YXRpb24+MTwvdGlmZjpPcmllbnRhdGlvbj4KICAgICAgICAgPHRpZmY6UGhvdG9tZXRyaWNJbnRlcnByZXRhdGlvbj4yPC90aWZmOlBob3RvbWV0cmljSW50ZXJwcmV0YXRpb24+CiAgICAgICAgIDx0aWZmOlJlc29sdXRpb25Vbml0PjI8L3RpZmY6UmVzb2x1dGlvblVuaXQ+CiAgICAgICAgIDx0aWZmOkNvbXByZXNzaW9uPjE8L3RpZmY6Q29tcHJlc3Npb24+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgqmXfYwAABAAElEQVR4Aey9B5yURfL4PTlu3mXJsEiUnBHJIIpgwIgBAyqKBFHMCfXErCgGFHNATChmxQQCCoqiknNOuyybJ8888377KXZefrDLnf7vzmP26c/y0NNPdXV1Pd1V3dXV3eaCkhKTEQwOGBwwOGBwoOZxwFLzqmzU2OCAwQGDAwYHFAcMBWC0A4MDBgcMDtRQDhgKoIZ+eKPaBgcMDhgcMBSA0QYMDhgcMDhQQzlgKIAa+uGNahscMDhgcMBQAEYbMDhgcMDgQA3lgKEAauiHN6ptcMDggMEBQwEYbcDggMEBgwM1lAOGAqihH96otsEBgwMGBwwFYLQBgwMGBwwO1FAOGAqghn54o9oGBwwOGBwwFIDRBgwOGBwwOFBDOWAogBr64Y1qGxwwOGBwwFAARhswOGBwwOBADeWAoQBq6Ic3qm1wwOCAwYEapwDCsWg0rtmcjrjFHNFi8jTbrPF43KIH2kQ0GjXrgUSjiRgcMDhgcCBZOWBL1opVVy+73c4rRDxPhHwsFnM4HJFIxGoyk6JpGkLfZrOFQiHeOp1OUqpDZaQbHDA4YHDgqOZAjVMAlrgJEW82myxms8WiBv7xaCwh/ZkDIPER/egJIqgH4kf1BzaINzhgcMDgQHUcqHEmIJHvSu4rNaCEOxEEPaN+4qRIOhGG/5JYHe+MdIMDBgcMDhzVHKhxCiAWiVqQ83GT0+7gT4vGeDps9nA4bLVag8EgGgIDkdIPxgLAUd20DeINDhgc+GccqHEKwON0mbU4hqCP53x42SWX1qtdZ9yYq7dt3uJ2u7H/MOpHDRBBB7Aw8M+4Z7w3OGBwwODAUcwBc0FJyVFM/p8n3RLVkPLXXHPNa6+9Ru6srKzCwkIiU59+cuzYsSUlJWIFwvhDBB2AMvjzhRg5DA4YHDA4cBRwoMYpAIfJsnTp0sGDB9eqVQtrD4u9mHoQ9EWlxbdPnnzjjTdiCyIFuU/AIiReQ0fBlzRINDhgcMDgwJ/kgPXGW275k1mOGnAkOOKbgTz2HGQ9gbG/x+m44ILz9+0rcDh4ZYrFcPlnTcDk8bjnfvG5w2br27dvNBIOR8J2u00zx20WtTisFgTicbEO4SEKZtAeNYyoilC2PiiPV7OJv6gWs9nt4UiEOBWrCtxIMzhgcCAJOZC0MwBkNL49CH2kNt+Np8TX/PFHnz59MjIykODAkEhE7QZwKZd/zEETJk68//77NVMcQU9GtgFgDkKFoEtkcVh+Hu0KgMpSO2ohxi5ZAxduJGEzN6pkcMDgQFUcSFoDNzKd+sqwHTGnBvC6V8/ChQv5yVQA6Q8AIo90hCDuQPGYlpaW9tS0abfffns4qIb5IiXFSZQnGdEBRCS9Kn4eNWloRJkeIfqpDpWlaqIsj5o6GIQaHDA48P/GgeTfCIa8RsQju5FxbPqdPXt2SkoKUo+fkiiKATYS8bjcLpdr2tSp/Lz33nvxFg2GlS8QwPL0eDyySZifR3VA3KP2qBe6EAVAoDowhJBgyFFdQYN4gwMGB/4pB5LZBIQgQ64h0RBzEvf5fMc0aMAwH6kn43pJh01Ws4WlYKfbBXwkFi3ct2/MuHHYgkLsG6icKyAxycUwORlEpBZHC6Lt1q9fv3LlyubNm3fq1ElNbkwHjj9Khjr+0+ZvABgcqNkcSFoTEPJLjWbNZn10q+w8CG6EHZ/b6/Ui0xOviGAMIUUMQXj+oAzS0tOfe+aZe++5B3jRFgIvz2RoMzEtMy39wfvuP757jzGjrxzYr//9905he0QiUNNE3IgYHDA4kJQcSFoFoAaz+p5eIpg7kO/I/d9//52viDLAjCMDeRFzpEQ4/U1XEph9gHc5nLm1az/xxBMYgsiC7QgYIuTCCpQETQGd984770ydOtWbkpKZmcmqOPEvv/wyCapmVMHggMGBf5EDSasAkOwIawJbfBHZyDuE/nfffcdPv99PuqgHIkCKhoBlaAsgMYyQDnydOnWmP/XUrbfeirWEjLwiVyAQEGXwL7L4fxZs8uTJqamprIgwxSEwAWKF/GBqjUnAwdww4gYHko8DSasAYqY4x/0jwhDZJi0eqvDHAqGvPv2ceQDDeRFtCHQkPtKcn3GrRbOYsYDH4hqin/UAu9UWCYVr59Z69slpD903JRIKxmPsJ2AyYYqhKfQzhWwWq7gPYTXiiCF+Wuw2ijZZLTy5eID7BohoujVFlaIfPEczQtqiY4hEoc7E9mRTJBY3W+388T8/BYnVYScvFeGn/FXXBJnioMakCNFPVFxV0KIFI8GIFjHb0HNhh80aDYfsVsvkO2/fuWOb1WKKRkLyF9eiq1etkLmOYpo+VeIpvKquXCPd4IDBgaOXA0mrAOSTiEBEnCEfy8vL/8J3YrqALWjatGnMAxCsjJTV9mCLFUEJTmYDMocQmxITBTk9gqeUBQylQ4bZZDWhDdAMZhuRWDQeRa5H4zY7ikOz2sxOl52IFo+aLXF+qgUJTZPD6UBC0TxJrK4KSH8mOugzUW/KiuVyQRUWLdSAIlKL8wqCc3JyHnrooaeeeor5jSghcJKR+BHwV1eukW5wwODA0cuBpFUAMm7liRhFGiLa9u3b9xe+E46hDPY5MujFGTNuueWWUCCI2UTwiF1IRLxMJpCw6AZuF+DPZrbwlMsGIsFQVDNz/ZjJYrfaXWargzh/ceYMcUb7UbMW0yLhWDjET5vZRFyLRBNIkN0m3HPkr5o6IL4R97xEZ1Br6CHCT9JRHlDFWyYoaIX77r33wfvvr1u3bllZGQBAoq7IgqqrX7++zB6qKcRINjhgcCCpOFDtiPJor6UoAGqBRFOWGLslPz//L1QKAaqG83FTdk4OOgAp+cwzz2D2sdiUAQcdAABlUQgRlA1nKfBUZWqajLhlESKizEbsTdP0m8fiDPbRD2RBrNswG+lnz5GFoKhlomBRbqk8Qc6TAE7AUGZEqqwIkADIW/KCnFygQ5cwp2BZu6KiAun/7PTpObVqQQyajCdV0NGr6QUK4BDMYEtw8pBXxk+DAwYHjnYOJK0COPjDiBTbv3//X5BlbAlm4KxOy7FYkZuz3niDzQRYhLKyvEjYhIlG5DIpTBeUILYocwrGHbfTVby/aMOGDQUFeyEAJYQlCmAkr0hql92FE052djaH0zVs2BARnJ6e7rI7OKuHpQsVeGIzQgEgy6Nqe7PUQrInqslP0BIY0aOlMAcBRry0tBiEbHLes2fP2DFj5s+fn52VzU+OSQKY7GBGWzAzgCSMQgmERsTggMGBpOdAMisAJCBiUYlL3aWd8e9f+5xgYI2XkTIyvV79+h998MHO7dtffX0mwpolARmn8wrkSuZazWiFoqKi5cuXs+0AV8t169ZBBjJXAIBhmC+UqEG61c7Rc+onROpD+x7HHTds2LDWrVu3bdsW0xNbk5HUSOcUj6e0tNTucUle8IBV4vIUGkCOTyeQ0EY81ZtSUVaO/+uwIUNQSg3q1S8uLgYtSgKcwOglKz8osKGB+JlAKxGeBIo7uCwjbnDA4EAScCDZFYC+/ItwRRQiwf+KFCMno3C2AtjsmHd85RW169T59ZdfzjnnnLfffpsNtLxH8oKcCOPrrz7/bMGCBQh9ZX7RNOw/iGAaihZUAEqUazEWXlgNUBI8Fg9rQav+ERDmbjcC2blq1bKff/7RbFKLwN27dx8yZMjQoUNRBgG/Pz0tzR/VtcVhTQ8ZzXgf/JTLHIXdzmgmikDQ33zzzTNfe42tbepki6BawygvLbM71c4G4AGjUJ6gbNKkiSDm1WElGAkGBwwOJBsHkvYoCLwnEYVY2NVTi7Mk+9KLL3Lcf1ZWxp/6hhjR1TIpo2XuDXaoywPCuO9oWigSYxz9+uuvt2vXbsmSJYsXL37xxRf3FxZiqcGzHmAkL8C5ubmAMbLOTbfjflO7dm3ZhwwNSgGwJc1s5QhSBuyYhvbu3bt9+/aCgoLy8gCLwQzkWblFmgPZsWPHiy++eODAgfXyGh1Mf0JSE2HJQa306udVQDO5Pv300+smXlOYn5+RmclbZiEQhhpQRbPUrK9VkE4i2VkThgyUBkj4yewEgASdZDm4XCNucMDgQBJwIGlnAAgvZBbO9ozNMeDYnA4ZEf/ZbxaOWyOhGK6ZTgSlFkGjuDH0a/HU9OySoqLBAwa2OvbYtWvXcJaQy+N2eV2xQIXTFmnUuM6gPr1aNc9r1qgB+wEcnLrmsFdZNIoqHq/FKwQuYhe1wVIBgnjp8p1Lf/1lb76fXQBWp2vDxlXXTZoI2OixV19xxRUM1RH0gbAS97Ls7IhbQgG1wQ0ffy0aWbn8jylTpnw1d25OZlZOdg4ZZf1AqUM9cOoDO+WIUigL2hV+f6duXWNmtawN3wik8xRg/beCJIBBnqgNimM6I3qCyYQiRkcoSiihQqRqSveYzQ5cXCsPVVXZ9UTgIU9+gh+EBBLBabIrIkmUOQqKjdJ5a9WtaJIFMNQYz8RPJl6AkYUnyxsqLy64VQWpI2CJl8QJrN+ATYpm2sQkiUIV5ZpahpEgeSt//fX/KS6RuTqc7EpBMfMWqoTVoralXmqMolcWPkAnkEQSOA+OMAhQtaj8WIINABJhGhlBy0/hPxEGCv9rQT6Eok3fVUMVCNSLn9ICASDlYP78r1Xhf4eepJ0BsAkLqcHAX8kFs8XtcE5/5hn8OP/sDMDElTDKl0atl9JJCNGw7mBjQ7UEEROYl9gVVlrqy8zyDBgwoGvbY5s3b1onJ9vltFWU7Pc42XgVRvpWpwDUljE9gJ+CCBRBU7a6cujSO/fsXbV67W8rVn373UKlY1yukrIAkOddMHL06NGdu3ZRvV0XnXj/kBHx/dtvv73wwgvvv/tuakYGYsukrxsf3uAQKMFwSLqK2Wph7jJx0iTOvZDTTw+HVzXVKRTyoA1S4S3sAYkqSN9MwBNhBBmCgVfAA0mHJK5gwsowJT9VbXVxz5NZmgAQB4A4MMBHTPqspdKxVQQcDEMBwA1EsyxfQ4kQJtkFLdnBIyns7xOSDnmSi5SDRbDERVCSl7fAgJ96KZlSKRAl4yHY/vLPBAHVosUVWOe/QAJGoHaY8qBN2MWTtxJPIDyEJOolDJFPABg/qRe5iMgTzKSTqNTqQQrvEFR/10/RdtQajUU7hE5SmNcSRAvSGoUPpCS+199F7f94uUmuAJQnPpIxbsKv5uWXXpo4cWJOTtaf+iR6r1CiWVMCVnU5duSqgUawwum0h4KBslJf+7ZNTj75pG5dO9bOqWWKBGiFkWgIt8t4DMO6ct2hgUaZPVQVUFR0RQIvIVVAKA7XTeYtpMQtVrcnbcOmzYt/WvL5F3O37S7n8rKycqUGrp103ajLRzdv0YqrjEuK93HV5XvvvffpRx+xS401ALKC1qwvPldRMgJXH9NTF2YAuCe9/OqrZ511FqefVgGsL4HIGRiqs1WuYytJoZvaqDFliQQhkT4JElLopSQSl0QZlvIW2kTWyCtV35CaeQgfJCKdWQQceCQRSCIMVGEvAMSFWtIJIgqpEelCkuSiuOoEWQIDMIJKnjIjkTgw8hbyOCk2keVg+H9LnFKqQ453GfVKEEllicNDPiJZCPxMVBxiAKiSJPqCfBfequroaz8yv4F7koW8CrOu16vjW5XI/wuJ0Eb7kfpCofCEn3wv4Z7ihR74Sahu5vdfIPWoKCLJFYAydCAj2ARrtX3w/vujRo36swpAi4VoRDabi6kz/zHkVEYWLpuMlFWU+bp1bXv16Mtq52amehzhkJ/i7EpHaIh1GdBTvFJAqq9WLVjZHawo1IfPIjdp04RQwEcpLo/HFwhylIQ3NYWxOQaiH5dumjr1SbvT7nR5C4tLsnPqXHDhRdt37vpg9iyQpKalscaApJZREnhkZHR4WyQduS8lEinIz5+/cGGHDh0i1SgMZlEgUfY0XdBDMxFSUAAIDiSICF/SVf31gT+1BgZKqAgim0RSIEkKJQ4wCGW8hoBDPSiCdVkGn8lFFtZdiCCniINN56Qaq0IPkBRKXJHxfzUKeEghXYQFYMxy+FllAA/pwEsWEBLAL3QmigCGFHhVJZL/dKLUV5gAN0QNK0JtVhZvEnwDADbCVd5USRIzP2pBTYEUGCJkl9rxLURxSl6lHqoxnVWJ/L+QSBuAYOEAxRHnySyQmY3EpWrEqR3fzmpPWiv3v4XbSasAGKfTsdlDq6QGm63Mlh8WLTrllFP+rAIwmbEjY6dxs1GXtuZyOZHDxUVFHZvXO+PM008cOCAUKHPqm3dj0SCqgQ5Gs2PIi2TkYJ8IO4ArR6lVfjDWAA4XQKRY4qoP4/6pBjGMelhJiOhX2Fsz9xUWffTJF2+9+6nTbbfZPcVl5S6XNyXdTS7kJqJBOgYVVyXqpoPDi0YBIBOlnxDJ37t3T0EBP1EshwOTIiaghIikBzIB37p167x589hizfr2cccd1759e3xtkSZwQMQQ69s//PDDli1bcD06+eSTGzRoQIdEPyFopBTpycpkxHlNoRCOVXRmXGDBRlkcToc/FZD4QbGcDjCsyM7K3LZ9BzsnoBYYqgw2CmUWBElS6w8++IBSCFdeeSW5ADuC4E7wH4TEwUD47JNP2TnBdooRI0aAQSQskUMEiuSFQuGMVOpfef7ZjHwvKQLaYBRVmzNnDvFWrY+FV8h0eMinBwbMUpEqyWCkDA/JKOoNYH6CTb6IZHz11VepL5NIXN1Ckaq9zqpE/l9IhHJKkXYOkRDMTz53SVEx+2lOP/10EmGCBNVxqjH9/RdIPSqKSFoFgGAloABoBCgAbEFbt2zp0qXLn1UAyHM1+owgF5xelzMQLA2FykeMOPuckwenpnnDwQBTBKedYRIOnhFunA8EOGiahVlZKVUrtJCB7JaGe3ibYAYgjfXgV3TLaNBHU6ZL01HJK+NQBWlxs5nY4fRu2Lxj2tMz1mzYmp6RHY5qTNWpKRkTTV8hqRwgH4z8QBzhqbMI5Pg11W/Y8Ndff0VeO1zuKoAxF6CS9DG1dDlyff311+eNGKEEE50QJRePT3388UsuuQSCIRuA77///tRTTkHAKAA9PPjQQ1eOuxoMwEAeQXDykyM3gD8NeKuVfXa4PIHk4YcfvmfyXWD+5ddfGzduDA5qhOcVHlM33XoLuXCRYsbDmvnTTz+Njrn22mtBDkzXrl23bd6MjQOFhLBAODKTqLJeiqOVqw7khSTYTrjowpEfz5nDD9QbAFI06QmTCJBVIjwkUSp4SCI/D8leHVgiIwqY+kpGyGAzR9O8PJrZQ48+MmbMGCQ4RJIOHvlA1SHkOwIgYNAAzlWrVl1wwQXLli2Dk+CHpQ3r1SPSrGVLElkoStDwvxCBZr4v9MtX4ydk9+vX75fFS+AphziyBYeWw1uqpj5o9TO//4Xq/O00VD1P/NvJ+n8nQDoYT+kJNAVcMP8C2jhnNljM6nQ2U6xwz462LfKeevjeS0YMc9o52yHIcW0er1MNz01mOwdN0+QwFnHUDwYTff+wtEJ14ls1gdGcNGjeC81EoNnpSsEaz4pd3GTTzTLIX0Z2bEeIxMMVsUhFk0Z1X5zxdL9e3cvLimORAJ2fQZzM/ekAIgVEclVZMgKatxTEqBYBesYZZ0AJI+gqgUkEIZBQSIRc27ZtQ/qTyj618ePH52Rn82rSddfhd+RyOkG9a+fO4aefThEet/uaa65p2rQpALewI2HmTMb4UkdBBTaIh1EqnR/64h4TCOxd1AidpoA1zetBvTqaN2t28003BQNqCUTGrcivvEaNHn30UbJSKXLhhqtkma6TRBbwpPQqA3gSAQwEwCBbsUJXDODkA0EqJMEigAVPIteRI1UWSuIhuf4pTgDgEoEIxFB31Bv8Bw/fBUaRLnRSBQETnIc/EwD4GXPE4eCBA/cVFMAiPgEZFWGQpwvWI+M5HDMph9Qr8bNK4L+QKERKw+bT8BMkMISPxSBGPh+JsIInlUoQYESq5EC1gqlK6KMrUZojbYJAE2GEKCl/qhbSmJSYDgXHjR1z3+Q7m+fViwfLXOwJiIVddhsSB+QMNLD2qIVim51lWwQ3Z/3wSxeyDNyU0KwyAEA7pqfxhDBgwMaTs4M8KelRtYZhNeMLaePwUSYEzljI73KYw8GKuBYsyN998003NmncMBwKQidDXYQU2YlLWcSrq6yiWTd3UDRF9urVC+WBKDkCPAJaAOAkg3RoRQax941rc9gJwQScvBMmTBCp8dxzz0EGpXz44YdI50WLFskQ/poJE4RO6gsBBGjWuaR2pSHkwIl0I05xzAPYGEFo1qyZfAjZzo3k4i0YoESRpE9NMNeAikDdya6kmM4K4kfgQ5X1hSqljcimD7oRr+BkaCxytsos/5ZEyq0ODwRAjtQFVrC5ZNOmTdt37Ro1ahQcgKXkhSESjlxfIOEwkOBZs2aNYhS3YXs8fDjRcJICJ4/QHqqj8z+dDtnQj3qGWshD08MZxTd9iQj+qPas74lB+h+ZD/9pUo8K/Em7QoLNhw8gN9zydKd4GBJcMvryV59/HvMxLYbGQWNC9ChhoSTvAdsofYlRLW9pXmrkGE/1+/LbHtto/JhL2h3bMlC636RZnOb0aMyv54o72SFM0GfnVkzl4gBD4TEWIa1YIBHfR3Cl45xOTprj0FBwkEmRLVIAv4ZwAJ0AYQq9eqgQs9g5FMihH92T6rIGfPvuv23iDTfcsDsYttndFocrHOHY/4gVtYGLJtWUbIc90VgiU3Br4SUbzZBuvoCqVJXB7LBFWdfQcyE+XnvpZcA8LmfLNq3KfWUcYnTWWWfgflpUVLhm3epjmjX78svPYT/s7dmzh89Xnp6eevXVV+GGi2ZbtuTn7j16UFlhPgvmeCTBAxDGdR8kFmyQRAj3hfO/37B+LYuc46+ZsHP37tmzZ2PUg0Gff/5pJB5t06YNmmDm668zQeCo1W/nfhnx+Vp1bHfK0GEcyMEZqyBEQ5fHYxjuzZEonk7oKmYMiDaWGS677DKlV3SnVShBGtIqeMIWxRlMH3GNXRSRgP/ZZ5/98ccfUZA9e/bEkUwkC0+UH0YSdBKoajVqiIer1+HirD0gfX7/uGuv8aalmkIhINF/WK7Wrl17zDHHTJo0CWU58/U3KO604ae3addOKbxweNbMN1nz2LFrJ8b3Fs2as+zRt3+/KEtZNivfxe3krG81GYVCp92Bj8ALz05Hanfp1hWqqNFXX331888/g3PcuHHLly+f+d57q1esbJ53zIUXXDBw4EA7MyouvYtyUhUjE8YSaiFh6iOP7svfg58tQnPa449BNrQ52XihRTmiRBkuw6GHHnlo0YKFmenpfXr1xr6Xmp7mR9lYLQ6ThYnjm2++ie5nOojNjfNLcIMOYM3TjyUHGy2KzgVv+RDcSEHbxtfrrbfe+n7evP37CuvXrUeru3LsGLqhKDYYRafDtMjFTX+sWpmbU6t927YXnn9Bq1at4AAtGbJpIVD++jvvcHsdnGfgcscddzDl5Jh1qOWkdXMsaqG/RSNzP//8448/3rpNLRdB2MiRI2WCC7fhksWpui0fmqcEEgk1TWck7RqAfFR6C82Lj0rg87Mz9uLzzqNBSDowtAAAVIdH+LCw6VRnojHSoik4dNHgLyo544whl158TmaGK+QvjwWjHleKFjbFrMoK8TcEm5ofcJEAhNPamRPwt2L5qjE3359du3YgiLuLGlBbkKexKJ05GK12UA8UVa7w+1gbf33mTKYydGx6WZWV4vIC+mc0pLyAcMFMT0nlCFNk9Nad2wI+H3x77LHH2EYAWa/PfOPU00/v0qHj5s2bYSPdFUmBUKDnX3755eDHXn/pqFFBVhd1cYZEhxJk0M8/LB564kmUPuP550decjES5LGHH7n3H3fz6sclizds2sSEgLVOp8MJThTlPVOmQNLt3NMQ0xgJorkZsD8w9dFrr5k4oF+/nxYv4aNjwi4pK/OkeD98dzbZpWoUxyvi2KPOOv98dt7J0L5yKBBHBF99+eWIbFae8YxCJFERctE2yLVixQpmM2CgItyqBkMQWM3bt0PApTjdgwYO/Omnn4DcW1SI84DXZuP42Ntuu00mE0LAhSNHvvXmLDB8MffL444/Hs737d1704aNFIHba4hzvHW5dOFFI59+7lm2+6HAGAZAgzpdnGC2ILiPbdkKDNOeevKiiy6CRS+99BLjAGwfk66/nvmZch5Aq2pMqKwzZswYcd55xRVlTo8bwsCDV3RZSWnj+g3YKcMZJ6rNc+NQNFq4fz+85ZAr8mK1Yxq0as0qE/ZMHVunTp3mL/i+rKKCPY+7tmwbPnw4XgBgI0AJTzTEQ48/DjEIWbobrYtECOAnHl6spvQ+vlcJZzLqy3JqgMaIx2FHLzKh4ROwE/7qq69mKUh9X7ytxCM5bnrkkUfGjh/n4yY+M97XPpyVWa8iLyMHni1btaJ1/fbrMj7K6tWraTYQw1CD4Qj8xIWCNgkYhy2iI6kUb4FBAZBOnMBb6ASYpwCTUkNC1b09CSpPm+aL0hb5wMRphQgLTtThAxP4SSPjLRHp9lHNwkZenImpO+3A6aAR+/CtuPXaKyaNHeW2xaKBcoeFVqJMrtzm8nexCGpxNaJquoxHGSASg63btLrhhgkl+wsYIXHDAHd+6aNv5bZ4BDqpPqzgKDqx/8AoflYHD6PARq+GOUQYfcMHFtzo22RBE2CokeysJCOOGVwDCc4dO3bAecLu3bsF+a5duw4uSMrlyVvS5ZVQTnHUkypDKmNnFhuwl/Hh+vTpM+nGG/maSOfzzjuPt8jQ/v3733rLrcceeyx5lTLTJ1IQyeh4586dSH9Q4TDDsP2bb75hSAgAo0LIRnxIERBANQEDAzKCn1QQWYNs5YmxRei/8847qQ51BBKyKYIIASQ8EZqKeN1kT4QDAbGzk5FjBN+cNWvxkiXjxo9/c+ZMaAaYyiL48OfZtHEj9LCNY+XKlStWruTMErKgMiEPkgCT6lAcgZ/kJQIM/GfNgwhVRoITmfrYYw888MD8H36Y8uCD/ISGu+66C3gOhYVC8MinpNxr1LaYHBQnVWD9/OZbbqFeEAYAGTExsWw299tvZ7//PgwnhT2GOH2hFMkLhUh/eIvLEIdfXXfddVD1xhtvfPTRRxQBAE8C2EDOd2TQcP6I8/DUSklNhYF8gsl33UVrgyT0FiUSSGcNCf3HjArHnpdfe+2Ypk0pl7K+/fZbkIANsf7L0qVIbpTNd/Pmvf/BB5Ty27JlgMEQqgkMA3/AiLBnftGiRXAVGNY5WIsCDEbxjShX4KkytQAYGKoAQI0KSWsC4nPyIfnMPPnefGwsCRx3zDAB9z46g0xUpS/pH94cCEfw50RqMQ7el7/vmGPqT7xmfK8W9cPBEo/dFAwF2QiACcJqx61F3dHytwR1s5geaOv0Gbq3zYrBxDdkYL91a1Z+/Nm8zJwc7qlE6XHOGzKGQWiVdMIQmEPrB0vv3r1hEdgCoSDPKuGFn9JD6DBjx47Fmk/pXC525+23M0ZGEEASKWQHM96TDLhIYTR6++23Y5QQMaSLMrVcCRg08ASGJ/jlJxHBo6oma5L6/oMOHTuiVJ556inWwk844YSbbrsVyQX9HJLKaBpIbCbjx46riKoNB3xroQRSo8Eg4kDQQqSceIpMR6NQ7qxZs5AvZAGbFCc1hUIiPDnPFQWJ1GBVgzkNwo5VDZxwZPkB/Go4yUkfnBSiKqGIBy0BA0VWrZyZ33xDnM9B0QMHDkQCPv744xz3NPvd93QolQeVyX/quG9NY/LBcJjz+84991w2f6RlZQIQCuNp5oL5fB31geJK7vPJoFmeFEoEJKjQiddei6QrD4e7d+l6x823qgX5Xbsg2J2WQgWB1CsbYtLG2syPixcV7FX3ZDCVoQooV97COgol8d1337W6mUbaOPVk4fcLSIFU6rV56xa+OD/FXwsyMEOh6j777DNG6/gUsV6C3Kc4GCjKYNmyX5k5oY7QNGhECjq+d69lv/9GfdHiUIXmm/Xmm+Acc/XVNCpOJeFM9R7durVt3YaK03gGnnACrJ5y772wuEWLFtjlQALyli1btm3ThozQTIkk3n///fzEKsUUjSEdEc7puvyyy7DjbdmyBYMSVaDlgJbKEgAmkJ0gLVxSasLzQGNNvqrSFGiX1IsvTSuUNk1rw1LJTxocgQifnB7FV0eiM8B32rnJSyvaV3zq0P5PPHpf25ZNgsESda0va5L2A6cJ0UGwQ/5dHINguhatlgiCBXu/jQ6K3Tbmv+LiC1o2q6dF/BjA6elqtF6NNId4eg54kCN5eXmcZycsOkKlYBeFChvpY3gNYTgCfuqjj2LRPvHEExlL8hYwEgHAhfzss88mC3dPMpBkCMkcnG/BWwQNYNL9YL70QH5CA2+JkJ0IMMhxInxKKgup9H98cgGgIIgnkRSeQJKCbygZCbxCu5ORADCFMm4lDn6WATi2DxMNo1f12mTCbk4WXjGiBwkpZJESiYC8R48eNCToBAbFw5NEJgTUgkQIIAsEQB4VoWgEMdnBQrmkUBZ4kC5sksAGQgrFwS4hm1cEvJNBAs3MZmrn5jL+RWOBGY4pvaLPXymCLJQoFJIuzRuEpHvcen2ZGVit3bp1Iy+5ILV///5SKTISoSyyQ6R0CsDIziswILL5SRYKUiyIx5HLNBJKgeDERRHCW7SgAtNZyticrR7IVuZe4Nmwbh1KThgIgIDxc/3atSh8+HDSSScJ5Rw/9dobbzz//POYfaCHIQLwBBQ58BCGmEZYcxouaJl8oMOwEeEvgdim+QknUcwogDp165JRmEPpzF1AyDIPVLFEwZwPsuEMHPnll1/IAkKyE+CJBOpLolCrU1FTHkk7A+C70hX5qHx7ntLiSTzttNOmT5/Ol5ZeTYREPr+DnqVFQr6g1Ry5dtzIc888TXnaRMvx+46GYzRkQYJDaExtpaEjqSHSfz9ALZTQ1hGDtGC6YSBQTi0QqBmpnmvGXTl24t0Z2Si/iJrJW/QF6qqopMpkoo89cds0ejhSM9FpqwJXo3V4iJqEnwzNMDswqsIo/OrM1xDG5559Dv0NEz84WZhVfSsae/nll/v27cuQEEKIoHoZ1UI7opaywrKOQbfUpx3USzowpRNXNdKH1VQWqUSiSF76KOZp0COtACAL35GfAEh2JCNxcvEEgAiQmICkvjiMAwY8P4lQECYCJC+aBlYAT+2oJk/BJsN84TmCQxnH9YC4F4RkIVCK/p+SiarqOsHEgVGahtP+cnMBADPEAAxyXgkZpDADQ01OuvY6CqJZYUX5aM6HZD952NBnX3jenQK7OJFQreVQY/Ws1ATgFDqZIqi8+lSDZk8ijkGO1DRJBIziKBeuUKhQwnosbyXomNWNFypjpScYYEoBsCCsawhgCKoN6NoCPPzE6UtPVsxX2AlmM3azevXqkcJXhhUUAbC/wseHI5HmARRdjnTFBN2pnwaoqNPpZGbGN5WzvPguMOeLL76gnqwkoQPEzMVoQ7VD3b4E2UwI9u7eA7V8FxQA+MHMAg8tkO2OiqpK2w4zGMAojiQKPKTNS6UUfI0JSasAaKm0MBox35jvSiBOH2Z8hNhiJoikIFFaIZ/bpkX2F5V07tBi/NjRrZo38lcUsUrEUT9WtzpZwWFzItGAZ6jNjih1uAR7y/6eQF9QEgfKeVKLaCyMWOAaYZZ727ZuecOkKx96/Pk6dWvhCqF6i3SAw0glO9UhmRGu6qiVMrSypxyagZ4MSxFaPLkYGUcdBM31119/9YSxbPpK8XhZ6oTDgNH5FW26rzpGWOzsyFaGkFi6IZieybBOiJeflE4iaBNCRESYiFEgIQXMqqa6sKAzA6lk00G+j1IdUPnDAWZEICSXfHTilF64ey8/Ga4y1AWMIpACoEV7QB4pCBQRDSQShBgEELNGmg1SBj0hSgI8CCARlyCnaPISieJ/YneypKl4p38gCObGCG7jYewPGHWkysJ24CmUggQPo/5BAwb+8ccfrLIyR1m5fAU45s6dixHj5ttuBZi8QEpeqAMzFIKTVyQSUTSznUI3bQujBDN4gAFYQeoRqSxPoYeMYIC9RMCjKo4gjqkD/sAAKpyOVI30AM9tVhuTS36B4cknn0QpEpGfRMqCQZhDBJzygWA1aBk0oB+IqFkdBeHBmZoCW1Jc6hIMqSA1AQ8fCMKiEVq1amNYlqSdYLZVBzFBm6Zh1GIYASpoBoYb94QbPPleIKEgrJRsPmd1GzKoHZ+PjswCBj8B4CQRngSKliBxEOrJNeWRtLWl4Upno3PSGnjySfm6tDZ83ejJNG4+PC2Dlkp6eWnphSPOuPeeyY3r1/WVFdvNUVMk5LRy1mbc4WZPlmZiN5dNzccRYjSbv6uBQDO9HpqpHWSHwmoNViVGQyxch4L+YcNO7t+nx97dytpwBCKpPj2BETESSq+U6g90vOqyUJbwEGB4iB0cSy5GHtZ46YFgwy0PGKhiKkAEswCmWMxETO1xu6LvMfuGTvDj/EfRwBAHJ9khlVfEBQBslCIyS8CABLPqujoG4kAKByQLACBB7hAgj1cEEnkLnuOPP15+Iq0wKTCobNKkCR4jvGXwyCuKoyBpEsRpPEIM2Bh+AkYEPFRKuMpqBMgl8JalWqZERBBJjC1IF3hw9u/fX4HF459//jk1RWyB/J577qFQqTXCFA3BQuW8efMwASHxMVxgYyETch2XGD4KJBFIIRelSNsmRVjEk3SCjI5JB4BclA4Z8oqyANMJOWBqk5+kE0jnJ7nIAn4RxOARJEQIvAIM8VpWXta5c2fiBABOPfVUWhEmPg7twEQjY3/gYSYfAuTSbDDUSBbWCeADcbredTdcjzWM9XmaB+2QBs3fm2++SS4oB4wh/2uvvQZFvIVRYolCz73//vtY/MBAwBN3z+7dZNHJjKF+gCQLc6/BgwdD2JlnDocSvhEI+e5A0pAgFe8vlI1UTXgoFNaop/qoyRpoBFSNNkrg2+sNWwvGgycMOSm7dj2T1WWxcsSbhXM6S8v2X3/z5VeNG2Gxllpt5RzhQyvRzK5YPM3Fpq5QORZ2ZrEmToPnjB7WALBTxwKROM6IDLExUGAIVn94rrFM7LLY3FZGTWrJ2KSZI5py4ePIB+XBzvlr+Nvh989oTE/BuR6nHtaW2U7AH86bcYZfzE81yglRFOBqzqEWntlswDn1EXWRWBzPZw8nXNhsXmQnAsbuSqen2c1+azj/pokXIcjVjlx7CidE8Gezqe1RGCegllPOrDbNbHcEQuErxlxd4Q9GOOTYzAiR+86q3QfAoSpukKgr7Oks1pT0tKefefrred9Nufve5b+vmDbtqblzv2b0+fDDj2ZlZHscDOvcTzzx5Lx5399xx+Rly35/6aVXHn10Kip43HUTc+rVwfeQaZT0ugi3LkNTMKy8WXBjUps2vLCfr4bfui7nrTApEginuLz6GrhlwYJF+/YUbFy7gSUbOIxGhP6NGzdv3rpt6+ZtxFHViptmC1XjcI7LR10BHkqfMGHinj35FRX+e++977bb7hgyZCgynYJUlZhYVc4XiUM/8GAeNepyVg0YRbz++sx3351N4vnnX8gVzlarvXbtuoL27bffrdhXFK7w33nHHTQ5ZC4KhAZgDkYwefOTRBYh8U1EuqF+tm3dajVjyzaxXdBhsb/1xqxLLx41btyEV199vbCopKSswh+m1ZDJcsLAwbQvW8zi4GApPHTZaqgf50lWaofPmn4KiCmAgUg1N0Q8W9BtTrcXeL4UvraqmelNjrwkksfFcYb6Tgt1ZKHKhaeAbdWK1fwRg9UWuKq8J7GecZQ60ynKdip6yMUGRZOlS6euLVsey+cePfqq779fCHM++uiTSZNuuOiiS+6++25kN6QjhZGtSF4aHqH7cT25mQIG3z/lgTdeed0Ujb/12psfv/PBij9WbtuyPTszp3nTFt26dKfpPzXt6SefeGpffuGqVWuGn3k2hXIfxn0PPMRnTfGkPnDfg7GItmPbzptuuDl/T8GGdRtPPmko3YK3kMFH4e/aaydB5Keffk5rRBmuWrn6cmZYo0addOKJaAncu3/+6ae2LVu1PKbppx9/wvxGnXNFd4R9+omHEF9zQtIqADozYxDkCOMaIghHUujY4Sg7bFNZBijZX4gcDvpLstNSnnv6iTNOHBIqKrFH4/Y4PQ1PH6fGAT4mayiKsHRZHalWK6KWoRadBznIdBv94YyZbVyvHrd65C9q9SBisSowEWZHjH4gBFeg4Gx9oEXRExAHUEVEknD+gDZ+YoYPhvFZwORNtzRr9tSYxcNdMGaHB10VjqlznblAGOFN4cgYZjV4cCPdETR2/doD0IKTJ+fqTL5jYmH+3mhEzXbRLoywqD7DJVihyyM15q3XoAFD+MoUNVimuwpVVT4FPxkhmHF0n/796U5MBfCtZDMO3Z7BIMsADK+wriDmZBF40aJFLABgLCI7wzG2KekWWDUb43NQKAiJI4WJMHXnBdRSBD8hFQACEQAgj+PkAP7uu++aNm2Kfw5xhvCqimYzshXjEoYmfjJyZKBLhCLQK6xy43wCML4rTZo0AQnWYcjgTgWc2eEANIOBQFmCTVLwNGWwz7Jk3bp1qRpVYDaDUz+0EQYMGCDCDqM9OHEwY10EzggSwcBCLv4q4ITEaU88cdVVV5F+7XXXcTmbstgoq0ycuVT7Dh0AuWL0aBaEuQeUcTE/Gx9zDKNjyKNciqMuFEcQ/IKTV4ha4rRzlaKPeEikFHioftMqKu1OpMME0glg48nCDDxyOJ29evUaNGgQPEkgBwNlkYWnwkPQ8/ITRrG0I+nkwgsAOsnI6HvKlCkUAXKxzyRmFSTSVMAGGlZ9McUgk8GDZQamSfukV+JeDMDkyZNbNW9+XI8ePy9Zws/77rtv4MCBRJi4s+lB4nhV8Wmw6NIAGOmDGUZBEiXypWRdnS+FSxVfn8V/CGBpnRkMnKTiYGMFADpJJ4VcEJPgIW9rSLDeeMstSVlVviv1okEQ4dMe+OT0BEZGNvsnH38y/9tvstLcpftLHvjHLW1bNImXFTvZexKLhUJBm4XxshpGIqjDHL/jcKsxAnZMBusMmlQkZqfx0LGU2GSspAYQdK4Iw1d0BGuvLnVAEJ2bYRh9F/mPDJOuBTE0OJ4QRnDqU29Kw5XHDHpQqUsoHWgfjiBCz0A9AzgmABCnnnYlNNVohbEgrZYeBa38U+04IuZacDdo1GTNuvXbd+xwe7x6K1eEIkacTlcwpJRBcVHp3f/4B3N5IUZEg2oJlbrqkFZBZkQwPQSWEmHuzH6cQDDoq6ig2+fl5d1y662oAbYho2Z4IkpOGjIEtPRtel39Bg3w4p/25JPqXGtd2go3QAg2ymJpHZHBbi96NcoDOxKvtmzZQjoyFNWCyMAWf3yvXrjJs5DI8izbiVnOyalVi97+x/LlXHoMJf0G9CcRQ3xmdnbdevVYgaA4UA0aOAicTPwxJeP43bpNmwnXXIMgZuMVdYFmYCADzsinWfbLr06XC7H45FNPIXdKSksR8cf17PnSSy+hFfgJMPU67fTTi4qLiSNosDijhDDygBCLx3nnnw8f+J6Qd+6IEdDPJ8fn/a677qLKH7w/B86PGjWK9WFYiuskcpyMnL6JTMRUdelll+EN2ahRI3gIbUKVRCgO9sIH1BLWLTgGQszibOPCcR6jE3YYYS8nPeB3jwREh6lmo5v70aOUqLcHDVcfhCMVzMzIRH3iu4XwXvrLL9DfvEWLk4cOZScd7Yd13a3btzc55hh0Ht8aMho3anzqaadRI8ZWsJRa4Jnz3IwZjfIaUy+6m4z9IZ4ABpbQcmvXxsZVgdUFb1f9Y/FZn5k+/djWrUmBHpCceeaZMIp91A63Oys7u1fv3tOeeoqWBk7kO/N4jpZiYAE3aHtU8/Thw3FFpWFAEqbFocOGqbpkZZ151llQxUiEKuMj1K59ezYq8rHosqFw2B8IrNu4AYVNS6M6cAOWEkCr+nRNCkm7E5gPL02cj0qEII3e4kxd8P23Zww9uW6tjLLCwttvGj90cL/iokKshur8Hjsj+gOSGtGqzvfXpTtiHA3AhAEXOQQxbZF+opo2bvZ2h41Buo3FWDOLVGGbSmdyjUxBqrEyiP+oFo6wCkY6xNAZ6B5EoFB1jKBPn4bbKYDZLk79SHlFAlt6ld6IYBqPhv2UyVnTGHsjSqkxYnMzF2e2wYCNJ3lxDmduItI8EsW4lbqvxD/i/CszkDt6m5bxkT7gViuHJeX+tevW0YXoPGIfI6+iTT9B4fAuoBY+9LU1YMhOFiS7aAIiSGSZXvCKnk9ZvELy5uRk7969hwhCnOyQ7g8q9QMA8ADTb4HnFfh5C3IC8LyiJyPlSYR+igCSLBApr5CS0CCopIK4f5BCt0c48laYTEbBIAWRHVlGlXE1IU5BTLZAArUk8gSMn9CDmzyUgEp+4oICQmQTHxGcSHmeUAKFvKI6EgcDxBDAAxKe69asYnUERYso79ypsz/gpxaMeceNnYBOXb1mDT6y6vPpARoELTBoFMpCKcIfSAUnlAh51IhXKAyelEsW4HlLEJohlQhZ5Cn0kFfwkEgukJAiEaoAKiV5K1fISQeJ+i66GxXA8JbsJMIT4kQolLdCM3HmAdjlGbRAMCTBCpCQhbwQFg4q117iTMj4UuQiQi6IIS6UAwxm9VH07Vogh70AkMKrRLuCVL6vFEFGKJEiKI4g34J0CCAjAWygJQtxKYIq0GLJCHLqzhMYfipWM8KrSSFpFQAfkY8tn1ZaD82dxHDM1q/v8fk7NgXLfaMuHHbFRSNioYA5FsYmzmEpmNax19DCcOR2I25CwfSAr7ysJH/Pzn35e8tLCov2F1Tgi4bZJhjnfDasSWkZ2Vm16tSuW79W3frpaZlaZm0fQ2yHnck2p3iiU9ABTqstgv1Wn/9CEq2TOI2VOP592HVNNiczE3bwQonbZnKgvMqL2Tq7ffPGvbu3Y60K+MowC7nsdnda3UaN8uo2aFAvr7k9PTPsdJUwynTYXeRl/qGP8vRi2ePvnfX2O9Nf+TAnJ4segmWJLkHTp0y8L6Y8+BBmGYQszKGnQRJPJV+qufCEOQc0S6cCEiSqt6EddW8K8NN/AJAuTbpwG4YDTCJ9jwhlgSFRIon8BJjS1Ryr8pNJXtKA5BQE9bLSxAEq3pJOicJDwQByej79nFUKIsDTmRP4+QkeKRcOIDqhNkGGVAdsBKFW6kKcIIkJAiBG6k4EMErnCVU8gaEuZCEi5PF2+bJfMVnI8uycDz9E8TALwR61ZvVaVquR73wCVQoDC/GbZKxQueIq2Kgv2EiUulMu8GCmIsJ2KQ4AIrwlIhmFEn5Kdn5SBHQCQ6XgD3FSJCMzIXkFEkkBnoycK8UTlqp45c0/ACDoKUVYwROE/CQvuaRE4RuJ8koSqQI/CdBAkCw8CfyEAJHyIJGlDtIBFiYLjBjNIIayCKCVD012IY+s4OEpCIEhu16mmoiQiOYgCzMS4GkwQAoe8CuMld5BKl4DQpIrAGlwfEc+M42YOfsjDz9xzx23pKU6OrVtOmXyTfGw38mZZVhTHHYGv7qxJua2WzV/Ga1+x+YNm3+YTy8N+Csw1nOyCm6gWCxoJKynKrmklhk11hXY6sOo3OZwtOwxoHmbNra0VJZTbc5U7oNh6RYv0iA3i+mBVsjYB9kk/Y3FNVZ4sf6rZhoOpDpMjpj/j1+WrFj4TTgATBSBzLlVaj+vPjThKtxAIKjRNz3eRq3bd+o/yJaV64+a3CwvK8sLQ281oFbLDw57uS9wzmU3Unc0E4M7JIDeK9RZkj/8tIQSpdskergiUHWHKoIyOlWKD14DKN0YeNLpSyIEQSWCCQDBIr2ajkpZJPITeJFlIhRIJLD+Jp2Z7JIOKijHJMKTLIKBpxSX6NW8FRlBceTFCEYK9Eh2EkUcSL0oiHTgSQcDYCAkEUEAGC0E0ZDILqQCTzqJ0E8uIYAsUqi8AhUplK40kD7s5S0ZCYwhsHt8++23ZCC7BIxoLPRg8r740ktoXWBWFWRJCrLjyiwOWgoS0QavwCkZpTiAwY8ao0TyAslPIhApkEQgiTYmucBzgAk62ZKRXMDIU72t3DEuGBKV5btLHJyUwlspSFkedW1HuhAgZQmAwECtkCpsoXakAwwMnKFQCCAXYKQQJ5HIAQDd1VUAyEgpQgbfUXhLduAlO6UQ4akYqM8jqTs/yUU6icTJnkCu4PVNCbyluAOY9epQLxJrTkhaBcDH5vPz5PPThfjMtHsOgWh1TIu6dbL27S145YUHmzWqzQk/Lhv3tkeYAXDioNfrZmTr1IJblv+24sfvffv3udj7pLqNart6AwWTGmrJKBIBrZSAHujg6h2rA2mpzTp3adqlmyu7VlCzqcMWWbzVVCsnkBdvNqVU9FMJ7YggdTmXHZWS4bYVbls7//MPCrZvzbAyitTlAigPTB1UE/dzL5jefPET0WzukM3Vf9jpzVq3r7AyNNbQTaCjw3O8pHIUstvmzFvz2GNTuUaYk45Ag/mIEehnn37Rq39v6Z88pe/p5ajZSJWBEZl0J7ofQaQS9Y6p809VlybQJ+nqZCfCE2pJJJfASzfG4YcU+ifMpFxKB4kqUbH5gJGaCFl4SyQYUyfJgBAw6clkJ84roZzSyS3AikL8firtGyARGUFEHWWqiwOABZVE3A41USAFGJ7AgJDacfgdgeykqOy68CKdXFI73sJnfpKddAGjyrwlTuAVuRhhQg9nKrDleNOmTXx0lpGZB5x86mnY0xn+U6hy7tJrp2oRVi0NPBQNKoomUJBwD1QJzAn2Ak+AG7wFA4SRBTAAEmSTAjbeCh5Fn06YPMkYjh8wv0jRgJFIHMc3EKqIDi+YwcN34SmV5S0RiIQGLaL2VPOT6lMFUsiomIa3gq7JwAa14AG/TriiXDip9Hfl3aIMpoRU3oKHoIjELcmsXLd5JSk8QUUgBTIEIaUTF/6TIsRDBmAkQgBPj11tBwMnteAV2XkLsDEDgDlJEmgc0hmoD18a6zA3TN176x20gZHnDRszemQ0VGbjpsVQ2EUHVoYQ3OjivpKiJV9/tmPlb9nmSIrVpDqoLqEY6asGpESEkhFiKgEtkpHAnIB0ImlxG4c47/SHXA0a9jxxWJOW3LJr5fQ45X+nS3+Q5OXliQ+ywhWMON0uny+QlZG6de3v3374lj2w32linVmNeXmPEOepNiLIGMdjjviCXpSWn1OAcFZK3b2/vM+gE5oOOU25LXHfExYhizkcDaVkpvtD/lJLw5EjLyIvI3UaeWlp+fWTbpwyZUpFSG39l9av6pUI1UyBZSRIFrgBvM4V9aBDSgABP+l7PCFYlah3OX4SB4aIGiPrI3SqRrcEj+TlJ7u3pDOTi0Re8VNFWPbQpYngETKJ80qkAJCCikRSYDNCRESAUEsdFcFOJa3IBTBPgkqEVHWHg7J1SCkiR3iLUJZ0wPhJLl6RQiCFIClkFHlEilQTABIBliJSdCUHSSL+BBWEcSgztVbzw0oFxoolb/EhFjxgOEC5zkC9zAMqiiJ4SylACiclC9l5BWHURWqU+L6SAgCQkhcwgAWDSrcfaG8gFAYSITtDCuEbwNAgr3jy3cFDdvkuwBABJwpACOBJkFygwlETYFKIQxscIE4KGWX6RWKCq+qVsoSp9k9ZQiT4SaE98FO4zVsIlKIlhYpIEVIubyFS0JICBsoVvjHTIp0gPFEV1Vts4sY3ftaEkLQzAOVxT9dSR6iz7TCKxw4jGbwmvPUa+/Zs/vjVJ7LTcVbBzZI7pzSOI4iaU72av3zbqvlz3gyVlLqdFobaobjZaVPjBYI0FImrn5Uyk4aVSCSid18GIOZQLF4Rt3Q74dS2PfsH4k4/bv5xzW6x768IH9uxKwtfxTs2YM8aaAAAQABJREFUuWMhXyyaymoeknrvljkzX3RGA+kMSSI4/1cW8H+LjrM8UVVo3Xto64F9OdYrbnF6IpZUmz1sDvm5PsyU+9ob7773wWdWV1ogHO/Tp/esN1/jgjPd2lUVIiPN4IDBgRrDAX32nYy1FXnNcIABAjIauyp2WDU6CPhOGtQ7JztdqQTcJ9k4ow8oHBatvLToww/nlFaUuzy2MCMDK/cyulAg/DEUkUjiqY5d1v8YkvCX+MnIiMEtnjkmbpF0WL+f/82K5b+hRUiWAQg04MWTk1ubO2Dw0NasrkAkVlFa8smcOZzMyUICe4PKQsowJH+MLBnYMdKSP9yQqvyb/9Xnu7dsymCgyVg7zihPC3EHR4RdBtEundl5q0avubVyOHOGIRXjoGT85kadDA4YHPhzHEhaBaCG/3pg6o3sQx9wxArPYOm+QX2PVzd1mTSXU51uz+YrBHG0eM/H77weqih2O1igw16JSyIHLfg4DII/Wzx8hD+7ibfAqD+WZ1EqTEQxyTLF9Tos334yp2TvVhdTV301gst+kfNWZ4rV4YlwrLQWd5pNy5d8HyrY6YmGHFG8DP1cP++wMSnHsxPNwuZbVIwytvPHVLrKv2yP/dO3Z3LtmTWEw5E5SK3Y0WZmgSPeuGEDtcM4EsbijN+nvhr8f2Ytf67JGNAGBwwOJAsHklYBMNBG3BMwAiLl+V4vv/wyzr+WmNa+RTNuemTVESXBKJtBM7dqbfzpW9+uTXVSHBaOAELQhrh7PejmVAaLk7+o2XHIn6RXvgXGJX94p0dZx8NfyGzC5c0dDefYYwvmvMtteLh44nzGib0Wp9dkcWbk1g1GTG6rLVCyb92yn2p7rF52crFXC2usbuRU6wv/54+pgHI3qvLPaQm7IsHfF36f4dB9e7AFscEfh1ItlpHi6dCuqc1qLirI5zINNQ3SravJ0oaNehgcMDjwFzmQtApAVrrgCss7qAG2O+KAQaRbhxbZWensaMEgo/SDFrVbMJZXbFo8rw4uHr5yL/up2N+FNZ9L2PHy1Gz8cUIEfxI/8jPo96nrSaMRDrVhIYutBCnxsG/njq0b1uFgiPe5ze4y290xqzOrVj2bK5Wrx1f9toxlSofZzsE/yq1IbSrGZ8gSMVn546J0/rgKWLM6+MNqVeUf9XVZtM3Lf1e3h3ACBGc2Yg0KM+4PpHhcJw8eXFJUxjrrc8+pi2QTS16iGv9i2zGyGRwwOHCUcyBpFQBr/chE+TqMeTmKEomP82XXLl3YK8QBYSQy8FcnLUSD29evMrHFVh3eYA3EXOUR3CszyiOOQNyTavYl/tIs/sRfSjyQ+Es1BRNxHyLb6Q7ZXWws4OpVbEFoGYfTtH7V79Z4DKfDtPRszewMxyypGbWdnnRrNLTmjxU2hzducautZQ7lkK40UKVPm1RBd1jQHfLiUWtVf3ZuBIlFC3fvDpSXob5iNta/ow6T2W23lBbt69i+LZOgtFQv/oiYwsSzQjAbOuAo78IG+QYH/joHqnYp+ev4/mdyIkaV9V+/PASbOJfwiaRzezxsvMJ5D+sKZ4DZHYyrw1vWLo/YTTG7xx/D6J6iOdxYgDjtrYTrTO0H1hIOrpmaOcQV6w5IT90TVOIhp8eHu5vFjAnIFA25HJyCwnlCkdKdOzgngp3BaRmZnPmjHLVt8Yyc3H27N2wv8NXJzA3HI+okCo4FdaorXtWhDwcFSpRf0VjVl9HHTRE8U90WR3FJUXqdOn4uCWDVWVM+fy5HSp3amUNPGvDV94vRixytNeutmXaOrzCCwQGDAzWbA0mrABhBI/vwhyGCjzlbwNRPTsR0uTlfVjaUM6zmfDcWZ0v35bOpymJPWb0tP2DyVwQZgtswwLNOsNmi3JAPCbzhyJxDxs7iDppn34uHeYrHzcYudpikuqxu9uaypBvMx++oIhBMS8sIkNVqr/CX5TVpvvi7+QtXF6R6A17OebCGU12s22KcMqVb2birPJcR2VSBJ3FSMhDqVQXcfThNgjpy1k0KG4HZE8OmBQ4MsnC8XcDkdF199VXfLPgxJyN7/rx56v6p629OoAFtQsEkEo2IwQGDA0nPgaRVAEg0Xaip5V9mA7JDFYmPf40W447HWBgLu5mNpjG1qSocc8YDEXPmrnJTwO4OxW1h5LfZGnM7vaFIESuxuAVx4IMtUmrGduT2+u3eVDeDfH+kImrRDx7BazOEl451fThay+mu2K/MNAE2G2KF4oAALc3is44IZ+bWb+T1ZptCftzwuYE4ELfUbtqsIGYKOW0FcXuo3EESN9axROGIZbKzOBoPo0W4mYyVYYvXFTJpWRFcliIup4ljRFmzcKIq0DA2zWVNM+N9ZGNbqZ19ACw2R+zOwhSrqzzodHHiwc5Mh/uRO8ffeNtjqd60Gc+8gAIQFqFdYBHLwigYnvLz8HbPW7hHOk9g0BkA81Ou7uMtugctxeoC3AaGt6K6JAs/sTuxEYydwKTwkywAkIufStXpZw1BEhh4S4QJHHjABqQgEeQAE9jsJkggBiQE8Cva9CMlgBfkpJCdUN0Wf72RAK6CQlsZZH8fpYu5jLLkbXU7pSvzGf8bHDiaOJC0CkC6K0c9Iho4N59twHwWBEFpaRkpiBIO1fc6vLFggFPYMnJqRQL7kOFpHntFIB7hXDblHhpjm7+tRZfsnDpRmztVY5t4KM2DI47NG3Ke1rXuzBnPpQR8HH0QtTpLnCmNevcptjriCPeQOVQWDFUUlhSuN4VKc21uRHpBNBIwWzxZ2RVhVmiVpz4zD0ROToN6uPjjJBrlLkMwcwZ1mOvFo5rLFLOa2A9gNznSNA9+nchVp8dRbMvkXt5wqZ8r7LlpgJNRQoU+BHLU50v1OivKQm37W9pgRPKVOWLcn2VxuV3ceMO2BI5I6Nr9uJxcCDAVlZbs3LmzYcOGyDXYIlISjgnTqmy/yO7EkQywUXZykt2mb1QWPGSEtyK7RXCTDjBPxDTup6hhp+3AjlzKEmAplCeQ5KUgvg7wREgkIkh4C51oBY5eUHtHY0rf8EoJfT0ALy6/QIKZQHbRIoAlzrqpsnYkChmJt2QEoUh/iiBdSq/utNRERiNicOAo4kCSKwAECh2bwPFnSAk+TFl5OX76GEjo0JyXYLfaA+Fgbt36u7euskRD6R77lpJQ0JUexk/TEvaX5dcddIarfbugO93DvICheCqeNVpm3Lvwy8d3lxQ0MnEFjLnIH9Zymx0z/KLCnDpebuUyOaK+sCNcbC9cu/CNp4I7NnKrt9NqzkjNQA+ZberUCc6fQ76zDaB2w4ZqN5oWS7VbfL6KaJxTHxgmm4Jhn+ay4MzD0JUZgcfuJhLWGO1yCgvHi3rYyhCKRLkkhft5WXPQauVxSVmZaZu7fpOyMOe0sDkNhyK2gqltDXZWpM0sPMSPO67n1wsXc8EAR43m5eUh1GQIDItEwlbXdhHKCH0kL/JXJDKQJPLkFAxsYpw8xI5rpVlxndLP/rXbKFodlYHtiyVpB/uPNXXIM1KV4sgIHoQ4BBA4nFESwSBzCJHdvAJMvh0yHSJJV9pF3+lNFt6CinSZAXDRiqgESSSLjv7PDdxBi7qiIJBIAIkQpg79MILBgWThQNIqAHqskmi6SEJeNGnSBEmhZIp+qmU0yAlx+lG3Lm5lsrbt2HXT93Pp77npHuueYnz1Y5zPjNkgYtLC9qgjvdSRzqqvIxryaxy7zGkMbk+7fqZjt+zaunl/ud/vdjfp0seXXi/flJrO4T52t8mdZffZ2ua60hvm7dy21hoPdmrfMs3rys3KKg+G2AfAqRBc/sVus4zUnNbNm+8v3FMeCwfj0UBI2Sw4QNQWs0YDap+yH6OTmeOlo3aPLWYLecvLgwFTABMI5/Fymk7Q5OEim7gt4srkWCMkbeO8ulEtjPmKo9Y5U5L9Z8oYEguze8Dl9oSCHBmKYlCHhcEimjFsgVEHpORBNpBDWjgA4EkcpQKShPIAEgENHnDKqJkIgSw8AVMfQh+n8yGYRiSEshzRzltVln6qqOQCG1+KdA7MoFC0jozuQQgeIQwkvBLigQSzlMVpoBB5AKeuGBRu9ETi7A7Jf9CTt4lfibhoKbQdEWGOVOfAWnwigxExOHA0cyBpFYDIHfoz/RZhwQ1HfCZEyYpVazTzcEQMzjlcj8VNp2abM6NWvYy6DYr3F9XOSMtwFgdw2ERwms1ul6lk2/rGfbr6wwGnJQ2DBJcC27h7PRCx5HQdNL7F6sXfxcqKmtRv1LB99zKLk5PgvLZwicnn16KZDnPA5y/cU5iSnl2yb3e3/sfXatAoEGPM7vIHwg6rU1nDI7GI1X7WWec89MD9mVnpvkhp9zPOSW3Vdlt+qTvI7ZWhsrJia7Ai166hV5Yu/sbrsPqsLnNGCjeSBWOYgjDyhMIVQYct5o3vdVijPXq0yuL6MqQdrq4hBq0eTDQ+f8Tp4rBrFpFtS5f9hp0eHyOORRJhKqKZ5wEBp2uFw5u0CEHAREQSIajxvkmNwVFRgaC6WMZi5c5MpVm5dSsUUHcAAAM2TDX8IZ3JBTwpiG/EekJzINopAmmLzCWRwL4Nzs0GGEgEOmIdVCLc1WuLBUheEZD+pJOmJgfhylPJKu8IpCxexfTT9A6vV0Li8+rgOGghDxUFGcRFo0AhQIcjMVIMDhylHEhaBcD3ULKj0gCd4vFy+92GDRuW/voHr7BEc6WcOoiBW3/VMbOxbieeNvv1F2qbQ7meWGmgJGjCcB5OSfNu/umrtucPTXEi1ALYX7i0kf24rBaUmTNsael5Q8+Lm4L4d5Zb3bFgNMsejVi4Gt6UYjN7g35TQUFgw5Z0ZxQXzT79B3nTatldaSHMF2zRxdZvDikLftR+xvCzUADKJBS3WdPr1Op5ktnKuq0DR9NGWrhWOPjrG69sXfq9Pe5loBuq32PYJZeGs2vv8VU4LVpKcX5qwZ4Pn5uO+Z+V00F9e3NdJTKWw6StXJMdggnqoivmPSi64tKK/MJCd2pqvUYNUQDSZBFtItyVdKs+IFsRsohCeZILcxZnmu7dsWv9+vUccM2Fgupud8Q7Z2uzyzocwfITDAT37Ny1detWsnOToivdweyDa564sxcMCFx0M2WquMXCRVE7duwgkUuyQIgy2Lhxo7rdNx7nUkMhjbqASgllLrxPSYEYMKAA2OPNdAEYgMGwcuVK1AOvevbsSbqq2j+T2wdLf/CsXr2a9gN+mg1KS/SBzskjcUmINJ4GB44WDiTtRjDpzyLUGAAiC7iyFcGByC8sLGRJjzEkb7FksDpqdbizW7Trc+LJJcX5HZrX8sYrLOFizOYcmebavqlo8Y+1I+H0uMmjWbwsB4fMLq5oN/mipmCFFquIWbDp+LkU2OUIsUigsXgYx2UoPer/44uPUuKh6P6CiWNGt27VyutJC1SoQSt26X379hcW7C8pLma83KZ1C66r9QV9DJAL8ovKI/biiHO/FX9S6864c5c1NbfXiaW5LYo9tfemNWzU7zStWaf83MbmdsdF8trU79pH42AJlgficW+Gu2ef3kGWEJwsbAQ40JiKcyoqjjrhcAT30wULFiFsOZn9uJ7dEa+IXQL0IOmIHLnJwkDAEIVAIluRwt98882IESO6d+s28sILTxw8uG+fPt98/TUDc8byXO3BBQQb1q8/+6yzOnXseO4555x26qnc8f3pJ59QLmqDJRmKE8GKVCUOQhTJoEGDBvbvP3/+fMD4OtzwfsLAgYNPOAEVAp2AUTqJr776KlVD7kMJKRxA/8orr7z33nvYl/gJqRdeeOEJAwYMGTzY63GD6shVO/wtjYf7eHv37DmwXz8Kgk5goIdwOPC/N4Wiqwz/3lIMbAYHhAP/pNsfvWyir4poowpqKmCzIWTpWnTljRs2IyZYnET0sA2AbWHqVi+rq1OPHl27dDCFylrlZdni3KakfP0z05zzPvugqKgg7nH4HdYKDg21m0sYYNvLTfEyVyyQzliUnV6a2ReN+jizzZrO2DTN4927ce3GxfNNvv3tWjQYftrAzNzM1WtXvfzyi+OvuvqUwUM7tejQs0334f1Ovevmm6Ph4KSbJ1VwL0HEv2nVSif7wcLmjFAsE++iqLUiZnU0a9t97IR2N97e69Z72g8aUGSKB6y2Es570yy+8orflv1g0sqY7FwwaqQj1RW1amXBMleqBychDpOj7sIKKvvMM8/m5uSG/P5LL70UKamYoB9ej4AToXMEAccrEYU8MYxwq8zw4cMXL1xIvHv37gzAkdFcKb5q1SoKIrDxAo27dOlSMPfq1YvpAnvxKHf27NnciIImJh3RDBkAQwBBSWqsQ/qqL8Uh1nnLT74gnw8AiuZC8Esvuoi5AinAMAngkI9TTjnl2vHjiYgm43Jarn6TpguPsOQAfOSWDDEJAIkLKzjaH5yUC6+oaQLGiBgcSA4OJK0CwNESiwSB3osLIPf9HtOqBedqprgsazdvLdc8QbObCxTxvXHa1J2QuFmWO9K6nD6yfZ+Bma5Yhzo2r7/IqZldJqt7x/ZNrzzp3L3M6S4LW1j8dETjrqgpDZmgbhkIlridEezvSFO7zVMcd+Y47O4Ny1a+8WS2VpziNF0z6bq27To99tpzvY7rtf6T9afZhj3b/6Gfbv366zu/nHb506vnbLrzzqkjLj7/7LOGqY+xZ4e2Zb3msFSYHExMcNcxO+JFlmCkTo6nbftQvaYlDmc0jiXd7opbudUonF+wg/29JlOTuq5Bx3fmJoM4FbOmxyMcbIGOYz07wjF23pzmD0+fVa5Z/NFwn959BvToh4DDvoF044kI5ifhYDl4aPtGBuoenwhij8v9xCOPxUMskZtenvnql9/Mfe+9dzhVCYvW9ddfF4lHLS7bCy89X1Zewq0bz86Y/u7sdz765MNatXNi8eidN9+qBcPq5jJuY9DnE75gwOqwhzgnj4kI6yK61xCvMN1cetHFPyxauGDedw3q1Ma5CvfXBd99a4MSLWaKRnCE9TjsO7ZsXrJoIRsTUrl0Rc3s4pFY2FdeykI3Djs4e3HJc9Qci4SCLOuAXa2KswXDzlfnzD0l3AmiTpS+0eclqCKb+hhYsiJE8KhCSYPCy5c2RS1OawBfLBa2OZSD1WVO3bBgylNrDy4rqy6OdLc3xelW+XEIUxsN1cdw0ziYVLIIZDZxRisp/MSVixTi0WCARF65dQzquhL90lr4wBglzFDECAYH/gMc+Ccjo/9Aif8llPRqhn4UhqEAAUffZgaQk5MTKSn4+aeloy46Tx2urJnwSmRkh+Bz2CwRf1Rz2rr3HZRRp/GCJb8H/li3aUfxPmfYanaXLf9j6YwXjht+Xq3mbQrNEYzN7qiXqxyDURwyM7iDK+wPpLoc0bKKJi6Tc9eu7155Prh7e6pZu+6WG7scd3xaavb0a1/67JGvO2V0TC1Jsfvt2JD8Zr81y3bDxKtOu+nM+6be/OyT0wP+MV9+uWDei9P733H3jtw6vnA5Po8xM98IA41VQ+aYNF/cHEj1aH6f0xTPcwR//uJDTrBDhFw7YWxWZmrEX4G6QxBTX2Y2qL9wzO72Zq5cte7NmW9n1aqfX5B/1913I2r/7GeARaIhYCw64/kZM2RUPmTIEH+Fb+DAgSMvHDlr1qyffvqJqUCr1sd+/vnniFVyMVFAinXr1m3MmDF333UX99Fv3ry5Xv36sD0YVndg4c4EtaDlCSRokb9K8MXjzDO2bd3MF2zcuDED/K+++opEwrJly955550WLVpgOFqwYAHrDqiNJUuWMPbPO7ZFt85dsAVRQZBAAJVFhjJXYEUB49L27duR1PXr1x88eHBeXh6OqjJFEALIBTyUqOkIazkp6lgOZi3MbEDFOscJw05Gdbn1u39/W/bbmjVrwAYT0tMzKQ6f02+/+QYbI3U/7awzIIuFCKYjrEl89tlnVCE3N/fUU0/l+cknn4B5wIABHNBNiayLYMLatm3bvvxCiM/Kzu7ao3uffn3Z142WYnUdfzbgjWBw4N/LgaRVAHQqeqaILViGzKKP0VHffvnlTVt2cDOiR93CGy0tLaWTY31hmOZ24esfi5hN9Zu2Gd6gWftOW7+dN3/2ypWhklCu1xpa9dvXa9bl9e17zMB+ttyMaGpjTo6zq9u3nPhfWjnGzYbbZpFj6dIv33zVtG9b4wzX+KuuPPGkIXUbN9Xs3nrWujs37Di2dQsPp0Rbos4UzvXUdu3f9uDzd98wfpzD6Q66I6+88Pz40WPe+eyr+Q/c1GnCnem5tYLxQNTlKY/Gg3GnzZnGbfBWJyeNBmo5bala/savv9701TtY9ccPG9ahbSuWfwMsJLidVpzV2WegzqvAwJJtMnuuGD0xK7dufn7BZVeOPv7448sris12dXnvvx6QwvCTSRVijkVmJlZISQQZiSDBa75Zs2aIZsQ3y6fHtmlNCumsz9aqVYuFXLKjfZmS8SEWLVo0+sorKwJ+EJEdSQ8kEUSwyqIvNoCHLMj0W2+5iY+4fPnyFStW3HjjjQCgOb4kzP1y6uOPg+2B++/n3h6uT/5QD/945MG+vXojMdENYABAqROzCT2B5FUmJrx9MB9Fo960NKRwuw4dsTUBRrkUSlmigUjB4Ye1CvTWW7NmkUtltFg6dOnMIoTFyV2/loXzv7///vupNcizsnJY93Z7UjhmY+HChVTmpFOG2pwOSn/77bcvu+wybFloBb7IP+65Z+K110574gnwoyYbNWoEBoxj77//vrJ3oRA5QBDtZYpPvufuG264wc+6UqXzq5BhPA0O/Ls4kLQKQHoyIoDuxJN+yEBv2LBhM19+mWsSt+3Y1aF1E4tLrWciqtLS0k0xjmfA4KIsJ3Q3l9vbpWObdi0a9d1bvH75uqWLlq1ZtdVkde6Z9+HWn+emN2lsb9QqrW5DT+3G7qzaGGriZfu2bV5dvHNT4Zcfc5z0kMF9zjn11O5du9apnxczu1gofvqrp8aMHPPQaw8M6NKvXm698kDpzyuX7ijYduHl59109237isstdqc33fPMc9P8l178zbwf5199ccvje9Vu3yW9VTtHTh0tLTuglXs43U6z1LaYIxUb13756ZY3XjXHw6POHnzJJedXBIvYM5zqdYdY6KbONifn1cEEpzd39FVjU9Jz9heVtmzdevLkyaGoGnf//zbvf60poVCRjNGIWjlgsRonol07d2LWx5rPBAscsBHphRhl0Eq5uPFs3rQJicmQH+aTi0TWh3H3ZyAsZZJChCcBkhBzEgGYdIpjfka6BEo866yzkJKYhjp27NipS2dRPyeedNLXX3/NN+3cuXPz5s0ZpPOhCfLpUQMVwQC3vJ06dCg4O3XuPGHCBAh+4IEHVq9aNWjAgB179lJuwghG0WQkQAnimP3Sb7311kMPPwxa1pk/nDPn96W/PHDvlKlTp0KV1+kyca8b3qtWtSAhdWH8Thx2oYR4x4r3ZaNGIfqh5J577mEaCp5nnnkGZ1x4RSkUjRr44IMPiDz22GMtmrUk8YUXX5w9+71/3HXXueeeW6tObbQR2+jAbwSDA/9eDiRtq6IX0Y3ph0SkhxNnus3gOByNr1y9pl3rptGQGp8iUPx+Hz71XAPJSf02O1vALP5QIFThd3Gdlictt0v7Y5o0/H3t+oLi8i1bd27fmV/00y73sp9KYuqSAM70ZxtrLFzmMYewOw/s3LTfoIG9+vapW79B47wW4RgbzTiTTWs5sNPsXz5b8+uq9X+sefONmS3btrx82rh2ndt1qHfs7kCpzcpep1jQFvGkpzz/4nN3/eOOb+b+vOXHzzYunhuzubOatspu1TaSlpPVKC8jbtuxe9O6Hxf41/2CIfyqK888/+wzw2E/h/4HfBVYt5EjeOswpEafYUx5Zea7q9Zt9mSk4xf64ssvpaWnO53WMC6qfzIgxOEkog3lgiy7+OKLH3n4YX5++umnZw4/g1E/Qs1hd3BSAlIbVp955pk///wzMwBENoKbjcczZsyAsYxseUtAqLk8bp7ydUCOQEyIUeLIYgqCTF4BhhFp+vTpaphsMrHqe8fkO5HyEMPE4qu5cxmPswTNFGG/v5xEsgMmCMGDvQirOz+nPvoYjqG8atGseY8ePVgkQfLiMoQaA4yMtBkUDwD8hFrowePojDPOILFfv37Hr1+/ZtXq55+bceftd+Cryu4HfG55hb2evOx2JoCKFMoi2F1OlBPYGPi/9MYbTEFIZAaGqYebeagYuSgCmxjaj9kKy+lNmzRjRtKhY8ehp57SrEVz1EmiOuAxgsGBfy8HklYBIGLoWgkJQs8kRW0Hi1szs1LefufdC88dDsT/x959gNdVHI3Dl3Slq6tquReMKwaM6b1jbDqhhISaAqEkpBEIKUBoAZIQkpfeQkklCZAECKkQeg3NhQ4GG2yKsS1bttq9V+377Vlbf38g+cUgY8Or8+g52rtndnZ2dnZ2d3Z2NyiRxKzRUZQuKy12uIJj4hwfGrYRZ8o5UNZkivv26TNs5FqbbLkJPb5kwZJcfa6xruHxZ6Y9PmXKwGFrO2VGRjWVleuvM3b8uDGVA/uNHj22f7+B5TV9G5zckOw4q0iV5pvTg/sMGbTToJ223vGtuXMzZaU777irS4OXtDljp6LViQ2lJcbmje2FfYYOPeZr35q829RFC+sffeiJma/OeWrKEzNffII2fy2f43/EikHF7Ttxl88esOuYTdbtqKttLTcUbSkty+SzimOV0+FDHWVVNTNmvX7Vddf17z9gwcIFN//1to0236R+SZ3jhZiulm6ofd/ShHv0mm4AJ9nNTjzxxJtvuom5/5gjj7xyqyumPP5EdWVVU3MTL309EDV35JFHMp3/99FHv/61r1177bUvvfRSc1OT7mdJXb1KwfakowoGmfjTO+jQ5KGFsV9QXgLxDScYnYF4kPSseAEPTW3jW+xL0CmJGondFWAmPsYhyUE6/mjWrFlKoeOP+Clo9hnwsHmLRJvBO/JE+WkNAyq9mgUG4/Fzf3iOLCwn8GWKxETOhElDYgGDRF5Ikh0kesHQ7SXmfvFmnJYEjjvuuNABLJv0WOEA4NE5bbv1dhMmTNhn332Fhw1fCzzC4tfedy8HepwDn9gOQPOjCDzap7BHa6S5CjiYFxY1NLctWlxXXcZaw0ukgL9kC98aGo4J1t5WTh/FqYJUxvkJTvDnXZJtbnCksx20A/tXt9rvO3L4qK03mPyp3V9+4eV11ll30JChTghd3NQ8eK21hg0eXp6pyOZal+Q7qvsNbGhqrKmqfGPW6088/KgF0ulTpzz62MOZdIZyufznP9tp+1023G7zA/c9YMLmm6naDKt9SUVze9u4MRs2183t33fAphusT28uWVRnbbC2dhH6Zy+ZM2bk2A1HbrDOiOGUTseiBc1lqdKWLBsIC0qmvCLvAFLLvyWZXHvRzy+6om//vgsW137+2KO232kbB9RVVFa2hoN6qKSg7N7/gzeJpnOmUHDfzDY23XHHHT/60Y/uvO+uN998k3fmhPEbnHvuubSwtRaQuP373//+ggsusHLLCsQTlI//qaecImufDHIdSJdK3Iqi2pUEMVGlqjVhVRYWPxP7jJLS/nFdATwwMD7FVJgpJmpeYFHXd+pNtwCx5EAoxroxyKQgSyXfTUH6EnnR+8ooEGUmUhX7MwCSQBsWbNvbitMls9+Ys81225ruOJIIEc1cjMJhGOEKB0giecJQvfjii8gbNHgwpkGi7CLD2nIy/JcK2j322OPbJ58czErt7YTEjT3XXXddWVXll445+rzzzgueykrH1tT79HKgpznwie0AtEMNnj7SeDAtNKGOjksuuYRdoKMww9uwNJ0JC5BtOeNW64rp8kr3tzhVh8cg5/GCVDrvXDYnPpdzHWwtcV1YY84JB2HtNF2a7cim2kv6Zip23HrrKVOerp23aO6iJVvtMHHMiPHOisi6xDHDONCWy7bWzl149U2XX3nNlXVz3mwv7DCy7Dcg6MdSY/DW0meffvKBR+69/MKLv33iySed8M2KAf0NX9uy7dmmwoZ5tTQpJ0G7aiuqKwYM61ffVF9RUWatwkEP7odckl1QUlJW5IKbcIBQc0Gqkll5fu3CpqZsEf/RVNmTU59/9sXZ5f36lpQW//jH51jaEMjnstZLOygT5o+VeaJCjPy0YaGqsrJPdfXFF1/8du07zCAjhq11w29/F2FY/yEW5mlDqdkQQH0bcT/yyCMxQwNe2pYeBKOOOlrD0nHUvGJifYHEJbpbQHJfZe2RShKfYlph8HoDNIgUzFvPT4W+KuYFTJcgVTigoq31ml9cA5VIb4Nr9d7ihubSUhmhwSMhYHh8jTTE7EAuxWl00NJiW4MiAJMrsGDud654RQXM9ih46y5QW15YGY7/Kyqa98474FEiC4HQkyVlVzSPPoCit6uOp9CD9z80bdq0RXV1Ov4rL7/cqsZRxxwNJnSJvU8vB3qaA2ucXCXqqT2TLg0nGjCtss+0cn/0zxQ7TPk7ClP2XeWLSlpTZdnCjBPzbXRNF7SX+dqa53Yd9r7aDuo4HCb55ABhKj1dmJr2+JMX/fRnw/pVL15Qe+2v/rj9Xge3lw9qai+szBSnW+pam+a3tzWGozedwEZTctPuyPdpb0w3Lkzn7blyRnNBqqJPcqywYVv77LkL2jOVJTVDps9884Krfr33gUdsv+Pujj8IFOZy/ctK3Stz1aUXbrbx+HPOPL127jx7ynh0NzW3zV9Qv3hJbklja3NroZthqvsNGDBw4P9c9PP9PvPp2bNm0nMFrfm+A2sGr7vNmRdcPeedxlRxZZHLA/ItnI2K8o3F2eJMvjTt1Byrgk4zpYI6CvkEZZvyTz4xpb5+MXXnEKG29sKLL72mX//BdUvqr7rqF32r+3Mkam1szfAiokAdFLqSj1NJOe+rHTVCY9517z2nnXnGV7/5jXRB8Yhha/MPuvf+BxysbdS7yw47uwnzycef+s63v3vUF7/U3JgdPXJMn6qahx542PzK3GPy3nvWZ5ssQ1Og2GXHBtakLQ+HpYvgaUOBhj0cyWU+QOBkBUmWUelZH6Vy2mm6lIm9JNOSay0MBrviyspqX92bXFacqSivcieyhHL0c5dtdmS+U7Gf2v+AQw47/HNf+OLk3fcYPHTYZltsudtuuxng0++ypohhxx/oykrLUy5ua+146YWXK8urcs15eT326OPBt7+jaKMJG9tjrR91vJKfb785t6KoxESA3p86dapCwZb2MdvC7g8jtk2fPj1OaPQBFpbRHJMrzbwFC6dOf+aNOW+ddebZ7uy85557bvnLXxTfaYQ3/PLX5c6PdYRfvkVDcLxSbA6mo7FprGQ19oL3cuD/x4E1bgagESIwzsopiPhTs3T+AiOMgBPUygrbS1MtdudoRE1tGS3K+qfBFcurMx9p/nAtu/a+zFhMYRl28ajj1ff2vPmfPfSQvfbdpymbH7/Rps8/1ZBrrHV1bmkmLEhSN2nu/K2tDY1h1h8Gg05SLi2h+bj0BQ3e1NpcX19T03f9CZvOffudU0//9vwFdXfd+8CgwWtppJSajORrtHv++eczN++///4yNZbvW1WNzmig4Dzjk7ccjfUYlCWZOmUKZcRRffjw4UaRo8ete8bZ5+612x4333DNhPXHzps7p6i4qjmfr3TlSzhZLRxuEza4ZZutE7LF1+eK199w44yrxwoKBpZW3P6Pe3STzfnc+AkTJk+ebIRLx+GesSc1BGZln1gXUT8a6tbX119y4YWQDOzbjw2dIYWzI8X3/e9/XxmByfHyyy8HIN8TTjiBEYY5yM+jjj2WAcSyu3DEhskCWBH1pjG1vDofTAvj38SkTjWDUXxeoVaV2eXHjh0LQHKPlVVHCTHaiOk0nYvHzMMOO+zGP/5R2jPOOOO0005jh7nqqqvYr8RcevWVFoGxJY7xY14qEYYoAN/4xjcAW2rmesqBB9lbcu4aMgSFajb2FpamGe4x9pe//CWA+MhaoSZNmuSnQn39618/5ZRTrBwY5t97770Eiqj6hJlnn332LX/6k19//8c/Nt9sMw615B9mBWE6gwfMdb/+1e233w7hz3/+8zFjxogkpehfuXncUtJ6//VyYCkHCufV1a1RzDBaJ/dB8yammyjowh1pJ97wIGlZOO/t+roFjzxw3ysvPmcQtM3un950i81p2LKKcqdv0oDhfMqWHJ2heRhaegO78rLLf3DaaZpxUWnhpZddMWgoc+6w/pXlBfmG116Y1p5rbGkJhmCKgGpjs+Zi6HiDQElxqqk5V1Hd540333l99tz1N5iwzrj1Dbqd6LnPfvsvqF344MMPrzee5aeDBllc31iZKdVK6TuN1nIfyiHRAzkZLT5RL9AvsW8ToHT0B//617+4pUulDwDZVmJbaercM8+45MKLn/rvg+utN3bhotp35s9tb5hPO0gbWURN0Ho0UWnfYe1tuarKCvq3oTE7eY+Dq/v1r11Y//c7/s5/BjzvTMitZMqOalPYlar3eOon1Ylg5CkRPThzxgz2FyNchaPOMdXZD/SgMoLceeedn54ejt6LDwuVvuqhxx4dPXq0xQx0QgKbggCwh5bX/z777CMth6LPf/7zAtwi7R3z9bHHH2e+B8ms1FBfHxF+7/vfP+ecc+j98ePHh25DR9Le/pPzzz/ppJO22GKL5559lvOlTkKFKviXv/zl399wg4TmW2pq9uuv+7rrrrvedMuf8ROAWkMzfoJB27FHfskq9/rjx9vV5XGS3bPPPOOTXKwbY6mwTsgpdSxCwmPXWUefB5KTz8MPPSRm7jvvqPqKyrJLL730+9/7XkyLQnw46qijfnXtr5jy7r7nnq2329Y2Mak6uNimUpiGEssAmKxQzz3/PEGF52cX/s85Z5+Nz/fdf7/z6dQgMAxcwTHXcux9ejmwYg6kvhvW5daghxMJbUW+NQBk0QLaJJ3iMpEXnn36+muvOf30M37284v+fcfdj09/4fFpz/3lppuvuOyyplxuww03qa6uCm55Js9GyNq3ZYDEZNDU2LTfPvsyQ2ufF1560UEHH2w+/cKLL73yyqvm3c7IfP2NubNnvzZ/YV1TtiVdVjFy9Dre7tMtLLZpyy3CVW/PX7zOehvuts/+o9abkK7q13/I8L/e/u9bbrnthG+f/MUjv6SHCM5DwQ+HnSZ4y7D86j9QHnqfxNBR39DgYpa84zxb8pQI072xqjEg151+A/qvtfbwSbtNPvxzRwwcPOiBhx7ccaed6HiznY033PSyyy5/e27t4Z8/srSMHWXwgKFD+g5eu6zPwNKqft4themR4yYMXntsVb+BI0auY2PasOEjr7jquv8+MS3X3rH/gZ/+9rdPpPGpPFQZC+vh0BY4s5K+JdxspKKCo3JXKMr6nfnz57zxRj6XGzlmzDHHHXvxJZf07deP6U6vaTLElWXBwtq35r6dbW5ae/SoQw499Jprrx09ZgzNFfHEWo4zBuF58+e//MqM4WuvvcNOO45bb10Ynnn22boli0eOHrXvfp+CGd94Rk6ZNm1x3aJUaXrybrttudVWldVVuPfs888vXrSI0eWAT396gw0nPPHUk4OHDu3bv98XvvhFDMfnSZMn1/Tru7CuTl+7uK5u3Q3GH37EEeeed55V8aBqk7WESJiwnmD600+bYKmIC372s3fmz3vwgftLyjK77bG73We7TJyorq0Aq+Pttt/+pRkz5vP2X7Bgm+22u/qaXwwZNrRuyZK1R4447PDDYWhtyfMf3WDCBPKgFvbcay8LJ/Y03HjjTcT78MMPHz5ibeMSvka2QSNs6tQpc956S3ey1957X3LZZRJaYSYq3KiyudzAQYP4pJpGqAU8DPOV1BpnxV2DtEkvKf8bB9a4GQANTrKpA6Mb8k07aJC2kv7qmivO+9FP2T0qqvulePK4EKU442DL8o6Gxsbmpsbcnnvvd/3117NCWyY0DcgFR5cCFz/Cc+0vrjn5xJOqq6r0Afc//nCpvbX1QQ9ymnS8TNj65fDnpgVGgvRjHCBLxVLh8XP263PW3WCj4kyZU3VYhDjZlBSXnnnKaZddftl/7rpz2223beRWb4m1pMT16+z1Ub1SK1ppp3KJ830/PbFSfBVQUt1bZ6Y6DBYhGqHYunRxSWVp+UnfPAH9t976V5uYqvvWFBaHNUMPSKnkBUMY5y6p420+eNBAKwzrr79Rv4H9ahfU/fPue3bYbhujbBlFeAkNw1dW+0uuXtSFvJANFRsOdcZUUt8o8wYqCdqo1qMy9QamaI7gVmT85DgvuTA9C1UsuyTAQrxLMhNHT18jvACYdCYcGCdTMCBN9TgdyVHYLA0eYJhgSuHcCHU6aMBAlACQFnJp0QmmpDQtHoyZELINq436ofVgY5y1RH6iPCZEs0xVq+SWsiFUBDkiDwYw4uVohoEk9FDrksTihxwTh5+ZL79k45sKxSLJJRTv1Ifjv/xVPeV/n3icwSfIhvuG9NCmbBZzErSAoUKSfAPTsu6FCxxAAJp9ilWvzwPQ+/Ry4INxYI3rACwzarTkO9GnOQ2MwdQGn5eff65f3z4aQGvOLSju53X9d3DsS6WdesCDPlW7qPGIL3zh6quvcv5jS3tLNvEtKUuX6jxGDV+7T3UfwytbLrfeZUf40yUZ6h7LmIwgoX3cvyugdWlyUbX5CmZR7SJN0bk6nEdpfysHyRJkyc9+fP4FP/7x2/PnaZysIBLSs9okc1OiVZYqLGoCWm9T9ZBR8FgJAGgQDsVJ4hVWjJ8+QcWYkKmodGVlprjkmWlPT9x5l0suvXzQ0CEs/pX9aniG6KioA6Sa07DqoLB/daZPVXVZSfr4rxxnITGba9ln//1+/8c/cAqKkiF3ak4uArKLmuX9C40OQI5RwSFSKWJarpx++uQd6YdZ0bBCRlFnyS6WXe7AYtkRI9IDJsAXhVOXPTB7OgMG2r6CgVBAWtggAeDtifGR+VhnmVRykZJEClVi6MgTPOj3lYaNlKBBGCrwYnxSBKliLt7C8ZOsY77QRkjF7MQmod6Fco/wEHok8X7s4Yc+te++VLg6uvvuuyER/7Wvfe2uO+8GvLixwWRCb4pCfQA8zkeCFippZR3pUUxrv8RSWSKTpQUg3iRDuPfp5cAH48Aa1wEQdCJOuDUGje2GG26weEv0q8vLOL0UtjT2SRcMrSnrk+4Y1KfM9t2pc1sX1jc3tZW0l1TOr2t46qknR48emRzAmKjUgkILd9//znf79+03auRI/hUMI9pVzg1f4bSvZISVChmlU2F4q3HGfDVUqoHNpKa6L0j0hLGb6UhHUNP8i154+umJO+00r7YWNleYZ5vDOJRScLAlPLGVhp+JCQhOriqhGS/rA6J2AOmTcKy8gHmZ2qIyiq3yFhYtrl04atSo391ww2ZbbmEY29CUZXrWL0oirdVIqWiQhiV1urF33nxr//32HdB/gOsGHnjo/o023YjnKjD5Io/mUhbAUZetlMToANDpgQdnqE6dUOj23GmWCuY2yAWUUQB+Zfc1stTXpb0jb87mcMlizBoeSZbyJAxzg6KXFsG47R1qQcZJJwFbzCKC+QoJDOKjLvbVU+rGtnhjTNIHoBNO8R0sg0m/BZsAJPFtjggJbGBifxCZE054SFYyYhHAywUY5MoCRjxUYgSUwlsxwXhHGJ9CLtkss76rDsSD2Xjjja1gkwO+Q+ece+43vnWCUHzUNRqcSqhcIKX1M2YU8nXJZxIZ32CkCji7v+oyMrn33cuBFXBgjVsDsGU/NjzSz2vigp/+lPGXykgzcDTXD6kuHT+sZkzfotHVBX1aageVZNODLL415pzsm0o3NGfHjFtni802DYeQaTQ286ZLv/CFL0C4eFHdj847z1IeDUyfGCm3tbfmck0aKQ8fByfnbPoy+Qi+guzXPCnZ9Nuq+tRwBDRSN/zVqvMtnOgNuejl9iFDB+ayTZtturFBWntre/A7iioy2X+GeI2zU9dozJIhY/mYWCtWBTRyYXqaSSiGcw5qbs1XVpTneYO0tV568cVf/dqX1x2/LsfY9lzzkIH9B/XvO3hAX+9MCf/YbDpVWF5Zxcrw55tvfuihh/nLbr3NVsPXHrpkyeKqPv30FlHHxdy9Q3bLep0VCMfynxQz/lQ0rIjaMCBJykUrxW4glDTRrWBkJF6qqOjF++kcIW+fEsDQOcUAgvjp60kcKITbTNuA/IkH3MlMPIyPhGiQFn7igSrxArJkK3cRprTpUrXcZkGJJ5d3pBxYVNMSCru7Bv7YdcVSRILVBWwo91WMd+weYln8jFkjrDMgOap8AqMeI4zejIO/eHmxBem8R48eTQ7POuuHxxxzjPG+SUcsgtvTTCOYI9EpJvBqWU/pp/mBjOCMXSaEAoEDy6ZiAab36eXASnJgtc0AiDIhRm1n2/NTy7GHtYRZs6PlqCO/wDGmvKKK4YVSHlw3b/Sw9Lih1dU84VsawsS3qLC5paOioPC5ppK7X893lA1qybbzDJ/x+qwmXvOZktbm5ilPPr73Hnsyv3Ixn/7ci6niMLqkh71Day9OjDB2DwSrS4NpuHZrfOfR2IQjhV2yVHLzcaUwOwl6J3li4+wSfmUj4acyKB3EcPtjZZYDqkSiStY+0RRiYiQ1ZtRsvZT5gxPjX//2t10m7Wrg+cpLr7CbW6iQygMmEinVypLUC/8BOIDbYZze1mapwEKFgOqIE7gusanfLuN7I3s5sCo4EOabq+XRKqKsaxJUm3YiINJYq6W94IQTTrz973cO6N+vsL2luD1nx+76o6uG9KuoTHc45Aww+7/BpWEpR89EcS9kweUCvmDBfGbx8uo+Nvi69MNOVJu65i+o/fn/XFxV3ae+MWfxIKpC2pO3ERo8CNA+EUDpe2OILkEuAr52yR99lXwRLEAdSygMT5fAHyASQvRByAzFCOYn3R11PWyd7IrEy5r5wLlsDhTrX1MzYtTISZMmOUbYAuPYUWMdpMz25SjsaEGGVqrA6t5n1XMAt+NMxUCB6leD6pSM9XbAq573vTn87xxYbStIsRlENaQ9RJVENTvV8sILL77pxr9U1vQLq6St2fJ8w4aDMmMHl/dNtxa3N6XamUdYPn2kwoLR2cSZvhZQXFN+fn7Q2io6c+bMf//rDg2PQ9Bue+3dkM0VZ4LzSYSMw3+QtpUigIMQt32ojKw9EILUertjoVE/pUwRxyeijZi7S7JS8VBFTQ0/p3jaX3JME++JPQF2iYxhkfzNnTlTO2/et771LSXygPfVEcROwXOlicKK8Qa8UsT0An9gDuC/avLGeT2B4UIUqlCLXT0fOKPehL0c+AAcWJ3DQBpKY4gqSSPxk0r94+//8NMf/7jv4GFBt2UXVbS3bDqieu2aVHlHfQeDaUGbYTlDfWg7zmmzIytZnQMrwJ7OpG+cC21pqvBJBzGyUadSDttaa/hQ1/hmc/nydFh1hAN8JEBa+hBkEh946JMnBsLvrh5dRWzP8OstpIUNYEzYVYqVi4tsURDOjpZtIVfiqPGj3pBvDET94moqKt42AtnYfowJdhhIjkhcHTVqFPtDDHtHm9XKEdQL/UE5QCTUlNRRwGIlflBkvel6OdCTHFhtMwBNgt7UNrQHYW2DSnWkwXHHHTtgYN/C9lxZQa6iIzumf2ad/mVVrUuK25zO3+7SVAsEjPNWAJyOYpOVHgSeXM5upBzP7nbXoCQDZON6G/RZcWrnLzr4M/v3KS/uyDeUFABzxkOL1VT50qEUq6wjGWJipJ8i4fGzO2ZTo7E9e3sAdwf5weKTqUk4wZQS94YEVR4BNMdwzNRPCv13v/udnw4dO/Hkk+MOZAwRo4yAo8HKzAZaP3uc2g9Wxv8jqYiHkpIl9egRiPX4f6T4vcVckznQrYJb1URrAxoG7SZASVFMxrnOS3EMGD+ITEFTwZJ3xg0o2Wgkw099pihfZHgdBsHBG6LFtiGeOkE9s2WHqYN4GCLNsHnenjvPQcS2rvavSU/cfuvpjz9QmK2rTgfnIJAQcV4PJ4ots/WjJHYGyael7johh26eJNOg9MGvig4gkiEXD/xo9kZP1B0KCEDukY3Orrj66quZmMXwevIVfPzkHSlUwE6Gx7TdlKw3uic5gP+x1jDfoEFPLNCTGfTi6uXAh+BAtwruQ+B8X0kpKW0DaKeCs4b56MMPc/osKmwtbKofOyi14fCa0pYlRW059+Gx9eRbC2zqb0ku2DIRcBokb6HOzKKa8zOOc2+//W9ijPWPOOSzc2a+WDdv9tRH/zPnpSmG/9aKtclojQUTn0gPkiJVMXIFbTX2DcDkGJN0UtIjgciWiFledAcKIz2drIs/ldchQiAp/Z1cWTxmDMchhEkFUgAqYcCQRNoi2T1CZy+SFXMAq+OjClSQAYfAipP0fu3lwEfGgdUmiwakCkltebQQTjgua62sruZX3pJrGtQnNWZwdUm+rqyoNR3O+HK8jON4neNfYgGguCSc+OhydMu0kCRj24AKHj/tyGf9+Me//1Xs3pWOgrFjRqWL2sod7J9qm/v6DJcX2husDcYxckwibdSqnT8jtqhhu6wMnyQxowcZwQS6hPxgkdQEw4606BSOj4xQKF86PeYrxir3X//6V56FynW0y8cTW7Ov2OIBHyG9IfEzWt4+GFW9qT4AB3Deg/MqS8UJfAAkvUl6ObAqOLDaOgDXrVDk7eGo9/aykuJ/3nZbdvHiCrouWzSspHjH4dXrlGQr2pqo+/r24vrCNHXf0dqSKmhn/y+0XbglnzQkQ+P2ko7CqnCBX6rRRvni4qFrDZpX/869//57daXzNAs22mB8iVucWrMmESXFZalFL859+bHStqypREs+HCrBjqSH6GyZsX1qsVHndsd0ALE9AwDpp0B8d5dkpeKREQfscvHALEZGOkgn0nv8dFhQYWv7nFdn3XfnXe4bsJdsl10nIyeeNa+ATv+PJHWWpRPtShEDGB5IJF8eg0jbj7v8i/gBA9EPxZ8KEgMio/EtlKhH+RbxrznvzoFFLLuCxyKvORT2LCWdVSywfLi7XGK7i6LljT8gRXYHD8DXCE+6ZBHDMWFnOAqed3d4Pi7x5AepCqJonToqRvZIEVZbB5BxoFuiHdScAfsFF5w/cGB/Tv0We/tXlfHhaaeyldnp0KSi+/3uTut3kS/P0Hb3OOZzW2+9ZVmmYtars3GHJ8wO22/lGgFS0tkO7RRbXLtgzuszi4va4zlaNpu2F4VV1o/Fo1eIzQPfhNn9HXAkRvjQI44w+xHwibiYGbAF9VShOiuL4vbEloax4rt8kETvm774Cj7CRMGVFmGeGIlCgZ6ic03Doy5i/6fUiu/d6Qm6ppHaI/TEiTVU6lQ49gEC3SEHhkUAcAajiA0ueVYAD7hTyEHGPiDCwxbxRLQrwNMd/jUwHn+wRYnQFgvrZ0/Rudo6AOthRo5OP3BX36uvvjpz5kxlY68uL8iO6F+ZcRmSwb8LAFx/WNRR3LHUeP3eYrcVlzY56MvGgKD/Gw88YL98ruWx/z7J/bO5KbvbrhNThQ5aD0drxaewJZdJdbw+44Vc0yJfwhiioMi6wnsxr5kxUfpJNoGgWAmHu9d1AzrRgw8+mCWN4Ugb8IhhoeqpUsgRQtnRX3DKnRTKRXznk2S79CXr2OwjpISo1cL9pPrD5ozk+nWVEuN7is41DQ924Q+q8CVKYCdn1jRSe4SeWEao1H6UkKi/ukNOXEESiSg3wOJ2xe7gyY8syI8+AH5psVdApE8eYRIFW/zZHZ6PS7ziePg3RoIjl8T0FP2rTfE5+koxnLijPTz1xJM8+kuLg0l9UGn7wMriVFtTWOFto8bDnt/igm4LnEsVu6yxtb2grKzCPGHU2qP49lzw8wuqKip0MOPXXcdROboZR0G0O6eTdaQ1V1HqcgnLWyQAAEAASURBVIHmOa+9VBLO4szrW53D2FMMXdV4iDuhj9KPgY6IcPuYGNPmzTbbjOjrAzQGqlbTEugpejQnuWjMEApDjhJkxGYmI08Md74BUH9IkioCiCHBYsTD5vFTgxeImHuK2jUHD5GOGlAZcUYY09Yc8nqcEhWtTqElnMorEOWhu4yofgMLIhEBpI07bLqDBwCzB2OJk8DyQhhTRVH8ZEgU7mEOD8kYIEXaSxSq7li0UvGrVfG1tYejwVpyU6c+RbGoSGfLD+9bmumgsi1yFrTx+i8OZ/9S5d092XxLY3NLqqw8207Pp9zMN2fWzHztfD3H6BEDB/StKmzjR5+c0EZeksdcobK8ZEnt2w0L51aVl5AgbO0O/5oWj0sKEXiS2Hns/6Lr6dMddt7ZkUexLUVZ6VlFA3PS7kJPrBE6cB/TiGbU7HLsfGIMeDFg/ESqgOQSxsjIVZ9iSxYQA8yzpjH8Q9KjFtQOTy1tOJZR9cXyfkjMa2ZypVPL6lHNqnTVjU4KqztqAftKhjFKKj+FIekOHozBB+R4CFL/IRcYYqZSSUv2OiWqOzwfl3gFiYVVcOUiSzrIyFUxng9ZkNXWAbg5y2EMNIjZ3J9uvnlA3z5EQF0O7VvWkat3r6NBfUFx2rW/ToJbwTjWPYN5N6K0tDW0t5XW1AwaMuS5qdMdo++w4s032TitfO0t4TpDa6QFRVYKdBKmjgXttgp3zH7tpcL2bAV7k+Xgj8mDReRAGyD0nocffhjhxMJFUaRBTNQvYsR3Dqw+fOFiw44IZYQAUqgFBhlc9sRc4i8HJWmcUqGksyuKATFgSDZq4YFkefJ8Wv7nJyCsmKosFjmqxU9eGTuriSJWRj8JgDcBiAXvBHhXAHPAgxEgEh4SDsm7wDp/+kpgyF6EYfwUE+UKEo8wg4kAsNgKOtN+HAMYokSxUXvjkkItzx8l/TDl6pbRHwbp+0nb1h4mwgrz6oxXotVP2ZyrWcU+09ZKZbew2xSkWp32U5DipNkdzpaC4sZ8hzu6WpsbJ+2684B+/QyKXYfSkmvdZOONCtucJx/EJfCRmYlrTXu42oWpMJigG+tef+XFUvOMj08HEIVAcbBO+Be/+IVTjIjI7rvvTst4fMJJJcWx+O6OdSsVDxWx06j0AQLxp7YX2tyyR3znAxIl3jEGhdq5MJpjTMwdgIDIlSLm4wWMD4qMV1gXWSX88SrC+6dWhSojeI1amJz4GbuELpH4hBsEAHBki5goFV3CE3vi7RO0Rg9+eoSjUMUAYcPzHhT+Lin5aCLjUE/pZBeHXyvmz8pStdoEUZVz5rRg+eSTT9IVNDQXIJebp1N1bU7spLusN3J4ZOZrN0CgILpeBninoWn+4nzadTFN9btstaVl8nseeLC4rLKgpWnkyJHkwMHRphpuarSFmKS5mDcIpemAs+BTHXNmz6ruO6S639DsCmYZK8vUVQmvbZBvO+MwcObMmQtra103OGLECIUlGVHXyF9L8AbcUzdGaZyBmYn5wl6K733ve3GclazVd1HgH5xxujUJrI6TdPRI7o34CB1jvvKVr7Amrb322j/96U87sfj0yWi9SqTRHnPMMeajHLQuueQS2kqkAnYW9hMWUL/x/omXX3557NixuoG4MVPtd1lSFU2o8MeBXe7plOQnP/mJccwKWISHRJ1OFDDxJYebb775WWedJWvYnIPy7LPPahQCugqousz34xKpCJiDG1H7OyvBdh9Fu+qqqzqL8GHay2rrANqKyzNtTWVFHf/97387Ciuy7ekBbYvGFi4oalqss2P/UcGOAypoC/XngOgSxwMZyBelXflkb3B7QVtzqqy+tezut0pzJSl9BH+hyfvutSjXfOdd/xk8cNCChQ3jx49vqn2jIqwitHW05YsYi9pb0q1tRbYSpIqt/2qIxQUtb858rrK0sKRseM5WA8dRh66H61BBJpXuaO1oLVqzrENEXH1rTkxn7pZSBPKx22676TJtAChO2yiXPMkeaR1dT2kavbU+oLUljOn45j76wIO6Uy4X0UNXpGpSZWjTOIX1EPEaW4LrK0O4sYwmykYVuiWcDz6+Lbf/+c/gt95666pMeW3dIsrC11AA07Jk1NzU0qykehE4aQqRkKCkLRuQQO6nSE/MmiBgThxLSqL9C3scAhWbEK0RfianeXun3QearFVQVSiUBB7xzQVL72qWCoUi0cD8Wlq41MQsXkZIkiRoN1PaZYtJGi0M4mH2uvuOf7vjc8stt+TX0GpIY3HLGDfpSqEFFrvVWAS/Y4lECsQi44mvkbHgZS0+LiqIBxbtwoFvyQOGfIunHAFkl26aEa2SupaIjngvXsJP2cVRNmQGGkoHiRyxSNbxHQqWPLGwvuIqFtWUlb0wffrV1/zilltumfnaa5b83baq+UX1r7wSAY5pgySkzBUaUx3tz02d+uK06fXbbF1Rmm7N59ztre4CpKuWkqGDrI3ejAnb2jtkCkO2qfnJRx8zduxf1actrDq35Juar7700tNPP/2Nt94kpxw8YJCLNyQQYpRAYKxckjmHd2BXslzhUzLyDCwiDJHhgJHNQVWMAFSxU4nl5esmMkKKgQEYRwx2BnILUox4AAgG2cmuUBWJYxiSgopPL50GoQcGTAYZiM82p7muKEwuW1xaeu8ddzjufqONNpIKDCQ+yQIqYfj9FHj/z2rrADQatCrtfffdV1pVwRk039hSXZ5xT2o31BuykyIeO8SprdDerY6SxU3qJYhdVd+aQcOGOfR42pSpkouZuNM2+XzwLcOUljyV4XLBUAGMRYFLifS65Ve4sbF+9qyZIycMNSFod/kXlcb3tKQ025RlaO+GmNUWjWIP9ykUOOhfOciB+17eVfFgxHj3FKEh02S2DqHh1bFf+Qr8RNzla9dddx0aSO2Xv/xlFQqS+jNIicwHRkBBiiTQKApNomipaVgqDYzeV2WSUGpiAEjFszdg6ChobmyCM2Jw5S9FoAfSPETKTuuCAVWxDVCtUT/CAKftJuI9dAawSLOCyMVX9AhHbSKVryBj7s4Fh1zzhhByuVRXVcHg1EFtzyfJAUvuHfAkLT+S5GdM4q3E8hITwcQEtAzfRJkqTxQZ5LDJCAEREpgY6tv0CE4YxKBwGQ9Drx850N7SWl6aQRYkQYW4nTQhjHelVAiWXWHiXycAPzxdP45VT1aqZReUftKLywJwJFjV+AS5iqDgGN/9hF/PqvpoOiUibQjQnQuNGj0aJ0Pa4mCoceVqwthw5qOfpUlPU+g+TmOyMEQLYqPdKaMArsIpawQblimIgAcPs7msT20OgkwM4hMnTqysqNhqq61kBOzSCy8675xzI2eQmi7L4DLIWE0SogF+ABRHRaKgy5M1VaXDP/SzJckIDCIBR2rBi4w8MQH3VV6hUImOxgRgshOINPtZ0BI0PoKVyNungGFZV4qHERhh8ISfBe3qFFpptSYxUIGPFYq2WBdEYuHChVD52iPPausAyIYC1i2pZ08oqxlS0NFaXV5UWeq8567L1d7BcTRcmugmR0vEeQOuksybtYusiFhumvvmm9859ZRMaeaxxx7DbgI6caftDTtZ+13uSp+oM/8LO1KELOjG5BfTkEtjsvnW2gXvVLzzxuChw6wTaDLFWpRqTtlm2278skY96l5Zyp172t7++OOPl2p72ezw4cN7UCa6LK/sogIii0OGDLnsssuwiFBmilM2IhDZtdZay/U7FAGRBdxWuHQE56cxiyEwCol1ZXVFVUWlCop3KYiEBzy5d2NiZZ9qLVkBKT79ga+ZkqARNJJZr7wqay2k2NQ+GXb5KiFqAchFQj8rMqEplmTowVxTfcMbCxdKEg0C7akwQHMpMbINJQmTP0oBPAo1P0VAoTYpI+Y01wW7oJRpct68eQqo+PysykozDtZm5YgqIKBKuitJEOknDIjXSpGkUP3612i08+fNi9R6i0x0R7o53/L6zFmSO/G7pqYGHzwpq1+JsnAlhbEu46WMpBKZ1vFoAG3tbrT2NUTSHZmyyCulcOGdI3KNpRwO2LB4Sd++fRUcf4yaY70gqbs7hLGiJZujg/W5tDNsGKKaNC7VTT+WpQMZsFG+fav7yAKMgqgmPZC/UHwcS5Y3NFXlCmEHeyWO+aVFYa4G5rWZs9QUtEpdUVFe17wEw8uSMhoWaLxy8Z4xYwaGE2yVG2tWwPgch+EhPFLJ+je/+Y2OWV70q0/woxCAJAaFzflcybI9ntAqvq8+gcTQ116dCYm0LM+DBw9WRtMYaXWEYGRXUppR9eL1dmpcpMoFgJlvvD5bQK3F4rvO04iGtHiMUbzjRFwAl7xlLSNVD4m3KvMzSguaFRajWHEVGXypg+sbm0imWXI+l3UlFBqYEJFBGCJtCtIjz2pbBI5ibQtYEeYUdrDw0P6VJd32bHb6EotCc0lnDxQVNOeZh0rfWBQGAuoDL1y9bXQwderUULsFHSOGDDJZZNChImSBa2HwSYSCjSfkwsAgqPzOlEsVts2e8Wy+YWFRQUuxjqKlNawYBPhu6ekR7n8AJKhCvyKTCXITZXrcuHEi4/MBcL6fJJgfNWMQ7uQCNQrC05zLLmmoN+Lz1j7pF9vyRKJQErryG9/4xp577rnZxhtvvskm6ujwQw+77bbb+vftq/3AQ6blLoB4bezeu+7ec7fdGXZPOukkfYaRvtubH7rv/iMOOXSn7bbfeIMJB+z7qZ+ce17dgloFV6ekSC6SC3uLzDc2V5dVTH38ya9/+fh999hr28233GbzLY/63Bd+df0vVXeysZy8FZ195lkHf+azRouzZs06/vjj3Zmz5RZb7LPPPtdff732iRidf0U68+Izzx1/zHF77Dp584022X/vfb95/Nd0uhSrVoobsvMOimlZP6Q4//73vz/3uc+ZpK87btzhhx9+9913Qxg5DK0kUVx/dc11n9pz752322GrTTbba/LuJ3z162/Mer08eEWVOtoPJVb1X3/9daUjz9OnT//UPvvuu/c+N/7hj8bLVLzLro847PBDDz7k4v+5cGC//tdfc+0Rhx52zFFfotROOuFb22y51RZbbLH33nvrkuUod0gikd3VdVlx+rGHHvnM/geiatbLr/zsx+dP3GGnzSZshIxzzjlHKiWN9NOJOLbrjjtvtN74LTbaZPeJkw759GcevOe+gTU1TYvrd911V8D4oOoR8N3vfpcik1Df8Lfb/nr4IYfutuukrTbfYucddjzy81949KGHqTwPQYp0AmYScKGFQf3GG2/82c9+9sUXX0S8BwBuEL/IcIxiDceoz3zmM2eccYZ43P7DH/4QC3jUUUdNmjRJkQHHXkFyP0mdvufGG29Um9tvtc2WG2+67RZbHbTfAaec/N1cQ1NHvrW6surmG2/ac/c99tpjz2emP/3pAw7cZIMJ++37qeYlDWwO2frGX197/ecOOWy7LbdW9r132+Nrx3916lNTKsudYhPkSvemmg7Yb/9//OMfqJWdEj3yyCP77befVvDKK68gg/AcljyXX375E088obA77rjjtltvc+D+Bzxw3/0IgAc2Q417777ni1/8IuG0yHHEEUc8+OCDsfj42V09rmz8apsBsM/0yWTeeOMNt66r3Y5cU3lZR8ae3G5KENZtk2ITZ82tozjzTn2+LvEx4eVvEGgHAC2AofrkhQtq+1aXdbTolhM9ngodPsSO/Um5RT1YkiwEOGHaTe7BaMzqk29c8Porz4/bYNPiVKlJSIiyAc198SHjNesh07omsjV71qzKyqBAjQ7eKxMBqqcfWsATsao1uswNbjFroingq2ZG9O3PAEZ8H7z/frUzfsMNNXIE185/x5mv666zjovRxUgF2APYsqHGrJYra/rcf//9+gOo7vz3HfoDXw2+NttkU1avF59/4ZY//+XBxx6jWBFAEcQcNS0CYEhG537qU5/qLL5h59133eVv5oxXzjvvPM2P5WHxwkUP3nuf96+v/+Xiurp1xo17ZcYMo4epU6ZYu6a+WQbemDNnh+23hxyqLTbffMqUKa/NmvXnv/z5sSeeGDNmjEhPtH0JKAJ67rrrrs8cdJDyKsWGG2300IMPPvTwAzoM9BvHQeUBecwxx9x+298iJ6V96cUXX3j++d/+5jdKN2LM6I033Gj6k09J8uD9D7gMjnnnycefmPbUFGkN0o8+8igjoBefe/4///o3mK8cexy2m3Dcc889hDncC/TssxttvHHdokXQnn3WWVZWDz3i8KgEQ47drAHgEuPGY//9L5xnnXkm/eU+UbXz2gydwU90nOeee67CLqlbvP+n9nNvR3FBuGmOCsaTt958867//Oett96ioXWQCig7z6OPPLJgYS2E6u7qy652sykK/cQfF1c4/VcuD099bN3RY2lJtOnb1IBuo6ZfP8PhmTNnQnvXXf+ZNXu2HgWA4QW3PlQZonlk9MzTTz9TUGCOovZdIW4NAHoA995zj5LiDBgdDCZHKSW0xh9f++pXdfBiCKEV42efeebp6dPNe3R1rbl8Xe3CZ6ZNxxBd6UvPv4DcyrLymj59SA4A9+7JQnbeM15+2SDslj/9GfOhkov52dNTp6FND6S+ZAfMRMpKp0GKgb+Zllnd/XffI95k9Jwzzxq21lqmuYubmh958KEDH95fZ88nghHivw8/cvBBn2EcA7ne+us/kDy6B2SHeuyhZ7XNAGLfeN8DDyhIWBZrzfYtLy3I13dXLotU7DP6RiMBTptF6fI5CxbnSivxwlj4oM9+lqDX1tbOeOklMaZmg/v2aW9pDlUS/jiShqmlSYC3ilFDugPTbT0Ao5IVlqpM0cJ5c+bNnWPe5axRYGATO1V3FK2eeKVTKGJthEUMo5Dp8xC8SgkiylHoY3NCAEpCUzQ2TLJeushl1c6cPVFz2oaHKjz/pz+lGY2DLr7sMqrEerKBkrd+m1qMmN+ZO5cfCOCBgwbdfutt48aMVUey+NKXvqRcxpWakFOPDPEU37jhl7/8JYUbWRE0YzKfQBtjzkEHHYQblM5NN92kdbMKGkhCcsUVVyADjBxhFmMV3bjs2eees53ipxdcIHeRGjN9BKHWDrMYg/o//elPVBsfFTAnn3yyrCk4TIiogHl0RWeeeSZ4ML/81a/+85//vPDii4cccoipjDjmI28wN99886233IKA4447DgG6PXc5JIk6wt2fTc1bb7HF2muPoHf+/tfb6Yvqisq/3nJrWCFobze+IeQyNUyWxGP5Ry8YpDopEUU5ddo0N2H8HqOSblVxcFLdxZ4ypnrvG0CozeQxBbn99tvVFw+9HXbYkbhdeMHPpjzxJN8zWolhzeEq11xzjY72qaeeMvpOWlPBtGnT8NzBtHBkm5t1e//817+AYZS+4dvf/jYK99hzz8efeMJA+Jprr5WjmFNOOQXxcVakNlmWLrjgAtWhCAcccEAoQlGRXiHKHgAP/DjgCT+SMmKmSAw34ACJnltvu+3W228HE2kDkMCGyeIdd9xhOqMLN/kT1mWaQEsVZ0sCqlXtY5HSkRki9/3vfx+Ge++9N2p/MkMaX3rpJVdOId4jX8WUFlhcJFAikiZ3XA00M+4lCwmQg4Qc5brb037wA2yk9A3wRaplZUd2lCU1LvIPf/wjuWUqP/TQQ7UXMXB698iz2jqAskxY6cbB/gMGuOWFWb+irKSD80V3D6NM0j7Ze62IWHJa2JAtKK3AdANJy4847lwExmBMZ14sL0ubKegrEx6GFcKkppj4wzGi5CDIBGuS7sdZQ6mCkqKWTHHBG6/Nql+8hBMCdRfguxkudUfjRxCPbA8RMQojx2hkVyFhsg4EJ08nGTG+8+eHCcgUYyHUNvA8hgXgDLkk7BWpFrxjRtbkSfbjTz3FYkA1SGik5hMahWPDgEF7qG9sMH9ftHChaTSRoNTIBrPyo48+Wre4zlDuxG+ftP4G4zPlZZ85+LNHH3uMavvVr36lTj1yhMSjTlFy3/33i9G69Dp777PPWsOHG91fetll8lWA6669lsURBVqptLhGARlwIY+6iWTPmTPHCvD8BQs0PDEnf+c7u0ycOGr0yI032URYzEP33884E5WLUsjaG3+oORrTj0MOPfSoo76ok4DZnEO+eBRmHonL1o9+9CNIavr2/cn55689YgT5/9znP3/ueecpOx1qUJnP5Y3BEXnnHXdIW7tgwX8ffVQSBn0EG6tSUaYLYrbbbjuiriBYGgY6BQU/+MEPOL/JWnFCZEEBqvyMWVNJYrp8wh5TlZKMWL9y/PE0tcQbTJhw2qmnRjX0r3/+0xh514kTqS3FPPiQQyqrqiSxHBW18OuzZ0tOOcpLFjqA7bffniVKpgbdYhSQZY9UGJ/RmLr2G2+66Yc//CHuxbrLt+RN+8STE/2lmV8oQntYHQWA4ZHnykvGohoFIDtf1cKmm25KrasIeeHM5MmTIwcARBhJfJXjjFmz/vHPfw4xb7bYnssNGDhQPPJKrKxApVNJpg7Hf+Urxx179J577DFxl118vZLnpaIVFv76N79Zd731jFT23Guviy68kCoxBzLbW5pWi0h0tNyVXabGE0iy2quksXUQkshV13djlEo8UH0lpTB1I5xz334bn5F92OGH0/vYYpXi7LPPVnaoYv8k8OGfHutJVpYUZahrrq+tXdJvcFjv5qjTRw/Qujj0vF09zvIJZWeaLyhMZ8oal+RamGkypfnG4DeyySabYM3cuXODX1FrK7FzwoR7X8wUJXDHu9RmTjQ/nlLuQQnZaKZ76GgtThWWWO/NO0q6StpXXn5xg822UYlW/5gnuluU7orGjyKOoCupUphOyq+5qYntK5QmaQnLUwBm+Z8fMoxXmp9aQwBUaIhkxKy1CgHxsTF72/ilXhAp/uKLLzacNBuY9corYMRECdYwMNyCzZSnnmKU82gD0prSkXgDZ7pYw9PA2Ap0BrKWUC1D8sZrr0mrgQV1n/iN+OphMo5Z7LXXXgDAU+4yYmk13DOHACOJvGgNkGFQX1BAxdCSAh424sbGJkNR+fpphG4qEAs+e/bsqGeN/lAod2WBMJY9dMnJ9JwbTAODcjKwDevPyrJ4MW4g4+23356TINHhyciATrxcWIFjfRmQUmQekYiUOyZARbWxI9tJoO+xWg4M/NFHH61fsYaJBllLMnr0aNxTWUptBZX2B6akWBGrBkyXDzI8sSAyUnzkQYWSyCulo8uiX6lplo6NTUyPxZiTKSvTUclRXnrugITDazKBa8o2p9JhsBLIaGtbd911xaMWGKlAfEeZCX0wlEtFXpmV4FnS0JApyVB5kVTMB2DKm+bmkchMnMqHrwnFABRZVwF5Z5JYFp/EKIJPqliJbJw0UjQ0YdSyBmlihzBdF1nBKHTGhMKMgRa5xHgIvxkAVGaTVDbK5Qg5XunVBExriFwUKm9fPTLFdoWNVPkZG1H8SbaRFFGhSqQkio8/seJQgv+Sk1g49RP6RWswcEYMH/7dY4hWlpSykqKnpjynzludXs+x101eJen2rOWqrjFlcx38CjNuAi60fWDgC7Prs4Xlxdl5C5e0Hvvlo8ozxWwLc157p6CjjOfshPFrN7MTpaz+BmzY6p1K/AHwtK2VQBC2cNyopXsfbTkuySQnlNltUNcwZ0bhmPU3yRRn2godVRj6FBIgoTrAemH10YN10HWBu4k1UjMQQxJng8TXv2DosGHNiUejFFF2Y9Llw90gW4loNjRe2N7WX2TTGbAsF5bT27jWcHZUecHdzXe2i3lvzz3ttNP+/Kc/4TKTLh33qb33ufqqK2JzJeXULoVCvjGzvSjFCWfOa69995Tv33rrrfVNjTyCqEur/kWFRf+5405gOE9JsXJssfkWU597TkuIuk+9wOarJ/aLGjxN4SdGUVs0dZ+qynyuWbxFIB4pYZXHdkOqoSLTZAUyXcKSGNnBfMFIVN+wOLgMFBZOmfrk1GlPKZKmK5eNN92UEUb71AFEkuAXL62f4DFHq0aSRypmxoH9BzCd88+JfyXFoR8dMnQQZ57SDMeVrIyqqis4BNKbixsbWDvXHj3K5EBXoedDPMvn5z73OWNnubAGRDUKuSXQ9nSqMXF3jmpaeZ2vm3fQCr+6ILB84YNOFI/JgZ5k9IAA35QIi7QO4aJ0iUNz0aDNsI0irKm5QbkUmuxj1/z571iv0f2ceOKJbGISjho16tMH7q8t/Pa3vwXQks8aS9moE5w0WAg7dDwFZaWhV6hfuMQvxbTy1twaZkKB1SWFtt3ks8HxKRlmqWepyywD0naMdIhM4OiIsI0zKnGT8vBgdNxUuIzhwRGMFFnWY+9FsdFhQkBxaRr1iq+wTQ2NfSqrbvjVb84///xZs15FwqRJk0468QT2NNYePx0pjBemNcK0s+kjdKYIYXAQNpQaDrYNGDwoOUKy1ZoJN1OnmVmwNUldVFcXCMDJ4DpCy7ixvEPj0Csxb8S4TJpTWSBGzSiONzdlxeSt5KdMPVFZiVRZfmYqK1q4v5tnFhYLDBo+bN6CeUzhtJd8AXNaF4j1q+a4YCRo3u9rtXUApelioxtOluSS10hNdWlrC63a7YCbS35ra7MOg+5pyrYsaWpOlZQRNSpeJ0ws0sVpFtJ0SUAxaFDoTlfqwUonBaUzNFzHogW1ixfVldcMYq6myuBRGfFRwUmrCH7BK4W/p4DDWCmhQZVHdUNBBB62dTd36qmcu8aDIZEV6BEGFJof981Mhh3Z3F9DsreT/rJQMXPmzCsuD2tosfV6e/yU9g83/5ECPeboox9+4AH2X9N/S2fRvRUMg48hsEailvxUBVYAZRGrQ3Idg7cnDhsNwAycw1ruMrdO66v0y8DBg2J2YbTJgGx1KNmRkHTuQSHCEMgrKmKm8FYQey+POeYYtPkUWmlRcVT9mihiEEA76ANUgf3YUclaolApkgSl01gfZgZJw4ZNhxfZRVbRr4xgIDGoj7WpmFQqso00WZwtY4QaLyzgFeMTPLiqCxTYZZddDAkXNgeHQjzxFunBnMCfZNTiJ/xyjEISYSJ+Y+EEPNRFkB9aKYzcQ90ZIIvxVXlr5y9QTGGLnJBYrogD4RtuuMHCjLI/9NBDSQcQgIFFYINxAVmjRHbB9KcHTaffeuON8RMmIEO8LTsAKmpqhg8dxkQurfj4yMjcHA1wioloEabvtC0glMjnZceh+6ouAlhHolt9SKpTiUAiI7Aj2SwixqDhq1/9KmCdqOUKY3kCwx5lmSHyoTSUIxQETFx5RoZPsjAlali8mHsCfLIgcj6ZQxgYGUsw9xnLk3adgc7OV6Uww+sUD5RAggafYqFCLsls2PgDQrl4RHqjCjxKZr/2enJiZisG6ryfnTZdMfUwUfygInvemCwV5DrTlXpCUVfP09b6wgsvkT23OpYUtA2sruJ/qWl3R4w+D2expLUj1ZhtX9SQd6FYS76VXXib7baVCiNuvukm030aacyoEUEKunq6wx8ELh1mH7jcksvOfu1VY5oSHcCyB3+jZIiI9dQdqlUaH8QzUUykMDYqgtIpT6s06y6Rh9a4THFEgEiPSCucptUijRlNXbUW9plIKklFP8ggsgUFdsky0ZhBG3NZVz3yyCM1VLqSKQ+rJeEBQYnoQrx50GnDNpriQ2DCMnsrMDJAUUYyfvzjH2sk2oYWSGlOe3o6kiy1OSaKLyGYQHayxhNaWnFKcwUvUo5a8uiRo8L21faO2265lXdK3z41Y0ePufrKq4499lh29ggmd6lkKgkksbtSJIYa9grllfsfb7oxnGViMpouMWYcNGTwDjvtqJX+/W9/Y3xQIkjke9FFFyEIVRtsOCHMrtrbDzjo01Cxm+m6/By7zjrj1l03Fu33v/89Avbbf/+gBRNrtXf8hAzYcEYSj0iqwU/xel9stFIa/GuTrXAABBAf6Y+aTpi7Dsr91A/d+tfbgk4pLJiw0YZGlw8/+ohSoNCoi0IEEMmTkXwjDaEnS5a7yYYs5B62hrV3cGe6687/VFs0KM3kmpt3mzRppx12MImBJNIvRwEMiakieWKUzctXMcgjM8IexIevSZXJRQCArMHA6QEQSfIVPIXOfghGmH8qlS1GP21SJRIMXimFr1L5KQynUgibgnzxc5+nfF949rlb//wXfkEmE9bDL738sgi/xdZbmRDEQYkOm3EMTmMFSH7961+jGSqQUeAJLiKVVAxWC4iJ9CusXsLKUChIUdHVl1+xuHYhZ2h8u/GG3xuy2G3gsMsorsbQRkvoVy6FRWcs7/t/r54ZgMKbw0yZPq0kXa7SSttbaspLdQVLpbgr8g3MMYzPZr6gdFGjuyILTOMX1dWfevp5GTv6Wpuamxr585i/lqaLTMw6VvIIEBSpaaM6et7OgLraebXvvDV4rRG5ZTQFmpPxkXpVVcJdkbnK48gTMZMNar3Jk1EkMcq3LJX1VU7B/z+DOPTQQDGE/OEM0YxtzIgYLB3DbYYzDCOp0XQcoRit0SZRJ9ImVX2qia/hmBOBPn/44VJpM6eeeioD/eRdJ919z93nnP1DczKLivQ+50LNZZ0NNuhUvrHlRHXD0LTb7rtzH+TKQlnrD7hIXvDTn4a219b65eOP1+C10ryOx3iCugw3SweVwVSiOdK5igE4VV7OUGCJmJq2iL3vvvvSxRxUNEqN7Wc/+xlbk65FVxELrhQ2mf/PxRed+M1vNjU17rnP3kcddRTPjet+cVXcrCcvBINXLp4eukazHJhNNe6++26+OsiwFMyjN655brPNNv0HD6qdN1/8mLFj1xqxtrHJqWec/hNryGSvsHCXSbs2ZhODezJcBRYf2oTCjNwIXkrJtEbVWETBfwbNv//rX50DTKyLKs8sEoUao4pEAH9ckzb6xZIphlBkPGsJ26hRo554/HEZ/c9FF06cOBFPTvvBaaEf1Q8Vp3IteUv31TU1ZNJ6j14NTuucjipZb9y6L894WT0uql24zrrjmLawHh6zq6h8hQViEbBUmBVLMVGuOsJ2TgPEUFEhxteoSeUbO4zSdGlLR86gAYBC6f5z7a1nnH1WlMlgk0m6B9yWBap+9JMfDxu+lq+cfN6e+3aqpJgkeGebc8HKVMKa1EbXJjSAb3VYzNFHH33lFVfAdMThh59y6qlWd0zjCCSivnHit+xa0LOuPWpkLIJTGvklagImwQ899FCkCs3UPQmEPJJBHhiFbGxkFQpjT1O3cEVJqznBNb+8/svHHtvclLcpwZBo1qxZV1x5BTMZEbU3XnKosPHMH/xAZ/Xb3/3OEAq2YBJZmWele4yVQd4FbKjARHWyp015alo6U2GHSKolW+XQTzbQVLcdEqMadI6Ha0tXvLmwQb9JfkTtte8+YUhSWGCBzkZCLXqzTSbkso1d5L3CKN1LPpxAHbx0+YuUFxe9MWtGW3M98fKQHpXnjfgoSStEtgo/hoaRPGTdyhViSMYqzO9/Q02/2DOqDZjtLm1syZjU+TMHH3YoFUbjX375ZTvvvNPRxxz9mUMO5nEBmHamDZXDDRByYFXHWA2D78q2O+xAoH941lnWORX2kssu3XyLLTR+GocL0O//+AfJP/eFz1Oj2hsVrAfChEimmlrcUP/r3/12v08fSAneevtfTzn1lAt+RmsXjB23zkOPPjpk2FBTRgorm88ZT4hHQ1myehlrGUuN6GFraGr8+je/cchhh2pRf/rLn7945BfPPe9cWn6jTTfV5uWo+YGkm4J6SuwMmp+G+pWvfU3yp6dO/fZJJ113zTUnf+97O+2yi5gmTd05J/X1u06efO311+tswlLqued+5+STObBDePoZZ9CV4UCqYAy3kSKne1AKGkfbppt1mTtPhEpMu4KsN359kcQS5fAjw1tCAdiwTs/HtKKjAhCHw4QYjCrzRi2eg48/xcSGKcCnDkk6gB+efbbc+STecttt9sKyeX31618365FGx7DrxIlf8+jU1XgqpRvTs2IFj0/ed/CcfuqpVnoRgyTrOuPWGSd88aWXnHDCCfDHjIRR4vETzXoOJEGCvbFoqBLwVSmCRk5G/d7IjgUHTwYam8KeEv29LCS5+KKLrrr8chz21U/wMHhTytHx1PaFffexuW5fhqAjjzrKJ7ygqTUrZABWqDi4iT/VtRnDHXfeaUnfR0MBFXfTjTcKf+nYY04/8wzL3ToodXTplVdAhQxHEjEu3fKXvzCBIiNwyazCuCepKTA4zt8MfvUYzA+h0tihw3yRTvvsIQef8O2TMPbpZ585+bvfufzKK0jInnvvJZW85Bsr2k+MizhFruzTrcJdWUTvB75TwgAvmPt2vpkfv+247UxA5alw9aNOf0VP6DdLckWlb9flUxX9GvOtG0ywG3H9UL+FLYwGznHg/DN82FDGkRXh6epbWKJiXgw+J47lCMxtbloyd/bMgesNjGRHGRLvZ6jO1fTQm4ZslqiIslEkcTSyC224ZSWnPD1EP/3yy9/8RvNgzaDBtUnNBpe8LVqan7I5BGeeoiI2a3t87rzz35Z2wfNs0M5+c8MNjNEGmJ3NjP2EewkMURcYpv31b7dTLkbTRqbMDjz8dtppJ0ZfFSFHDNHYlAYGMVjhYZV+8MEHDdDsVKKVLAbssstONf36UiV6JsiPOuqoAw88kC01GHCtXoTToorjCYsYa9xXUpbhR3DlNb845IjDeT3KCNrRo0fvNml3Q7/4EzapZO1NhdEdKKQaqBVZI0y+EyftYoZ+8KGHmN/Iq7QsU7dk8YEHfXrLzTdXKIvJEhqPKxQ7GKbZW6EgECrUYUccjhgs5eQWS7f++utffPnllKOhpTfmywUzmeOHDR0quVUBNBg/EpJrr71WjHwBIJId5ucXXcS8w+4RsyDSHuVSO1FvRqHQ3xjtGl1qUwOHDsEoa/iNzeEsoE0225QViPqe99bbam3SpEnQmgrojOUlI4MonQKLn5UPuZABqRCz9sgRd95zl81fap99jLTsuPPOigzGIRO4z+MeJeRZoVSZzeQ8gqLfPW8cdFKvOikwEKaKAk4cuOTSS62jmjaJRANKrJrQ40wkNQP76/Zi1ThcAW0WS3H7rLPOssftpVdf1kHKgkwC23Tzzb11HWgz97JHF2GmO6rYIyOblTKl5dtsv92jjz/GpY00Qsh1RxG23X47DOS2ELoojptHHM6dYepTQfZMYQmD5HGyiI0KYqWcXNlxRpgXLVkirbxU3KVXXqng1lrkGMm2rWTPyZPMQaElIVitaISns04nT56M1XByFvKGKtbg+38Xzkt2Frz/BB8GUjE6kz/3xH0T9zyovO+w8o6Wgfl3dp8wcECquaCluYP7SFePkyD442QLMzNzlfe8uKCjfEBtfePZP/zhN044uTlXV16R+sWVV3//xFOdnPeVYw49/JB9C5rD7pv3/1jgM/Dn8uBIQb4P4agY+4cLikduuaeGDY/6xl8VI6y1eN4/8h6E5K1B75s2NSypH7nW8L79+nGfX7hokV1VkbAezOv9oNKoPJiDIfgT2eItUnLNNcZreFSVt5tsxIOkQ4m7cExCCYuJNlNfiYpW4eHF4yc8EGoYwrEidIG+wknfaWCQaABqR+uKYJLETMWDdO6PeLNsQzBIKsrKjf+lylo5yOYs2Nm9Rk3IRSpbDovLQnJVT9EAE+8tI6eQRvwRMuSYHMwpIBdNWiphuccCNuTCZBSdvsYmGumx2Egp6PyWZ5Hk9TG7ZBTML05hrQGigV1SjwgA8lhYkVDhoQ2lzlKFHXmLlyzhVmekbCzK3CGJSCQpcr++Nb/+zW+PPfroGa++St0gD+WegByHU4WcYeh61Jri0CxyCdwuDANh9MvLA5vie+OjB7CairWAEp+K02EPnXgPUsVEsFiPpl+wiZeveNMdytycuyQc15KsCVH0xUW8YtSF7jlAFhSGI30IkvF12JwfzlzSZwS0DggrCCcXIXVxvlmjUIn6A0Uzq8uyMZk2JSdWseCHhNlcYGkux3msvjk4KCM+kkf763gSNQ19HuvQqfgxeeAV75KksJGfsdSRRSoRw6kDhSLNdikHVmSbVZaEsd6xSBKY+TipUKhi5eJAKKPrTMIBlKHb9kYSYvSRupABfWtiQm+ovKUFYDbkHSouaWL47EFA0k0Ceb/PSvcY7xfxe+Ai6Z3RdfaIhwpgOOUq1eGeXtPZzq9dBsICSHHx2wtq3SKWa2lty+X33n8/TIkVOXPmTKn4f6011NB4pYfDMAczbmgJdoYVcFIqam/LmQVoUYk7gSoEA6BL2j6ySCISRYF4OWwAbbLmw/CREfCujMg0/aIKCJ/HVzWCUegkoNoGCulo8aGdJNN5TYt8g5HQVz9BAtD8cBsqkT6FhmjnjjNBi4NzG/uyUaGpsbCaAkD0YdZ0hWWqgryFIxkhbVs4alGMWpO1cTEAjdNbLg3NTYgU8ECFDI0QwjD899NEIV3CEGTYzgAV811cv0QSGCQRiK1aLsrCrKQZoxC1gCUUE0fNvlIE3mhATyQ1MkRGYsRDAhu2+OqTGDTjQ0JdUMS4hz+IjDpFoXyCE4yvkMRPyo482LDF+D3RaKFDgvyPN9507DHHXPWLX1hIBy9rGGJy2XlC1smwBkKfIPHVQ+XJFAAkckFezA5MzBrTZBqZKSxJ5LDRNCQJbtwvSmXSDHQaPD435Jp5rwamLbOvSg7YIwvYxMcYPyGUXCqRABQHAQIwyw4AsVGcyDo8RJV41AJDPMj4STyckDNX+uSnRymUCIao2VW6P/Uox2BpcZidXoeXjjmHyWIqdE6xCOTEGkyoiGQ6IgvIoRIfRV0uYjoLCEBYLgTYE2EC262HJd5BUMXiK5EeCIA+YEljg1xkKneYvZu0AqcBx+FOwiXY5Au5Qgms1LPKTUBKiMUe5V8qCoWF6mD68zPt/eGsXFSQy1SqXR68CqgkXS9itHQUZtraG9ta52Qra1N9U9m5e03ebty49XKLGkvLywjSm7Pm1lT1a6hfUOF0//aV7gA0TTMxnkAyauMektxRYz25ddGrFYPKW7KaYhlfdbXFvdc9xFYJV4rRPQas0ar7wqLK6updd5t8+SWX4uo9d9/NCQGTSQDedmqHwPf/N+nqMRKWR0TfxZ80tYbGqh7aanvwqxFvVStq8DA+SsZQ1FvYyp0qxmSBzreZAHg7fXAYkqXJE2FIVKKGaj4R9kx5GyZq3pqNZu8tLH5pM6BfllkSY44lpWk0RKNgOBuypSOTKm3NtRYXGrMXqHQ+Fc7YkXVIaJOgNmw4VhD8icOOAXQF0sJjuuNNO4QfHLqXZaTIISZ5omKKYk+/SMsh32HILVle7YUCoMSwmgvk7UVhnUh2pIAs5FQePgPg7hCyhYG+EGdEbMBrkCvsPE6nMggECI/pgne6uMXvcKxKG2AdmBLphzCHaqhvaFh3/fX+9s9/sG8E+kHhSeIElZSP4JeFJTBueB0dTvhTAfRMUJDFKXfp2ZYVLmkKBDBVJ/kWhxOt+c6hWeZZGw8Sh2nbLPyMOMNyyzIazaqVuixd5h34UFgS6hZcclNAeyroQVSZ2PkQRBy3ddgii+nbUNeBIezjMAZ3wfBfEdIVZUrt4LTAMcqUE2YqLS3R0gqMsuFfyjQdhmpKFeSomnB8mFFF6GslQIVNwHlFSBzqFVq7IU6kK2QWO8WOgsBzlgEx7sawGLusFhyLZGQYUHlidsn+0bAlIqliLJJTqOTA96WGGr/cVtgpOWE9KgCE/CB3IgiyTSaCwISsbWUNH7HOOd8hROCXiV/86RtRXxp+3/+W0f2+E3x4wKCUCgsd0mKCp3fQYDLBChkQJ6zuOgdDiLyj4Q0k8zm13NTQcuCBB2WTrT20nsdaKLSMcEy0scPvGlE3sdLiv4Rkw03CmpeToonlwkWLaTH7SlJJHSPeQ0S6QbPKo+Uex5LGOMyL+lXz2SuvvJK4R1Vo+IA83YAA4FVOUG8GayoH6H2P9kUwTAWceMNFqjtiSQtzNo/Cu++7z1EcMclqlPPu6OyN71kOfBSKrFNjUrKRenrWyobZEKVr3bssQ+WGWbnjOrt7ivjpldfMeLslz8DQkeOXu8WWW1GFobu29pvPzZz1CpXNMXjQIC60SXfZHa6u4sNQxgAzXB/Q4mzqgtaccUhZSfGSBodHOvLM8MS0HI0BM2NUVzg+ijhNupOfRnMOujHnZSu87rrrDFgMW4z4vMFgb2Bp7/N/kgOEQesgLWFkk9hzNLoVyIN2ZGnRMqOTh63E4FmcUf2fZN7/oUJ3r3F7lAmdqh9WUmi48frst8L4xBUtHa387uPV7Uy13WZb2FbbknmllrdsWWFrdvz49dbdYCOjYLIdbYJ8Y+OAhfUT4m7xdPPBfBCRCGL6txoRBv/tYVmyqrovf2CzZSZISUV6r3LDSjdEijagUzoDNAGc/NGPfoSZPAFO/8EPmFyjSQRDfI0tv3tMvV8+yRwgqGQgDgW0C48YotJdmX3ygJfKQ7SiqHcH3xv/yeDAR9QBYFbsAwiWAOeEyD4K30WpDEBcMH1izeyOrfbxvfDmwsUlZZbyGhcv+cKRx3WknJESnE9o7UV1HMmNWfIjR6ylL4nHvHSHqst4Y3xalfUtfi0uKeUy0thWtNbIsY4hCXZtw2oTFmfEJa8ukXwEkbS/bkl7lpey77rrrnyfNV0/TQKM+uPiUmBmsi70EZDUm8UayAECoKGRE2+q3CMmik2X1PrkiWIDICaXqkvg3shPDAe6Vbg9VUKS5IGNIMYwOeO4aHFJTHDY72gNK1/JEkkwrnTzWJB7Y0lrW9mALP+fttYdJ+7IdyRjNpDYjvhTE3W6b9vkdlBDmG7QdBttkQo9cezD5tPY0pFPlQ8du+HgIcNbuYSGxUx2lTis7n6a0i36HvuASPZ9dGKjPonGt7GFFaiqutquIq5QJkDaLTBZRs73WN69iD4+HCAhUekjmTB0Pt2VgKjENU8AAhHeVLI7+N74TwYHulW4PVs84kWkOjsANmv4g6GGg25bS3k6LNQb1K7AZaW+qXVhzvnPfaz5961Oj1lvtDF7YeL15Oaf+e+8Y9rqPMLRo0fLx/R3ZemXeRwBWQywF7O9pKLPkBFD19mwqDhjYqIHiy4lsRTxYNGVzaJH4E1TsJGhHyUaqj7PaSqTJk2yCEzfu7ckusTIS/vvbcA9wvOPIxKjBLXf2egEPCsoCOEnWgBigIxpnh9gILWCLHo/rYEcWOUdQKfqJ3+xAxBgu2ehJGpWU228DR72YaoKoFsZXbi4xZ70xdnF/P/33HPv4nQlTR1ubOQnnnhJx3VZYYIL1crymrrkLqHNSJ5tbe8zYNCIcRPyhaW5pYfsBLROqwhTlRVOpVc235WFR4aGqoAcnyh9fQDK3S3Vks2WVVQ44pw5KOp9kNE0tLJZ9MJ/AjgQDfpkNZaFkHg6f763gOBj2wFjDEGE4vzyvZC9MZ8kDqzyDsAF1rS8P2crWmEVcLaig71oUxexcLwXlSlsT7dlWTRwNupZR/TzuywsTtvlYpNPayozo6CmvLB1aG5BvrFuo213LcuUu9WpkTMQx+eO4rffWmiiazNBZXkq17gkkziMd1lPTgDjA8y22dqeaulIhSOViwudJFFenBffUlha314+ZPTGI9bZkAWlsLXBonChzQockFvaUvoaexfs+1h5L6MuifkAkSYj/JENzTTR4CidKmxsyU3ee88hg4fYT8c32XG14m2DDOzOhImC5wNk9AGS0B3vej4AEkmWR9Idhliu9767g38/OLtL+3GMfy9nYkx3ZeFCbkrNtZwLhJ0HcZeAHW3dwX9c4pUaqWp/+S4wRn5cirBK6VzlHQBVhfvKgOmGGB41EXYzhlrxqd2GoeWJ8BWwmM4W6wpvTjgL6xZJW5Jm/u5w8J4bl4xTLAZ46zPYvgPqwgJO8f7HyWyXjCsqLiHdwX20oK2ksC1TnHJjCK2a1eOUltc35Uavs/6Q4SNtG3XsA1N6l0jWwEjnU5pXcfd28hRWJ8zB+5U2hX2Yoi3froQ7f8ZwfMO//M/O8PL5ilz+5wrC72rYK4Ds/PT+kXcm6Q30cuCTyoHlde8qKSOFHhs57AKUtZhOLyBLuGkHMIQ9wUHZBv8amj82f1vww6RVb2BrXxGt7bYeO6FF8Huh5fURVhCy2XDAgLOZyjNlFpIdkMCYlHztujg5O31tbuzgKWSjI9/OfNgV6JDR4vIFS7Lrb7JV/yEj9AVcP60E2BvQNZY1L9Yen1hq1ys+99xzFlVC1/uRe3EsX9cx7L08t971c/lPy4c78Swf+a5w5xChM/AugHf9fD8435Wk9+cnhgO9td9lVa7yDkCuWB9Hat4xbOlSvJ+2gVkBtj86GGXCE479TwLxq83n7lVzDVBRY3OeOtYHTNhwY1dKsIDbJx1d/tlCXp/1qlUvhRETs4hI3vtmeZIVc39pWHi2As2RNNXS6v7fyvGbbls5YGhrYUlHcVlzjjVFziu9tfq9OX40MX1qaj590EEJS9tZgQR0AJ1+HR8NDZ25xCpAw/t/OtMuH4Bn+Z+d4c745fF3fu0y0Jmky6+fvMhVWl7IVyn+HqwOEgJbJDi+/YyRPZjLxxfVKu8AItNx3HA+jEmTPiC4soRqCQP/jPtwLOQGFjrGcOkGLv/CtZ7hpCQ2+HRj3qXuzr8rWbBoyeTJu9utYrRrMpFvybLeOCArnAVtumAJIalrn7qrEsla2x0kG04QtJsK8paOdEll/1Hrb1QzcCjjkJO52zqKXDKjX4kHlXSHao2KxxCXNDnr2PGAD97/AFuQqzk+Avo7m9a7uKE6Qo2854nx7/3UHZ53oY1gIiH27kwl0Bn5riRd/uxM2OXXT0ZkJ696sDjL41w+3INZrApUndUtsCrwf3xxrnI/38j62Fzjm6r6fz872sosEzPBsP6EbcDU/dJOQExo3wWuyClqaG4tSpX6JG7suuN0JB5Md3IfK5ADEukDS8VOG3NaA+NQcC/txvqdb8857zP4PIQTx8py7amKPoPGrr9xcWVlLt8WTpKSHxtROGTRGgUs3fYla1Stu1doe3cNurR60KC/3347Fw4+QjrCFfSFPUt/bFqxZmF+b0t7b0yXBLwLz/Iw78Ug5r2Ryyd5b3hl4d+FobvknQV/F/xq/4ngnqKty7L3IP5VyqtIfE+xYpWS+lEiX+UzABz34D5NFI3US8Uo9MRhJTjtJHKBpGPmce9r6KSTn+HwReb4glRD1m1tzugPJ0FWVfcJO3OTI3Gg4JppRcG43tEnzjAx8uWouQL3R8fN6iocMFhYXJbvKBm89rgNttyxrbyf+0niIrXsnevKNmRbQTgU6GPy6BFdQ7HlNts42FZ3OH3qNPafMIn5aJ+k+kLlqWtP6H+WPTFm+feyL+H/u8jsxPOu+M6fhCrmooweaP2MkZ0w7wr8rzjfBd/78xPAgS5FosvIT0BhP0ARPqIOAGUaeWylnVSqBiagoP8p/NCckyeZo8WfSYs1XSg0NndImwCjjVt4jO55uWTKnJ++hP+PyyCD99oy93zmBQmXoXv3f6ksAWSb80WFJWPXnbD22PGNbamcu+nRYsNXctamvQD2JZTYZfDRetG8m9aV+Y23Rv177rmnw8MNv12qVV1ZFa+eWBk0PQOL/1G5L6/uhaMMxE8x3AnQZcbL1+Py4QgspjO5wHsBlse54q/LQ37ywj1S9hUgWcGnNYSZKOx81hCS1hAyVv0IMZhagn8P5+KlOtr4nbHGCmtRSVtr3vWLS43/ztrm4O4GqFIXwuVK+Pi3NNpzW1haXd+RynXkndbtjML+1eWp1oY//fFPd955z7PPvPD2ay9W9e1X06fGfcLl1aVV/SpblzS1NrWVhpuJki1Rju/Pu+Qr7JsVoCX/v/buA87OomoY+N27vW86BIIJCSUEpCUEEgMpgID6WmgqoFIUEHwBlSIdFJUmIiI2UC8fAABAAElEQVRFlKoUCSioiIbeg3QJgZAQQihJSLJ99969u/v955mwX34f4f19eY2G4POwPJk7z5kzZ87MnDNz5sxMb3FF/w2HD9lg08qaATmmE9cM9xYc/oM8i8MO1bc2EZoLU1CmsoD01Xl6lLIo7E3zUEUQwmTCkbXYHGxcwTYSthYnA9jAkP937Ls6ma0EawaggC4XtNOiprb2qsuvuOvOP47bfuzkqclhQVxdk4PykSQA2NxqpdT/X8FYIjQbcXtjpoMowg0sLjo/5WQx3zz66K22/ri78aCrDFAlLlOkimbPnq3UI0eOnDx58ujNx+Rz4VJGl9i4Ujx68Urr3DIwbgpw2UX/gQN23nlnHl9KJJ6q4AEGcuGCN9x/UFZaSs+Nm7gT2+EjDz509913K5TLV0dttin3YTcxuUbDfYRYgQw4Q2FdxFFS9tLLszfdfHMzJDPFq6++esH8+a6I+tH5F3BJQICy8KMNOSZ7LOQoldbCZwwSZQfDRc1ARMHra2pfffXVG6673sV+n/n0p13X1y5BMgpRNMBgYiqRKA9pkxWRwP9EV/kaY+QiEIFlhwBkWySL8d4wxFSQQO6rn+LB+Kke+wAkFx+rqSefc1AgtE73dPu85hZ9o93+BUNEG99+SsIlAn7hlWnwM3TUxK966bJlzhyEUKZnn322T/EYErkHPIkTh68OpQLvqzA6YyDEJNe9yEKqPjr9RFWEF8aoWBYJbRCSFuaIJL4jpDDk4FEClcgAn2xdjpl6i4yYsz0udyq78rLL33zzTfi/853v6IYCHYVwJY5qkkXMBZKAnxRKnogzooLcg56P3vOvVwArTcwxMdZcYHSGqOzhZBNcP9UfIxBLkDpIGnRSE4HnIUAQFIcRZdiyFa5RrnTp6K233spOM6D/YGfDNbY0V1TXqMvKqgF1dQ1zF8zpX1vVkW8MFdydo3jsmNJM8gU3EGm2NRttMGrQeiPIyXwh7EbzFMJlkOUICPX8Xk2r8NWV/tpHSRmJvqIvwRyPGLK63VuyogEl/2jBBHCQ/cnu4jXTrnQGV/3ROdi4aNGiuXPnurv1nB/98PLLL//yAQeQy9Y2PAQVwlZTr4X+zLAWJH7yRDGB5wMbGo477rgbb7xR191nv/3E6Jb6HqcsF6uecMIJeBqL55rWrnz+wgt/at8yHC7jBiAVeG8CEm0ChgufmLzLtGnT8p0F+0XkKLnchRFw2S9+oUm4I1cfLuS7nn322Z9dfLHP4DfbYrQ7TEhzOyEcNuVGXElCgYuL/fzOcd8ePWaL4088MV7w+RfPn/9sc+I5554X5Q5gDUbtxySapEzFkPioFQBQWV3lHjGeZopPkVx0wQXYOHmXXaRCG8zilT3YIROaYUCt5HCK8UlhhXEpflJeX4U90nr3xYiXNkqxmBwGkTAAi6nCO/BmhS5XEDAeCS1eXfKznwkfceSR++67L+J9RadSiZQE5ogWwkB2chqu75HgmBFIDhqu7nCT8EHJAxKAihPAl1NOOQXByg5VxAY75GKglRweOIXdzRb5Bh6kSBg8wrHIOIwkaQMiJx4mV4HG4idUh3iQkTxgcIqHXww8kouUY4QREBMAEpz33HPPX5Jr6M8444xwu5b26c7nhCcgPStlEQiGJFICf6RKLhHyI/b+d5QKQ3ENQ2PAO7ZUzGXidx1SwYiYREoUQATTDOKyQOg3PYWK0lD3Ejqy/JJLLrnppptdmaZKdOyPb73tiI1HLeUe1NLS1tY+dNiI6oZBzR1cODUW1/OE3bAUSIemUlk+cPjwzbae0G/oiEJReaee/Z7pucKWWp6ooa+jw6ABPW7XDH+rW98WGLSfMH1ILB6aaWw6rltyrXWCXhHDQ/SHoq+hxx7OcANUa6tb6wzJKQCdkPS0M47A5SWlP+gneKgjadP/i2wVJFSZ6586O3VmAbsuiADiQHH0IdlBLgulfv7557/3ve+J3HDYsAsuvPDU004TSRwagjm5L4p1o3tIQs0maEOv0816e80VInkrwJLeKPeQS9JX++SC28BHjhrl5nGQAFx3PnLjjSk8NAQWJ/LUwVMbb7zx73//exyIHVs8eDCwRekfJYifiIxFEAbmHUUMZgoTGZEMkcDEeOCUqqayymxAlXK+cklv8DHu7lFgTgphcO2+qp5en7zdPiaG91pY4Er+ODT76Rov8D6F63CdQBtGRq4SY4h0XG64HMpP8HLhoSwyAqBEQbxRiy3CKFQc4U0322zrbbbBIkT6KTKAJZhhc61uX+7weyJXvaGKP8E//vjju+yyyxOPPy4ylj0gdwV0UmXVVeHOZJ9wILICwW6zUtJInhKJgZ9SBIm8WAUww6ZGYkB1CEQKAQjUVlVHBoapmMGSBcDksi1o3fMVCA4OH2FXvHcsC8gIJlNfldGf+oXZOzStDI/BHGoDqfkueCLyyFg4VZxPCPOOzUNY0dAp5iP5/DtmACszDls9cRSGucRRp9suw/ST3462yx2IiAlXG+vuOigRWZzpqbRnN1yDmS8tqZw+fXpDQ/2SJUtPO+2Mbx75LSu+BN+yxub5C16f9fwz2bKK9YaNnD/3lZLyrryNYzYO6FEVNYPWHzx4vaF1DfVN+eJCcXmxu/nc45wMENwFHxrMe1VOC/gvtJIgcMIJ0Kv7aHBShsRxhMKIoA+4/IbECZufg5bJBidXuWhnq4t+1fCyM65sbG4isY49/vhP77kXZXn00Ue/6Oa1TOaOO+742qGH2NkPjCTQG7Fm1Yg+IFZCvUgWOoPqU8bnnnvurLPOevjee6UIRX5viKSfs7Fcd911kuinRuKuGZF2iy22+PKXvkS//ulPfzr44INPPPHEI488MlCSyIK62tpjjjlGqgFDBhvCu0+1uq6WwNX9tBMw8oVQXhgoMqZyifkXvvAF0koM21FZWSmeuh43bApJui5p4mtIg/e9vUyFbe1tET6yHrUKRZ8RDfQZKxD1JtxQWwfMfMIUEyoPSMiJWwDEk1REBT+F+OQ722lf45JQ6uRK90gtzMruk+QgpaKlEI9CpRMQ6RMYUk++HhXnpxLBIBLZ3mJiErkjkl6POhgGZSRAJUzIDNJf2oaBA++77z4wIpUCAAZC4gZKybFRKgTEyMCdxKQjIAZOPwFLDlv8KXf0wCOhrx5FN3UWgyQDDlcNyzfXHaRtLCxUwEyVZDSgX3/xsTixFQlDCCaySy6Sa1rSGti1tIWrfRVTWkhwCW3eUkmisPH2C/DiPYGgRL5DAgxmVAGuqCuvpqaAJYVSClfsBgq7CyrinXfekZ0iqDhI5Ig275iR5MLQ9hU5yeQj9fq3KgC1gstqWr3qyjqKGXCLHV6ZGmM/LdmNjnSCJQF3gWeDW2g4QYigrkqGKEYMtIAKNoxQCd/61rcqK2ryvWW0RW1dw7hxgyfutGNra+OQYSNrGvove+dtcwgrxeUVVaWVNdU19cWl5a0aU9g9EEYUxmCEe9iGXJT0wFI3fYe+ZCtajzkAWoN8DhJntZ4wxg/tleVY6bp77GC2JFBUavEBSk9oXtC+J8tWC/n/AEzLYJnB9bMv/mPDoRsQrEY3hx566Ne++lWpGOK/fNCBsW/HNv0/oFrlJ51BfKQ/dsJdJ08W06+hYY899rj5d7dAG/uhAICJEycOHTqU9X+rrbaK8aNGjQKvby1cuBB5saPqiqjSye+5557f/OY3Mvjtb3+rl+qBOrlPoUaSB05JJA8NKHmg/cPvfw9eH2aVru/f7+j//haxDoaO+fSnP+2c1M033zwarOFwVOoDDz301a9+df/994ccMXKBKQo1md5///0/+clP3njjDfH96xtYGlnP3cqt1IDlHraph+oNA8lQKKK5q4twueGGGy68+KdkB81Ht33pS1/qTS6WaGlq3m+//YjRSZMmsaIoppX5Z595VlgWRx11lK0bpHNtdc3cV+eed955lkwgcSPjpZdeeshhX1+yZMno0aPPPfdcF1Rz6br+2uuuv/56go/bG75Rq+p3+3FjaReyDHnoRBjMHvOeL37xi+hREcDAK104L6QjB8kTTzwhOyJSKssnupI7hLHaT+VCg4ByUd6/vvIK5dVqpaWeUW4Bxs9YC4gxyNC6cE91//jHPx4xchOHfSmUfTtvvrHwwgsvlFdQAAMGTJ069dvf/jZsqlUuagSd6g7ZsoNBXXDoUPYZM2ZEbWr6cuaZZ5rAaTCQ8OGW8OeXX3HzzTdHBUCSnHzyyWEbvDPcLYiE6ZCzgRdffPHFf/3rXyGXls1gRetNpLlSY4u8zj/3vJtuuino+OJip8scccQRWCq54otBHlI9yIuNMP78iL3/HQoAB7EV4/AU34WZquNP4/NWijpTS+IadQSeJ8MLYjRcw2Iq0F1wN1dlWTB6hu5fFiQC8Sq5fquCnePjk1XlTuNbjcm5EdlsTcP6NXVDZWeiTPG77rkz2IT0ypqi3pzLZywqGFhKaJEWWLA4mYBEIovDqrQnxGd7LVKv1uM69GRRI5DvQbO3EUlQUx5Z0ApBza14NLXVwv9BwKjVlE85/bQw8+3t1VvE1PVrAK8nv/7665ZG3122VE9TMP1hRev+IHTvi5dQKmWJbwEg+x9wwInHHWctlwIQ44k8xFhiQpeOfclP8WYMkuD4mDFjDELFACCwkEo8sVOp2WOOPdaCKmEElaNAACiFHmu2rrrhkTwUISHD4R8GnjP+9jdoly5diuGPPvqoOlMB77z19uJ3Fm23zbaDBw568omZ7qAndF7yzJpleRnPIfF/d1dQKpFmiuSIww83lwmVT810FR555JHTTjvt+JNORICcQao+jwAZFyrOvDWbNQ1iJ4lI4Dx65pPPPvX0lVde3tjYTAfPfPQxMBttsGG0vbD/MD48NfNJCL9x6GGhgfZmXnlpNtEjJjSPTObNBW+89I8XWe2MTzvb2qOl4sLzznfUq6xRN2zYsFdnvzz3lTm/ve76u++7h8JQ9QqYEF6MaQhG3kMPPhiy3mgjUwozKiNxW8RtufnxOT+89tprY14Ke+pJ30Pnscd/R0Uk5QsFhErdSfXCc8/HoczsWS/5O+RrB7c2txiWobO2pmbsdtsT2ZEn/3j+hZmPP/Hc8/+gjaorKrUKtqOW5mY0wDbv1VeffOIJBFj7UVLk6a2QqF9PlLBWrQ444IA5s2ejSv9/s7h49ouz/vrnu2bccw8BXV1Z+e6771LJzz7zjIRY7T2nUPjspz79zaOOMmssqM2irBneZz/9mdkvvQRARq/Pmyft6E03qza9a20Vo7HRLkcf+c3bpk+HQV7KO+el2dNvvuWuv/xlx0kTEQYMzQgLAInGQmEMf8TeqysH/jfF7+OjWtdWMNe97YkAMQTn4tlj2mjJLXKb1AeD+wEyDpRdGpw4rhA9qiEOH9Dh/B9CQe0xATHXyIXJ3hk+7bne9kJRU66rg/+oS+TdGNxLKehazPw5y2TV5sJss9aEe4KqT/Iqylku7jU1sAodnFOLuhkQO0ySV7fAYRySLbOcoBNyQ3KkqQF/Id9eUcyS22P4GswlpgVKZ0man9Eaepgtyis4PnWZ1ITeRXEWFxMi0JO2H/vYxwQ0fW0dD+W+utmiWloJ8ZloICz+/uyzrP/s72EbNqTJE7uNoLxIJT+FJdR1v37YYSHToiJ9GBKPX95GA5dddhnxBNigmErwSOKJOKPoFyl3vVozQAz88R1QJirNWO+iiy4K3E3mH1YCdt99d8vFBLQYBP/Xf/3XL664wmgR/fKNKxDg/aQgj/j617W8HSdOnDd/PqFm7VRLJXN5KwVaV7LqSBLLpe2RrY899pjxprkXyI2GDaMYr/rlL++/7wG7MAwjQ5u3kGjVl8wiClGbaMH4zhmN1tUd/93vYqDc8QEljz/2GJG5eNEiMTxYhJubms4+6yx4jj7qKPFPzpxJVpp7Kfsf//hHxOgUwh50Ru4pr5+SYFTQoBUVAmIAX3vNNTfdeKMpgoG2LESeduqpUoXelEhD4TCD7O0lwdWIWhB/yCGH/OxnP9t+++2ViK1MjInIlClT3njjDQ42VojFBJz0cUL2CccfT/qj589/+tOSxYt/dfXVusbjjzxilhBrVr5QoQ3xcvQ+9thjg/TPZn99zTUaFSLRD2zXadMa6uvV1yknnxylP8j5r7322KOPakvy/cWll958001YypB43rnnvjx7NszUOZjXFiyYNm0aQ6iMjALUHeIpIdIfbVaw0Ob4LO1ERvvtuy8wfPAoNQ6Aj/rA14/k8+9QAH2MU1V46qc60P0xV5gnTsJqkkUNBicFQ2R1Y8QMGBiDj2G05isSBqliQgM0AfWU7wp20sTbpyQMtsPZQT3ZSuZIJ/p0ZEt7MyWWAoxu2vmnkOzOHip2+JtTpF1JHPwPHQwXZnzJE5aWCOhcZ3ubSebyZX3E/38GzKRNRsvKVphc4WxvaX5j4YLZs15cOH9+U+MyGSpLeRkbiE6ryGvm0WOxQi/yxhb5CpMmsVTbbLONlq2ksVnrgaubK5yYLKG33qUimHS8PTKCTV/xNWYNgFT1SY6+Gsl+6lOfIguB3XLLLXpX4HmiG3wlPggghB122GGGq5FyvS5ik51HOMKrLQDCkX7ZhawTyzUz7jePOMJXuVMG7B56tcEy2QRGPIdI8wzjZclDwgSJeCYaayQRIbXBWOGhNmIMKwGECPAzli4mVwS2eSKPZPRo0hhy5ZVXIgakYiqasCSSk79+SugR6Q1GvJZA0jHW+8kUg2bE0Km//vWvA4WJ3QknY9hPsyiWHMpy442Hm3bQOvysUAIGA1UxSPgj28F7IhkCvgZO9fYefvjh1k7kThcS8T6hxxRKKsnhMYQ3RlYF22677eTJkwF4xo4dy9qzwQYbxJ/oFDj11FOZU8Dvs88+kUiaHktfeeUVNj0AvG722GN33DO0p0vkxMJjRAJG2WWhMZD+kPAaePShhyQ5/oQTAItnHzvnnHNYeFii8Nl0xCwNwMEHH8x+hQD2MYzi74t+hYqo4FdG9X7mmWea9WoVhinahJajaErt69VXXw2P5D/4wdlSCRg6iKHSUI484VC/SVMXFvD+SD6rLQhWlwvqBt9jKk1EreMmy13/itLWjs7imgHd5R3LOjPr11dmulqKw04vewR4LcQkunq23NE9mcZ+3V0d2Yp8cshlZbZnowH1F59/4RFHHFda6JAFc0Hw4ix0MesY5epcqlv3CQZbR/5nSow+YDVz5V2hmUNL0sdT/cklSbtLqosKzRWZjvZ3lyxe8LYG193bWlrRnVnQb4PhmzYM3qijx8JRj90JZZx8jDKLK1bJiuLSMhYM25stRWW72ns6lr/5j6dzrcvrKyuWd+SWl9YM2WjUoKHDe0sYN0pNTorKgw7+51sY+1dgW2cwAZuhF1mLyxbf+9e/VdXVtra0bD5mCzKC4UKXq1QFyRkXq6T/gyL1WERGOoWBQejd0ZkvLinDVvOPYnpUgYpKunOMK0HPkT6OJiVu3nn7beP0004/fbfP7MUix05mfQedUBFkoQJKsl/7xmHLW5slgVY8mwlzVqnpHclrtTFxGSSJhcSEAXMyHREW1AbC4fUF57yqYxXb25HrDDc59xgFdGoB2p6HECRuovCSKvCqBNPyTz31JA0m4QvPPP3ic89KWJItJT6IWssJP/jhOQFtaTHpqEXCI0+lo2NNLI2CUetWBlaLLcaMUUzPY4xCxdkcRgfnhuBWVlya7eoOtvV8IcejBUz4WVY8+5WXzDuVa+KkCT71lha15dpHjByOxwrVkWsvr6oor62cNm0KPXHvvTMeeOA+0w6ieerUqbvuuusmo7dApBsgIFSc0BfCvHkFkca8+CZT9e4rsmGlCFs62rGroqh32IjhRSxFFsPec8eER5JYC0pqXIV7kZPC/mTQ2tYa5smZ7MD11s8xevK/6tc/4A4dN19bV/P3p58M/s1FmaXL373ht79Bj/NZWNWwyORx4Vtvjhw5Mhz/UhqcktVfdWnJ24veQbeZ/K6Tp3RpVaFz9nzlsENUlirD7b/cO8PAzSjx83t/rrm1ic9UW0crVuyz394/v+QSWc9f8FrQdkm+u0zZub5fnV5clikdNfxjW44Z/cILLxjZhYbQ1f30zCfwfINhQ5kuI39ABmNcb+/LL80eM3oLxmaMkhtXY+8QSFpdbHJ4pfErFL2usyu+sE8EnYAnahokffiff7kC+CAWfGbvfa/93Z01yVkLby1eOmbgoK7OFgPIZLfXKhLVlJZ2FniwZPIuOuFS051rXLb0mquvO+iQfTVuCWjyWCuqSpWsAsUHRKmw8KXQ9c7CBWU9zUtee623s5vosn7cWWjuaGpcOH9ed7asbtAwGkXdF3IdFWUVQb+s6rHAUF5eoi20tjaVZ/IvPfdMaXeOTnJsdWVldUtn58IF89pzXRtvvqUBZGmZ3QFBmvzzjzaXjFLDzhrah9/DvHnzrM7VWgbo6bESSPaB0UwBaLiOQf3nM4UhihUB1gzIBWIu4jHWeErWMd4g68ADDwzjUHo02YNjH1bD4DoLejregIEDrQ2ABKAgkECl2+cV5z3rtp9R+sOslrlLRjA/iaogWxOdFASh/RfJfZkhSYJKkwAAzCfvSFIQcIkHjjkKacVV4JBDDumiSGA0UjDzMIuaNy+WyMY0SIThlEpaiR1ASEIZM5ZUhH0J5gHG/KZg1B4K0aBoCuJRLimk9YiMNEMYxptERqIvGc2KywMekHL3AGhpbfG+5pprDGl/9atfQWUszHrj4Wh74cU/+8Y3vmG6TA3IThIp9QWZUUVxWCBrn0K+hHQirYSppcgln1ioAonJ1xjwRgMiPcij+QAzK4k01/TVT6wQXvzuEhyANiYMw/B8DjEBpqTEAiyGwMCqKiZRSF0mPZtuumku8UmT0EPER5uVjPAQe9nPNA+SN9fRiVri2C1S9AcTqtYLlZKqUKzYZJNNAv5Mxpp5mKAIcz7ecEM4EQm54ptUUQDKooVArmqqaqr/evfdFopDBSXJQyCT0WusmZnHxCl1woAg0NEQ2SgGE2SNb7AFTZF42YqRF8yRPyLXiWetKYAdJu38qxtuKXR1VFdVLm9pau3KDiiv6m5tz6x6YJ0ZNKj2jfnLumsaCpmylkJ+YG1tWT531tknfOYLu6kJNY3dsctFTbC63O/ubH3y8YeHD65yllBFpW0BPR3tLcVlPfXlJa2ty5cvfqe2bmC2NLg/2sLcSUSERYVVPM62a21pqiwve3fRm+8ufK0iy421kCnkurWifFtFRaUh0uJ3Xq/vX19cUVtT29BtmrImHkzQNPX+0LdDCy8wa2jreshBX/uaryb1kVFaqga9JvJcgQM2skaHlCkaIn5GrpdfftnoGBCmXX3NNawEMYGFj9g5dXUOfwyy4ukGaWEALKAvQYh+y/u6WfCRT6z/IH2VYxRAwmJAKprI8srQeggRwLKIn3wNkYlxA1pJxAQM+JD4ifOsj14JWDRz5kxImAJqqmoDDSY63QVymY3CxRVEAJLQw0fIV9KUMmM80edbch2ElOJEAWSpWUXINz5SRbLly24jMmoyVMk6lKVQsHuZ6Mn3dBNwy5cujaQqRShabw8zCw8lNut7773XYjb7J+8aWXzn2GPZwSuqKuN8hSym2uWijEHyJgWHQQDxIV++dgkDA0AACUsgmCgsRz8FgMVPIMWE+MQ3H1c9ypJYVJycGHSA7gY+VD1u9PZinSSI8VOhfnDOOfx0NcLK8mAywnPuag39+0GCEoR5sLeEp3ZNjeRg8FAkOW3122IM5gQWlZZ+bMNhYUKfDdaqOC2QEWOdpZeYMPqholnWTz31lEoJCjU0pSx9DLMHtb4L4MbBhxzChEUbKaz6FS+ypq6B6Re3lUjWPikjDAiWVlim4mEAIB56AQRHtL6CxCWRIb8P/bMaI+V/viyxbUU8W44baxeH6sAvx7At7SjuylRUhSP9V/0Mq67sny2qzpayM5SWVzZ3tFXVVSx5d5H1N/YkVdLHd91YNawaSxIbqyoC9IU7WhrfeeM1Y/eino7uQgc7VLnhfljB7SrP9jQvfvupmY+FvSWaRUk5f9UPws8OVVZSXF5avHjRW60tzWEPs6EKRAwIrrZ0+2RGy+55ZfYLc1+dxWLyQXhWN14TjN0eK2L7Y7zGGe2RQ2SQaDU12KLJrtmmKS8dI8ia99YGDOXUhYfDpeEtKaCbsQK9/fbbyNB7RZJxOI8YW3mjKOQtinLyLg4nNRUlghyYB+WxdISOT+IjgE9SefsKPnbIyDrC0SMs0hsMSVdZEdY2/fSO+UZSrZGIhKEzn7N+YJ2A+GDQt8ArFXFANOAbMlAObSQmoM1mtUA/kcTc3CeM+DLBLEkoZHJjHRhVoHRcYyX0xEar4JESlm4DT2Di7ayWVywpzAsXLrzplpvP+v7Z8+a/dsBBB5586in3PXD/cd/5drDPZDKvvfYazYF1aPZWUo9U8vUVEpGR8pBpNjSP8DVZJRIvO/HhU8JbAcnFe0deYb1IBQcQmdkX7zhekCIhBA8Mc4CFyVwyBdEZretYjOGPa13E1nTFhwqrAXujTQBCY3YUCPPWjVkDswaw+WabkfK0Y7gGKmnbVgK0MZlSM8p+zdVXYzInHxUXz/+AxNAe0zCcOp/59ydnzX7JNngmHhwz9h+7wziF4nRL7jOIWdm2WmayYkE7skUTRacRTDANmbiUlJhe2BEtMhKs1IBRRRlHxRCbh+TKouqlWieeDxS4a4r62Fbej23YRsPD6KQrGIRbCsWLWrvyxRWdHyy3+2WbNh1SVtz6VnV3c3m+uTpTYAxeb2DDT3/6U6MhLSb2HBXQp+Tfn6mY2Ezf/6lp2dKWxuWV5TpIOKA0l2fQYIUuyrW3mZ9XlRXPfemFjuZGV0jKyL2S78cQY+CvCgaWwruLFjP7kPyW6DDBMrRNi92mAvlW9qvqsuyzT83syodB2Rp5tLnYkzVQ2bG9aN+R+db6fCLIMCcWX3iNZApJKFoc7ECdiAwyjjzS2//xwgtRtBFnRsS88r2tOlq4kxANhIWlv1AlibdiEJdJPepduhw+Q+4B3CeP4s/Y2bxJkAgGHkwsO3gDeRaSRx99FCVwykJCkbfdfptILQRDZJrknDXG57kvFeng1AQzp4ceecTqpcXD4Ekyf36kB4BUeOgtUz+hVQSjS9ub+aSZyvBOCfGZzAEHHADGwYVhjN3T8+gjj/C+Jyz4+Xz/7LMrbINIJGCgLZM5+/vfRwyXVvw566yzLH6eesopsaTekpNHKPnJhRc6dUO5GFgeeugh9r2gdzMZ4hWY0W4sDgpl3SeDcIYkBRZoxl4OasnoNXIY8khDTOtnbCQx4M1hj8aQRHaWQ5yAtELf06xhS01QiqoST9DgiclptT332kuk1dorrriC+QUz7SSw+08gZq3uECk5MPiHDx8eV6TtNuB8pbL44AbzYCZDebDzUMOqiVHrzjvusGeQLuEBpb4A4IPhDrYoo+SEO0p8+tvf/nb77beHqWcyaIgF0UgYzaR64fnnbYCg42kLgxWnjJhLiUeY7vOJnXaaMH68E0OCFikvt/9U7UyaMIGNCK+w1JaIsdtvr3P5qRkkpQ9tVXFiiYQ//M+/XAFgAdb0MaIvbPPLFz73+eVLlvDsK6vrN2vB4nxxeVf2AwVrVaZ15JDyUQOLKzverS1qyxTsCsm0ZcqJG61BS4otL+Lva/19+a4yEJt+/GR7TVchlwjrbmtxVr1sIc5kS6uqa7Wqnu6u4cOG/mH6Le4vI7UdHbFKhCJzDhrp7nXFdlOLPTeOHcnzCUIbj0+epVRKrc7v9u2u3OABA6Nr/AehWq14HcmjRJqv1qndy1RbZ//Rc7DFVwgTqCBeVwv5/wCMOZ4w2EyyEI4Z8faLqYg2Pos6m/erc+a8s2ABUYU2DwAjr9A+eno4qhJz5LXkBIoaRDwACHUnILFTRZEXk4vU5yMewB7AXD+JObqHGDVUBGAaRHzgzD0zZvDz09uhkpeYgCf4ambXG7r+FVddJYYUJrun7LzzbbfdJvdvHH64bW6hgInzqxi0xYQiEWflQ/Oj0nbZeWd7vha8/jqkP7noolgcpbjiyivBI4kH50477njMf//3Kaee6sRWgowCg1Z5+Saxk0RuXHLRRQ/ef/9ll18Og6/xIQGvv+EGANjo7DkW7d132+3+++7D9htuvFH9GhF7Q6VoksQqIAepGVySUDz+qPfSZOwiHNUnCgXCKu9KSyOSxHyVERmhpJnMFZdfTqQaBYMnSUMuSby8oMXPWDUoEWO8fMEFF1AeUBGUZlTHH3+8hArCzwo24ZiL5greQ4Zy4Bm1ySYosfdtzz339BYP+Nprr0WwgEa1m/rNZCiGL3/5y/vsvTevUMRc8vOfT548GZ2Kw9nXtgDIOcsS6GpT7Z9+xhlqCkAklSI5MPENM9uYOnUqlpp7QXvDDTeYQygCYvz04KrJH7QSxnpEOaoUX9PSnZSEegATCyLf+MTkH/538fEnnfRvoDLw7r0nhrs6cr2F/B1/uLOiqi7YUvPttaVd/WtMKVdtO6vm5F9U2b9uAPN6Z2tzR76QL6/uLK2pLi6aP3euZkI/x0asegT0Qxm9/3mPivCvr30/58+Z+7cZfxw/bsty43RXEBRX2I3sSOjufKddxDYh1tTU/+LyKz73+X2qa2pznTlOLH1pVw4UrMcZJJaUPPzwA9Z+BzTU1VRYnc45Jc41ZGQb/1b9LThAlFU98OAjO06asnLy/3U4lpdY0f4alzd+cf/9CT7WzGuvvy4qAPFRNGCOh33hf53XygmtWOowdoFO/MQnJkyYYE6N/yb+ivyJSZNETplCnO68y+TJAv4fv/POW3/84+Bpqfa2Nq7iBrA7TZgwbsfxZvTBHGz57z1Tg4zQGYRRYpQgYupq63R1PjAD1xtiI5Jp1qiRI9lbxu+4IyFOb0yZNAkSfdL2H/tvbbCCTSfv7OjkpzRx0iTAViDRvMO4cdtsvc2OkyZa74F/0002xTS8Kq+sHDBo0O677nbSSSfZuepMAmfPEEDGBHion5MjmhdBUF/fsMO4HYwi9957b4u3TY1NPNaNcOkeAtGqK3E7ZvRotgvAgwYPxhCu9KxhkKDNMDm6MFJ7zHRSIWzq7rtddNFPx++www/P+aFMNx89Otz0mclssfnoffbdl5VJq2XsVl4iErYJEz9BElVWVTGvYZcmnUumd3SmHwrOGsM2goWElIQTd5rIVWn4xhtjLOCmxsbx43aYOGHC+Ik7gYHBWw2qBYUVaKirU3GNTU38bZhKDNJhw4Hddt19wsSJ/uTiJ1Cu+nb/jd5icy6YhmU4SbEx36OZbWrMFlvyIv3+D34weMiQzlzYp4afEqIKsP6ARf0a+jksZLNNNyV/7eveZttt991vv6t+/auhlnaz5i49ljq+8PnPTZk6FQM9JDs5TivwhsIrMdCi3Kic6wHlR98Y7JvMaQ9KzdCnLIFFuRzuaZPKi/mOk6LmKV3samltkzuWDhoyRHPlIcqEpUWJ4bmkBrUfSTAH5X5uO3as7CLfFAf3vFGCdSI//E/R4sbGfymVOAU/pq+ci59m0a2tzRsNH1HbMMjH6u6mkdW5nUb1q8m1rgzZF2aL7yopLWRrunqKOws9C95tm7Uk11Jc01kcjivR+s3cQz9PVroM04zB+9KuHIj0xJiVqXpwxozjv33o9b86vxfW3jK7UG1QcF5XZYY5x07jiu5s+W133rXztD0nT9vLAXZa5Mpo/2+4JHgL1FZXX3LRBQ/fc9d5Pzi1ZfGbbD5FvDs628uzvTYIBC+Pstru0tqvH33cg08+L+3KVP1fVKsTsg8WkrDtzLz7x+f+6Ps/0Os0zZun36pLkEdxUINXGqj+EHwT18RTFPYd9+rt8lULmj6s+O/ehTgyigJFvEi9ggOQhUhkgMdZLrj6vxGW438BoC2UIpmgCIDhKyIAWUhb6K6rrolKrrU7bzrFxOvkHxlpYY1tLWWVFSz9CJAQNkVWxQgIqIq4XRYcKCo7nzCHtKC9zCV5o7LQVTvNLUnS3NqKgIqS4FSGm7xX4pYUASTADCc+wymhU89WDC2rKjrbOxwTTbjE0jktkxOqTSewKaCi0XmSCwsgsm+keeihhw4fPpysIciGDB1q21dLU9OIjw0H861j/vuHP/5xcGlFZLRjJCcUwaB0KHTaLd62dQSdJ6PApWQDjZVZMYiJuUMleZBc2FmU6eR9kUhhN0ZUlZXzt2nlgfoe5+GAH3IB43wZ+aRV4yT6YymcxmLuwBlXGSF34ko4f031WUPrCJ7Z8pUqljcEwolYYVjmKB52eYSJkBAYzE7mCoHkFDn9Cq863Nqd7P2WFhm+4qqwfXHoifwUozZ9BS9H75BFMgk2BKGxUBITxk/C4sEoiLAAhKHtJYMMMQG+LKxOgY9S3qcoUmQKPtLgLVNNCP2cFAyzfIIH2pi7eADeH/7nX64APogFnbnm6tp+p5x+1pVX/LJWb+tuL8t3bLfliK2Kmc477S3Jd3ZgNHEZq8cQkIizp1b7yOr2xVXvNOffXNIyp7FiSWdRWb8hi5YteuHZx0cM32R5e8FQIZtvCnWsUeoRyamBJIXm0u4GgmR5LdYW5OpM/d094+4vfW7vP0+/akBFODvCQT7Ogi4qqc5ZEObTb8xeVDL/zXd/d+e910//8/KOQnG+BX5ItBVlRKqO4R2Pjqivqf/5T39yxsnffeieOzuaFjpRoqiUc04+bA8rzjrY2PFExaVVV/36mo222YVhl+h0ZhFhR1DAqa/zQ0UbwYQ8y3nBeyOxGBiTWvg67/zzufbHPux8f8mYrWqqqp2h2t7cYrQoHwc//G76rZN33W3lKoBy5Z9peG1xQJuMD8nCSM1aTewd+NWvChNDrNu/ue46ALf94Q+GNUFcJu6Ga4vaNN+PKgfCpG+tPIYehlH77r1PT3t7GPZlSowWXnntra6iyqrqhnx7W3lvob64t7Srs9rmK8bW5IRbcwXz1OAz3NJYVVq02Yj1t9tk0Ohh/dqbFjm54elnnmvvbOOax/lSoYhOb+KepOY857CnBx54IEa+v8jAjB6WL29yhii/5WxxmE4aqZKkhidGQ7wLHU11x513z355VllyW4UkSSqAYQADJgAnW3JYIw2OaIZX580tK6/kxGacVFpaTgfkjXccYSvQ0brtVlueeeaZZpcweBvSwoC2quSMzKicvGGmXQyILKYxU1r0tt6oIH0PnRGHRZXl5dbcmlvC+YssD1MSL8y+wqbSv48Vaz0Qxx9GDCqFl5TxufZ6w7XXfu4zn2G2/o1z8TKZ3dkp9twTjNa11glOCfhIcmCtNaxy54a3te+4/biJu0x7/NGHq+trWVqWt3W9vHDZqA0G1pVXlhdae5l77J80l3Z6c1GhKOzhDaeYmVYbEJWHXZ+54ZWd2UzVOy1lTZ0dWzP4ltpZ253rsFEwHC4Y9uQk0/ZwqMvTT3PYeHhmONk81uXKAtGkkB56+ZV5Gw8dn+ntZFFhcDClYzewxdONXvA42XfHcWMuOv+8n172q7JERosknSHURYUhLORsXAo7YMl0nXrWrNkbbzQp18k9v5RJgBAXa8Bu6sxIsvWWm9uG+sC9931yrz3LkxE/z3PnAxDfBAQizX7hMfM2K6IVeKTISy6yW1kuCDM482fnN83JZOCAgUuWvnvq6ac5koK79Uey4a7rhYoTR2YEdW0xYO68eZadrZNzmVfFlnkZhaL/LtUexh8fZHJc1xmR0r9WObDWFAD7b3lppVMKLrv0Uut1HXkH/tgM0vDy4jfrB1ZkSx1mYOM33vS0OAO2jDjPWh5mNeQdziro6LaevEWBsvLevM31nS5CKSndeORmbU2NbtOoKC5i0dSRjMzJZTNoXSjamkWuzPCoA7zXGzTYVvbrfnvTJ6dO6nYctTs32F1YdJzsYxs7gzW7UG/PppuM+vV11x/z7RM323QkoRyH3kHuJ/YZP80X2Gp076rKaptLH3jwoc/sNc0ppshgIGSzpUmAmdCT11VVNVttteX++32xuaUJgAMrQoGTk/oT/cFZKNjKKQxGcJRzWiAOFIdxHzwtKOuoJPzbr77BTZAwKCzxseOECcFu8J4CALBywdPw2uVAnDLGJqQ9sDgfcsghAupajavrCCCsNkPj0RzTJ+XAmubAWjMBkUuateKMGPExy/TNy5vIs5b2rubSyqdff9sBKG0llc4ztOXWmf7EqwcwVyCraYQ/aWZLWGnWID3sUOeENnHnKRZX3fKV6eGSGLpQ5JVc+HgZ+xt9c4wRH5+VOUk4VlZU22f45ltL58xbWFJW5UifcEKcJcAEj04o0NHe+vEtRuuIRxx2CJyGb3qpHuuBHABvAQI8nEaSychL+R5/8sVljS2ZbNj9n0h2tiWXlpQ4tdTpMJ3dRbtM2tnYjpOcPUKEO3M/gi05KC8tIgtv8d52fvJ9lgvPFq4dJhkgUe4TwcF8ZOfLmaef4c4TKwrnXXA+q9GyprDCD8azcnnT8FrngBpRlR6UqGsBb/rAz6jyter4VRtTxWud4JSAjyQH1poCsCjKl8ZYp6215dhjjt5hwg52mzP8cBFYnM88PufNppLa3rqBXcUVFkJrbbR2b7vxbLbIbY9d7CR2bDktpNDdVly3tL2Iqthx/ITgzpHc8mLpVJ+J3YYMZV43TLbFfPz48X21GHtXIhuDcOTs5V1TV/e76X/s6i1ybLttM+Sq3ghGJ7QS4fS4cWO3wbJnnp7JaY9HuSxMLMhfstjAPFh4siscG2rr6yxXG7fd9+BDNmIGpWQbPNeIolIzmbZcprVQ0tlbzjWNcL7isittRXn673+HikOFFUIIPbKOo0LiPtr9yQV+hzwcaJ1YBG+Q1lTOSw7a9VMxeUAzXtEcMPQVOQ18eDigWlWN1iUQ39qGR/XFeJGqlVbwaFcfHspTSj5KHFhrCoDHVra81Ig98bnKXXfNVS0ty/Jd7YXeuvZM/Vsdlfe/uHjWksK7PeX58vLWro6wkTtTYNMoqyrPlpWYFftzONviTO2iziI7r7cfu607VRlFTCt6y8INsYQmAR08KN6bDQwfPly472esSP3NM2BAPxYb3p1333P/G2++HWYAuiJHvuQywt7uRNo6TLyy8pTvHecIOzuY9t1nH3L27rvv1kV1XTsGIYQqWu/DuVS6dmnJ7+/4U2lZpd3F5gbWgd0F0FHIlNcP3OTj47aZOGXa1KnbbbetKzT++pe7p02bduCXvmyvDcHNJoB+mOEkC3j+OHKAt6VIOiOZdYTt5oiMJX3pxRd/9curBg0YuLxx+Sf33MMRCyYwH+QLGwuevtciB2LNai1R0Gtm6tGDJPOA2ERj5QKIQ5m1SG2a9UeVA2tNAbgAxgKtEzEZTHq7c+uvN/Dxxx9uaWrszC0trShu7802ZmqfXdD+3Ou5lxf2tmc35AOqe+gS4XZ1Qq+0IltWzu6/rLvk7VZn8Bdvvc2W+c5mqNiWDN71qNiLjJ4YcNSf5LHXrbIu8/nC5/f5gnXg2tq6G357Y+InHOYdUkHlyec6dEVrdJ/ac48NNhgsnqM7ObvvF75g/4vNinJJgINfuR5rG2RtfX1lddWCBYvfWLhQDCTtHR38Osdsvd0243aq6Teoo7fEOSc/POcctiMzFft07LafuOOOLvPjthTHfUphknHVVVfBAMzw36kp9I1IxZEjZeDNiMTmA2D99dZ3nMC48ePZlB28s8rCppEfBg6oL5WrVahZAfVIHyBM5caAeI8YWv/DQHBKw0ePA2tNAVicdT54qTPSSiq6i8ryvSUf337Hvz3w0NAhA5cvXcSH0iJsW1ndq21ljy0tvX12+yNvZ55eWjRredHi7ppFnZXv5Cveylcu6Cib+Vpza3fRvnt/duP1hzpszbn0rhO2yzaRxUEHRCmp5vQ0kY5s7Mrx6Qy7eITDFoGSUu98rm3qlN1bWtqqq2rvvf/Jl+e+3u7ocsePcwLqDVZ4u1eonp5soayi98yzvte8fKkT39jshwwe9NrLLx/zzSM/t9dey956q7u41OoES35DfdU2W45uXNpSWVl7773PMusXl9V2lQ4csdW0qsFbdhbV50IhO3M9RRMnT/vuSacsb2xub+t0zdOQwUMeue+BvT/1qROOOcYxRHa+zJ3z8i8u+3l1bVVtfc0Rx/13SW2VbTutjp8sLbF1gOZwTktXb0+pKweaG99essihV0MGDLzzlumXnvcTkwZCRNk9uBHZQpN99JryulUiu7GcyG8G7C94+CR/wmHTmR1nSbz5LDunLXtxs9W6VcCU2nWCA2tNARBGpGo0ZRjvMNTYpGcz5IwZMw4/8sjWphbG7iCzmH742PR0P97c/+Fl/e5aUDx9dv6OV3tuf6Xwhzndt7/S3dzi7M9ex+Ra+QSP6QZQ5Ls35CSd0ZOMYmVQBn0/BUhGYN5BL2Qy4dgAF84ke1N/evGlpWV1Dlc2ySC6e4pKO/KuEOOeVJXrKZ40Zdqf77oLWusKTo/Rk3lePvb4Y9894QS3fkOIZvi58SHAQO83N/52WXtXRyE7esttqmoaOpM7bykeDz4oqUOynHzSuHx5RKi8dsw7Yebqq69mC4pHZtrN61gVe+sdtBK5520eYObB5Z8ToYPVHps506GS7l1BlXUCxxI46V4ZPVH0I0zgf5gJrROtNiUy5UDKgTXCgbWmAMgggpJ8JEaJJILMT4PZhob+F5x34R1//NN2241dtqyxta2jqLjUX6Gyf7tzIMr7FSr6NxdVdZbU5csaWnodM1558623brrZaOMo2Ai47nAD64rzmwhHFlXiD7NI0nnz5vkZx8JRW4iXL1VhJy3TypYf36qttb22rv+cOW9dc80tFRWD3S5ZyBa3dBZKqvuNGL3NNuM/MXqrcQ0Dh+00ceLsV14x9HapkINEmltbHFNjnVaOyCCa4fzil79kJZYK4o9/7iVXuf6sbuDQ4OdfXiFTWSMWQBj09fYe+o2vsxS5SYrcHzFiBK0Aj/Mj4XE2YTD2sx2NGcOVyI3b+Y5OATvpmpYtd2x6e0srNjpLZ4sttrAscfLJJztwhkYU6by5RP4Hd9jYYuCJgfSdciDlwH84B9aaAjD4jaNR8igOzMWEUX+hm9OOLaxcHh946MHd9/ikIw2WLl/WtvDV9rfnVXQuq+5qHFDcXtfTVNTyTrb13Vun3/7JPfZyjLu7A8OaQm+w8ziphYSFWRYq2Fh4yHrrkYZ22LOSAyATSWoxwOQbkhQVWYl1LUlzWwsjUkV1/W9uvOPZ5+bW1A9q7ejtt/5Gm2+9Q82ADTq6Slq7evMsQckezhEjR3IHuv43N3D6dM442ws8MJO25jQN/ft/5WtfdWVSfb+GJ194dZddP/3MP16urA7n8gdFlSxQoyRwINmxxqRDgh9w4IGXX3GFU9IcMkMjArO9C5gk1hWkBR+XB3yy8iyeqpCpQFw/FL///vv7KTLuHZMKwRLG5q68/+HtPi1+yoGUAziw1gQBYUQMEVIEkwDJRWwlY2cjZAdRFTrzhdFjtrrhtze98NLs2+/843k/ufCoo7+54bCNXFHq/DGeoM2t+R+d+8Ntx+2Qs1/LOJ1ETg7eKjgAKxGRYgg+OE0vHFQpL/YQ9zFFOUj6+ypr+YIkYWmRA75ykDMIFy1ZXFldV1U34NjvnvzgozP7DR664fDRRRV1nc7BraqnYugMI3cmWhgam5sZebYfN9aFJ04ctJXXHwHtq6yPOe44Vwy253PFlQ3F5dWf3GNP549z9ucUhAwnYYcjxsL5FlkH+0DLOuRsKSN9Q/6Qvri4uqqaHHdKkGmCFeOwZlLmmlhnmZWE82Tshi6Ewyk95goKpSDUhqPxlFoFMxnJCAYAfiLYGyu80yflQMqB/3AOrLWdwEQSweQhjMgsUlg4SmSBKMvIZaaVDYZtOGDQwAnT9rDd64c1lU/PnDn5E5MaBg4ory87/NiT8p0OdueuH7ypw6C6p6usnLmj0N0VrPBELCR85LlO/v7228lTJvLPfvaz9AGpDdyEgLBOpKcLu1lRKs7/yQV77Lb7sublNtbWNvQ/+ZTzzzi77JjtJnb1ZEtc5RU0StZZE9LCYMriUEHnRzpliMfn3vvuo1xRtSiXMxpHbjLq4l9cesxRR1XVFlngZbrZferU226/9ZN77NHmeLssR88VF+QGYe/6w84ctCxLhDTvEKS2d7QT4vSB5VyuPoMGDvHTKe2Y5qBdwOY35gHlNRX0Cr69tXChWyzcwWK6oHGPHTsWJciO053IYcTL7T+86afFTzmQcmCtKQACiBhSAUSSdxB/yZkHHC+DA7tz0+wV5jhTWtLY3GpI2+Gw35JSFxzOm/MKreECr++eeAL7vbG/5A5eD3vAkllFGAyHHWNhRFxbFS4YceqvsxPoA6fV//Lyy+kAZ6185Stfcc6iZWfy0adohDEM33bbrZ74+2MHHfTVl2bNQlz/2qpTTz/n4cefO/n0M8Z8fCt7wYJfRpgFVJi1kL9ou+3W6Sw2jp3YddddyWWrtd6Kw82nPdf5lYO/VlNfd+gBB7U3Zcjouurqz332M4cd/o3vnnTikA027HWqbjIFQQPlUVJWauTuxLcwA8hmres6KcwRMXEPszMjnTBHpUXWKbgie9BQWV9tTgCDk4QV3F1UPFadwO7kfVoKKmDgA2veewukT8qBlAP/yRxYa8dBk0RxHBplmZ9xcFrMOdPCLNfL5BTPDofnuOWRCyYjD9tOpnfWC89PnTalvr6usqb219dcs/W48bbOgnGfl+VQYs6RzJ4ly1znufzthW/OmTPn9dfmP/LgQ3NeeYVYXLxsqRxpgneXLJHR9mPHukLI/GCH8TsZ8buhxEHrBCih+cxTz953331/+f3vZ70y1y5eyuATU6fs+/n/GrfdVkPXHzRoo01Y+R3Rw0XHcQ7WmnfbbTdn9bBHQUv6E9NsO1Gsu8iiY8nSZ/7+1F/u+tMN117X0eWMIeefFu970IEuHmHwWW9IGNezHcn37rv+Yimif4ONxBlFoA/MUUhzViA6gQKAk9LyEPtKSgmBzPV2UZAirWh3OJGiN3PwwQe7K6O0uhpOGPoUbeRznwqRNn1SDqQc+M/kwFpTAKvLbj77RCqR5z1y5EjCl5BtbWr62EYfc5acS4gMug3JHdHjSJw33nijzSU+TU1GyDJi/QDMTrJk8eI99tyTSpj76qviG/r1Iw2tObO3kL+2X7m7zi1CjCfB2p7cWdrVnbds8OSTT7oh2hKCW6F5YUpC5bDDuOf29NNOI5d5fLrrLphZEukMOVm8chnpKDR4OOfY1utgZ2sG3Htaljca5kMoR8dRxCzYdpwZJDkFYIGivqHBnVPWdWW6tHG5RQLFtMmLzygkyusntlAGtIXdZK5DAs+rVaS1ipXJSMMpB1IOpBzo48A6owDC6aHJgWuEmsMSpuyyCxHr3rmKkuDOz+oi3vCfCDYiJmdNIMR7aAUmcNLX8N9pCg5YN5rmHOlstR/96EeAuOMHzMkNTaQ2ub/xxhszDbkAz51zm26+iRE6zFQFCSsXyK0cQGJkDV6Y/2XUQACc+ywScFQAEoL3tsoRmY5CX0l2usRN4o889DA14GCJSCotpSymIOYP4L98wAGWAVDiKgI6T1r2Mck9lEGkSgHl6402xFMe4ldoL5uEk3uX+uo7DaQcSDmQcqCPA+uMAuD3QmiS1EQb2cosPn369Jtuumn+q3MVxhjcm9yNBSPuSdLhyTV7xukEultko7mfbCUf7QkgQw3Gn3rqKeu3PHMefOCBmLYmOW+HMGXWpxscQUH4jho1yglrVIiErOosSKxSxuySELjesqMJ6J6OXNjQLzmJbGubob1cyO5lixcZrZs9cO0PTd5ynwAABO9JREFUUxD3GCfUKo4JBwBP1CimIA4F2n7sDmxKFnih8sgCWk9XT9i2Fh8FifHe9AdNA6cAnGKEAfCMSp+UAykHUg6skgPrjALg6RgFHAlOxhl0E5eKZLzsqOfFixcLGyBzlXHRti1dfpKDFAbpyXBEFhsmC4iPOgA2YYYjUhUqYtpNWzNmzLj99tsTGzqf/l4eqXVVleBBwtAncAWMrKWFUC7kPnEvL2CcOkleSWTnLXfUBsn93mX3lRWV9AdxDyFISx5ktmOFWJNsKHPlNH3DktOTCXfkwheSJvMJpYNK1vHpiwcjxptWAClVVG8ARK6pu3+hSp+UAykHPmIcWGcUgDUAEs1D2JG8UaoG+VkShr1EsE8i46c4ZFZVgIlF8cJ9X8GIByMgPiYEFgfOBvLUCbMME42h+j133WUNoK/WJSSIQUY3fz8DDXH+EQWu813ek8gROQCBXEfYgBYJk8rQniuq1YvRW40xU2FB4ucTipeIe2CWnSMwReWJSggGfkQRzPv/yQiYMsa3T7GA6QwAK9In5UDKgVVyYJ1RAOHstkSUexOg3mSiNwHnJ6lH3hG1AiRjkPgO5k8mAUbEfsbhuSTG7MA8wCSPQlaAB06MARzxh0hj6nyOGYfFxpZai7cWaa0we5sxWAawHmv8LiPTAau1gcXcWJNHDLsQse4xot9o2AaMUc6IJus5HZmpxHmJE9yiKI9zCBTKHXkmHcoiENUSGJMGes4uuEh8H5E+eZDdp88AxHCATE1AsT7Sd8qBlAPv48A6owCYgIg5MjEaRohFsttg3D2NZJ+wT54+8Q0sjvQJyhhJtgIg0zEBvK9gKAZoRbICEa9iwPgEQKRPLEQRQ/xKpAIAZj4SV2tDOHnkIotyN0Em1hhIEAm/R7irK+dnpMQbjEjYJJGRGGBQxXD4VFxM4osEIxK8TFBVaqPxe2N/MeL99EgS0YLvMxYJpwoAi9In5UDKgVVyYJ1RAExAxBmRRxoSlOSd8hCI3dlg64/CPWoF8SCN8ElqklES8SL8JFIFAEDiEYhC0yfTCmHwpCf8MYmfLpiJmUoLLIZDfLjObIXJRWTIMRmt5yBNHjD+9Y5PWXGYbYCMsh7Bwj5F5305miV4i1mRKtEWwugBLBCzy9pdkDBBjCfCxzCACKloYGIWqRdQZE76TjmQcuD9HFhnFMD7SU9jUg6kHEg5kHLgn+HACoP1P4MiTZtyIOVAyoGUA+siB1IFsC7WWkpzyoGUAykH1gAHUgWwBpiYokg5kHIg5cC6yIFUAayLtZbSnHIg5UDKgTXAgVQBrAEmpihSDqQcSDmwLnIgVQDrYq2lNKccSDmQcmANcCBVAGuAiSmKlAMpB1IOrIscSBXAulhrKc0pB1IOpBxYAxxIFcAaYGKKIuVAyoGUA+siB1IFsC7WWkpzyoGUAykH1gAHUgWwBpiYokg5kHIg5cC6yIFUAayLtZbSnHIg5UDKgTXAgVQBrAEmpihSDqQcSDmwLnIgVQDrYq2lNKccSDmQcmANcCBVAGuAiSmKlAMpB1IOrIscSBXAulhrKc0pB1IOpBxYAxxIFcAaYGKKIuVAyoGUA+siB1IFsC7WWkpzyoGUAykH1gAHUgWwBpiYokg5kHIg5cC6yIFUAayLtZbSnHIg5UDKgTXAgVQBrAEmpihSDqQcSDmwLnLg/wDFXTLudBSKHAAAAABJRU5ErkJggg=="
    }
   },
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.052022,
     "end_time": "2020-12-13T16:45:07.016605",
     "exception": false,
     "start_time": "2020-12-13T16:45:06.964583",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "\n",
    "\n",
    "\n",
    "![image.png](attachment:image.png)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "    Bug - 0\n",
    "    Feature - 1\n",
    "    Question - 2\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.053162,
     "end_time": "2020-12-13T16:45:07.122512",
     "exception": false,
     "start_time": "2020-12-13T16:45:07.069350",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Workflow\n",
    "\n",
    "1. Problem Statement.\n",
    "1. Geting Started.\n",
    "1. Train and Test data \n",
    "1. Analyze, identify patterns, and explore the data.\n",
    "1. Model, predict and solve the problem.\n",
    "1. Visualize, report, and present the problem solving steps and final solution.\n",
    "1. Supply or submit the results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.055314,
     "end_time": "2020-12-13T16:45:07.231468",
     "exception": false,
     "start_time": "2020-12-13T16:45:07.176154",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# **Getting Started**"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.050678,
     "end_time": "2020-12-13T16:45:07.333552",
     "exception": false,
     "start_time": "2020-12-13T16:45:07.282874",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Importing the essential libraries**\n",
    "\n",
    "\n",
    "Importing the required libraries at the begining of your notebook helps you run your code seemless . We frequenly get interrupted by the missing library errors or any other imports which can be avoided by importing them early."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:07.447420Z",
     "iopub.status.busy": "2020-12-13T16:45:07.446598Z",
     "iopub.status.idle": "2020-12-13T16:45:16.606924Z",
     "shell.execute_reply": "2020-12-13T16:45:16.607452Z"
    },
    "papermill": {
     "duration": 9.2226,
     "end_time": "2020-12-13T16:45:16.607633",
     "exception": false,
     "start_time": "2020-12-13T16:45:07.385033",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# This Python 3 environment comes with many helpful analytics libraries installed\n",
    "# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n",
    "# For example, here's several helpful packages to load\n",
    "\n",
    "import os\n",
    "import torch\n",
    "import pandas as pd\n",
    "import torch.nn as nn\n",
    "import numpy as np\n",
    "import torch.nn.functional as F\n",
    "from torch.optim import lr_scheduler\n",
    "from sklearn import model_selection\n",
    "from sklearn import metrics\n",
    "import transformers\n",
    "import tokenizers\n",
    "from transformers import AdamW\n",
    "from transformers import get_linear_schedule_with_warmup\n",
    "from tqdm.autonotebook import tqdm\n",
    "from tokenizers import BertWordPieceTokenizer\n",
    "import matplotlib.pyplot as plt\n",
    "import torch.nn as nn\n",
    "import seaborn as sns\n",
    "import transformers\n",
    "from transformers import BertModel, BertTokenizer, AdamW, get_linear_schedule_with_warmup\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.056372,
     "end_time": "2020-12-13T16:45:16.722447",
     "exception": false,
     "start_time": "2020-12-13T16:45:16.666075",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Load The Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:16.843010Z",
     "iopub.status.busy": "2020-12-13T16:45:16.842305Z",
     "iopub.status.idle": "2020-12-13T16:45:16.883307Z",
     "shell.execute_reply": "2020-12-13T16:45:16.882631Z"
    },
    "papermill": {
     "duration": 0.105601,
     "end_time": "2020-12-13T16:45:16.883426",
     "exception": false,
     "start_time": "2020-12-13T16:45:16.777825",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/LICENSE\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/.gitignore\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/README.md\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/MANIFEST.in\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/requirements.txt\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/setup.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/optimization_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/modeling_gpt2_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/tokenization_transfo_xl_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/tokenization_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/tokenization_gpt2_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/modeling_transfo_xl_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/tokenization_openai_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/modeling_openai_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/modeling_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/LICENSE\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/.gitignore\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/README.md\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/MANIFEST.in\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/requirements.txt\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/setup.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/optimization_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/modeling_gpt2_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/tokenization_transfo_xl_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/tokenization_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/tokenization_gpt2_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/modeling_transfo_xl_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/tokenization_openai_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/modeling_openai_test.py\n",
      "/kaggle/input/pytorch-pretrained-bert/pytorch-pretrained-bert/pytorch-pretrained-BERT/tests/modeling_test.py\n",
      "/kaggle/input/bert-base-uncased/config.json\n",
      "/kaggle/input/bert-base-uncased/pytorch_model.bin\n",
      "/kaggle/input/bert-base-uncased/vocab.txt\n",
      "/kaggle/input/github-bugs-prediction/embold_train.json\n",
      "/kaggle/input/github-bugs-prediction/embold_test.json\n",
      "/kaggle/input/github-bugs-prediction/embold_train_extra.json\n",
      "/kaggle/input/github-bugs-prediction/sample submission.csv\n"
     ]
    }
   ],
   "source": [
    "# Input data files are available in the read-only \"../input/\" directory\n",
    "# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n",
    "\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))\n",
    "\n",
    "# You can write up to 5GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n",
    "# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.053424,
     "end_time": "2020-12-13T16:45:16.991244",
     "exception": false,
     "start_time": "2020-12-13T16:45:16.937820",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Let's create a dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:17.105253Z",
     "iopub.status.busy": "2020-12-13T16:45:17.104384Z",
     "iopub.status.idle": "2020-12-13T16:45:21.212931Z",
     "shell.execute_reply": "2020-12-13T16:45:21.211844Z"
    },
    "papermill": {
     "duration": 4.168747,
     "end_time": "2020-12-13T16:45:21.213076",
     "exception": false,
     "start_time": "2020-12-13T16:45:17.044329",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "data_frame = pd.read_json('/kaggle/input/github-bugs-prediction/embold_train.json')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.075882,
     "end_time": "2020-12-13T16:45:21.366670",
     "exception": false,
     "start_time": "2020-12-13T16:45:21.290788",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# **Data Set**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.081687,
     "end_time": "2020-12-13T16:45:21.526227",
     "exception": false,
     "start_time": "2020-12-13T16:45:21.444540",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Currently the title and body is in separate fields. We will combine the title and body to keep them into a single field called text:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.051569,
     "end_time": "2020-12-13T16:45:21.650976",
     "exception": false,
     "start_time": "2020-12-13T16:45:21.599407",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Train Set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:21.768974Z",
     "iopub.status.busy": "2020-12-13T16:45:21.767941Z",
     "iopub.status.idle": "2020-12-13T16:45:23.008890Z",
     "shell.execute_reply": "2020-12-13T16:45:23.009964Z"
    },
    "papermill": {
     "duration": 1.302342,
     "end_time": "2020-12-13T16:45:23.010160",
     "exception": false,
     "start_time": "2020-12-13T16:45:21.707818",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>body</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>y-zoom piano roll</td>\n",
       "      <td>a y-zoom on the piano roll would be useful.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>buggy behavior in selection</td>\n",
       "      <td>! screenshot from 2016-02-23 21 27 40  https:/...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>auto update feature</td>\n",
       "      <td>hi,\\r \\r great job so far, @saenzramiro ! : \\r...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>filter out noisy endpoints in logs</td>\n",
       "      <td>i think we should stop logging requests to:\\r ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>enable pid on / pid off alarm actions for ardu...</td>\n",
       "      <td>expected behavior\\r alarm actions pid on and p...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title  \\\n",
       "0                                  y-zoom piano roll   \n",
       "1                        buggy behavior in selection   \n",
       "2                                auto update feature   \n",
       "3                 filter out noisy endpoints in logs   \n",
       "4  enable pid on / pid off alarm actions for ardu...   \n",
       "\n",
       "                                                body  label  \n",
       "0        a y-zoom on the piano roll would be useful.      1  \n",
       "1  ! screenshot from 2016-02-23 21 27 40  https:/...      0  \n",
       "2  hi,\\r \\r great job so far, @saenzramiro ! : \\r...      1  \n",
       "3  i think we should stop logging requests to:\\r ...      1  \n",
       "4  expected behavior\\r alarm actions pid on and p...      0  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df= pd.read_json(\"/kaggle/input/github-bugs-prediction/embold_train.json\").reset_index(drop=True)\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.091824,
     "end_time": "2020-12-13T16:45:23.184458",
     "exception": false,
     "start_time": "2020-12-13T16:45:23.092634",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Test Set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:23.303923Z",
     "iopub.status.busy": "2020-12-13T16:45:23.303264Z",
     "iopub.status.idle": "2020-12-13T16:45:23.800024Z",
     "shell.execute_reply": "2020-12-13T16:45:23.799353Z"
    },
    "papermill": {
     "duration": 0.558267,
     "end_time": "2020-12-13T16:45:23.800140",
     "exception": false,
     "start_time": "2020-12-13T16:45:23.241873",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>body</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>config question  path-specific environment var...</td>\n",
       "      <td>issue description or question\\r \\r hey @artemg...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>crash indien vol</td>\n",
       "      <td>de simulator crasht als hij vol zit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>unable to mine rocks</td>\n",
       "      <td>sarkasmo starting today, when i hit enter  act...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>not all whitelists are processed</td>\n",
       "      <td>create following rules... order of creation is...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>add ctx menu for idafree 70 and idafree 5</td>\n",
       "      <td>associated with .dll, .dll_, .exe, .exe_, .sc,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title  \\\n",
       "0  config question  path-specific environment var...   \n",
       "1                                   crash indien vol   \n",
       "2                               unable to mine rocks   \n",
       "3                   not all whitelists are processed   \n",
       "4          add ctx menu for idafree 70 and idafree 5   \n",
       "\n",
       "                                                body  \n",
       "0  issue description or question\\r \\r hey @artemg...  \n",
       "1                de simulator crasht als hij vol zit  \n",
       "2  sarkasmo starting today, when i hit enter  act...  \n",
       "3  create following rules... order of creation is...  \n",
       "4  associated with .dll, .dll_, .exe, .exe_, .sc,...  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df= pd.read_json(\"/kaggle/input/github-bugs-prediction/embold_test.json\").reset_index(drop=True)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:23.930209Z",
     "iopub.status.busy": "2020-12-13T16:45:23.929401Z",
     "iopub.status.idle": "2020-12-13T16:45:29.304670Z",
     "shell.execute_reply": "2020-12-13T16:45:29.304094Z"
    },
    "papermill": {
     "duration": 5.443203,
     "end_time": "2020-12-13T16:45:29.304824",
     "exception": false,
     "start_time": "2020-12-13T16:45:23.861621",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>body</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>use a 8bit typeface</td>\n",
       "      <td>since this is meant to emulate some old arcade...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>implement wireless m-bus binding</td>\n",
       "      <td>_from  chris.pa...@googlemail.com  https://cod...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>add multilang support for timeago.js</td>\n",
       "      <td>currently it is only  en . \\r required to add ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>scaleway - seg-fault on shutdown</td>\n",
       "      <td>tbr  irc  creates a new scaleway instance with...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sistema de pintura: no se guardar los nuevos p...</td>\n",
       "      <td>este sp ya estaba asignado a un carro y se enc...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title  \\\n",
       "0                                use a 8bit typeface   \n",
       "1                   implement wireless m-bus binding   \n",
       "2               add multilang support for timeago.js   \n",
       "3                   scaleway - seg-fault on shutdown   \n",
       "4  sistema de pintura: no se guardar los nuevos p...   \n",
       "\n",
       "                                                body  label  \n",
       "0  since this is meant to emulate some old arcade...      1  \n",
       "1  _from  chris.pa...@googlemail.com  https://cod...      1  \n",
       "2  currently it is only  en . \\r required to add ...      1  \n",
       "3  tbr  irc  creates a new scaleway instance with...      0  \n",
       "4  este sp ya estaba asignado a un carro y se enc...      0  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_ex_df= pd.read_json(\"/kaggle/input/github-bugs-prediction/embold_train_extra.json\").reset_index(drop=True)\n",
    "train_ex_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.055158,
     "end_time": "2020-12-13T16:45:29.415635",
     "exception": false,
     "start_time": "2020-12-13T16:45:29.360477",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Looks like both the train and train extra dataframe contains the same content so let's combine them using concatenation command of pandas on index axis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.06492,
     "end_time": "2020-12-13T16:45:29.535141",
     "exception": false,
     "start_time": "2020-12-13T16:45:29.470221",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Checking the Length of Each Data Frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:29.655844Z",
     "iopub.status.busy": "2020-12-13T16:45:29.655136Z",
     "iopub.status.idle": "2020-12-13T16:45:29.663321Z",
     "shell.execute_reply": "2020-12-13T16:45:29.664031Z"
    },
    "papermill": {
     "duration": 0.070834,
     "end_time": "2020-12-13T16:45:29.664196",
     "exception": false,
     "start_time": "2020-12-13T16:45:29.593362",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "150000 150000\n",
      "300000 300000\n",
      "30000 30000\n"
     ]
    }
   ],
   "source": [
    "def dataset_length_check(data_frame):\n",
    "    print(len(data_frame),data_frame.index.shape[-1])\n",
    "                 \n",
    "dataset_length_check(train_df)\n",
    "dataset_length_check(train_ex_df)\n",
    "dataset_length_check(test_df)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:29.790645Z",
     "iopub.status.busy": "2020-12-13T16:45:29.789753Z",
     "iopub.status.idle": "2020-12-13T16:45:29.833782Z",
     "shell.execute_reply": "2020-12-13T16:45:29.834394Z"
    },
    "papermill": {
     "duration": 0.110313,
     "end_time": "2020-12-13T16:45:29.834587",
     "exception": false,
     "start_time": "2020-12-13T16:45:29.724274",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>body</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>y-zoom piano roll</td>\n",
       "      <td>a y-zoom on the piano roll would be useful.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>buggy behavior in selection</td>\n",
       "      <td>! screenshot from 2016-02-23 21 27 40  https:/...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>auto update feature</td>\n",
       "      <td>hi,\\r \\r great job so far, @saenzramiro ! : \\r...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>filter out noisy endpoints in logs</td>\n",
       "      <td>i think we should stop logging requests to:\\r ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>enable pid on / pid off alarm actions for ardu...</td>\n",
       "      <td>expected behavior\\r alarm actions pid on and p...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title  \\\n",
       "0                                  y-zoom piano roll   \n",
       "1                        buggy behavior in selection   \n",
       "2                                auto update feature   \n",
       "3                 filter out noisy endpoints in logs   \n",
       "4  enable pid on / pid off alarm actions for ardu...   \n",
       "\n",
       "                                                body  label  \n",
       "0        a y-zoom on the piano roll would be useful.      1  \n",
       "1  ! screenshot from 2016-02-23 21 27 40  https:/...      0  \n",
       "2  hi,\\r \\r great job so far, @saenzramiro ! : \\r...      1  \n",
       "3  i think we should stop logging requests to:\\r ...      1  \n",
       "4  expected behavior\\r alarm actions pid on and p...      0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_frame = pd.concat([train_df,train_ex_df],ignore_index=True)\n",
    "data_frame.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.057591,
     "end_time": "2020-12-13T16:45:29.951068",
     "exception": false,
     "start_time": "2020-12-13T16:45:29.893477",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "If we look at the label feild , it is basically divided into three segments as we mentioned earlier:\n",
    "* Bug - 0\n",
    "* Feature - 1\n",
    "* Question - 2\n",
    "\n",
    "Let's categorize the data fields based on their labels as **df_bug** for all the bugs labeled from the dataset. Similarly we do it for features as **df_feature**  and question as **df_question** respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:30.194291Z",
     "iopub.status.busy": "2020-12-13T16:45:30.193602Z",
     "iopub.status.idle": "2020-12-13T16:45:30.209970Z",
     "shell.execute_reply": "2020-12-13T16:45:30.209408Z"
    },
    "papermill": {
     "duration": 0.200949,
     "end_time": "2020-12-13T16:45:30.210088",
     "exception": false,
     "start_time": "2020-12-13T16:45:30.009139",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 450000 entries, 0 to 449999\n",
      "Data columns (total 3 columns):\n",
      " #   Column  Non-Null Count   Dtype \n",
      "---  ------  --------------   ----- \n",
      " 0   title   450000 non-null  object\n",
      " 1   body    450000 non-null  object\n",
      " 2   label   450000 non-null  int64 \n",
      "dtypes: int64(1), object(2)\n",
      "memory usage: 10.3+ MB\n"
     ]
    }
   ],
   "source": [
    "data_frame.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:30.333013Z",
     "iopub.status.busy": "2020-12-13T16:45:30.332165Z",
     "iopub.status.idle": "2020-12-13T16:45:30.546655Z",
     "shell.execute_reply": "2020-12-13T16:45:30.547245Z"
    },
    "papermill": {
     "duration": 0.27764,
     "end_time": "2020-12-13T16:45:30.547422",
     "exception": false,
     "start_time": "2020-12-13T16:45:30.269782",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Counts of label column: \n",
      " 1    207318\n",
      "0    200481\n",
      "2     42201\n",
      "Name: label, dtype: int64\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fda11660c50>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAAEMCAYAAABtKgnyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAba0lEQVR4nO3db2xT593/8U9iMC20VnDWZC5BS0kL8oiqaM7Ktl8ZmqMqbWUWbe3kykVVu4WuVUG3lkWpp24xCkk3s0x0bKFsatWqUqBSh2gUl8qsQ9rY1qpVeFB5rgpCgalgJSKhI/xRwn18fg8oXnOXP06Iz5Xg9+uZz/ecy19ySf74XMecU2Lbti0AAAwoNd0AAKB4EUIAAGMIIQCAMYQQAMAYQggAYMw80w3MJdlsVmfPntX8+fNVUlJiuh0AmBNs29aFCxe0aNEilZZOPvchhKbg7NmzOnTokOk2AGBOWr58uW699dZJ2wihKZg/f76ki39It9ttuBsAmBsmJiZ06NCh3Gfo5xFCU3BpCc7tdmvBggWGuwGAueVylzH4YQIAwBhCCABgDCEEADCGEAIAGEMIAQCMIYQAAMYQQgAAYwgh4DKy/3vBdAtFgb8z+M+qwGWUzpuvgS3Nptu44QXaXjLdAgzjTAgAYAwhBAAwxpEQOnXqlNavX6/GxkatXbtWGzZs0OjoqCRpcHBQ4XBYjY2NCofDOnr0aO44p2sAAGc5EkIlJSVqbm5WMplUf3+/li5dqu7ubklSLBZTJBJRMplUJBJRe3t77jinawAAZzkSQmVlZVq1alXudV1dnU6cOKGRkRGl02mFQiFJUigUUjqd1ujoqOM1AIDzHP91XDab1a5duxQMBpXJZFRZWSmXyyVJcrlcqqioUCaTkW3bjta8Xu+M/1snLlhyz3fN+Lj4L/7GwNzmeAht3rxZCxcu1Lp165ROp51++xmRSqXy2i8QCCjS1lvgborbzi2PamBgYMbHDQQCMz4mLq8Q84e5w9EQisfjOnbsmHbs2KHS0lL5fD4NDQ3Jsiy5XC5ZlqXh4WH5fD7Ztu1obSpqa2t5qN0sQmDMbczfjW98fPyKX94d+4n21q1blUql1NPTk3s0dnl5ufx+vxKJhCQpkUjI7/fL6/U6XgMAOK/Etm270G9y+PBhhUIhVVdX66abbpIkVVVVqaenR0eOHFE0GtXp06fl8XgUj8e1bNkySXK8di2X0nwqZ0IsxxXWzi2PFmxs7phQeNwxoThc7bPTkRC6URBCsw8hNLcRQsXhap+d3DEBAGAMIQQAMIYQAgAYQwgBAIwhhAAAxhBCAABjCCEAgDGEEADAGEIIAGAMIQQAMIYQAgAYQwgBAIwhhAAAxhBCAABjCCEAgDGEEADAmHlOvEk8HlcymdTx48fV39+v5cuX65NPPtEzzzyT22dsbExnzpzR+++/L0kKBoNyu925ByC1trZq9erVkqTBwUFFo1F9+umnKisrUzweV3V19XXVAADOcySEGhoa9Nhjj+nRR//7FMyqqir19fXlXnd1dcmyrEnHbdu2TcuXL//CeLFYTJFIRE1NTerr61N7e7tee+2166oBAJznyHJcfX29fD7fFesTExPq7+/XQw89dM2xRkZGlE6nFQqFJEmhUEjpdFqjo6PTrgEAzHDkTOha9u/fr8rKSq1cuXLS9tbWVtm2rUAgoJaWFnk8HmUyGVVWVsrlckmSXC6XKioqlMlkZNv2tGper3dK/aZSqbz2CwQCUxoX0zMwMDDjYzJ3zinE/GHumBUhtHv37i+cBfX29srn82liYkJdXV3q6OhQd3e3oQ4nq62tzV2rgnkExtzG/N34xsfHr/jl3fiv44aGhvTBBx9o7dq1k7ZfWr5zu92KRCI6ePBgbvvQ0FDu+pFlWRoeHpbP55t2DQBghvEQ2rNnj9asWaPFixfntp07d05jY2OSJNu2tXfvXvn9fklSeXm5/H6/EomEJCmRSMjv98vr9U67BgAww5HluM7OTu3bt08nT57UE088obKyMr311luSLobQc889N2n/kZERbdy4UZZlKZvNqqamRrFYLFfftGmTotGotm/fLo/Ho3g8ft01AIDzSmzbtk03MVdcWtecyjWhSFtvgbsqbju3PHrtnaZpYEtzwcbGRYG2l0y3AAdc7bPT+HIcAKB4EUIAAGMIIQCAMYQQAMAYQggAYAwhBAAwhhACABhDCAEAjCGEAADGEEIAAGMIIQCAMYQQAMAYQggAYAwhBAAwhhACABhDCAEAjHEkhOLxuILBoFasWKFDhw7ltgeDQd1///1qampSU1OTDhw4kKsNDg4qHA6rsbFR4XBYR48eLWgNAOA8R0KooaFBvb29WrJkyRdq27ZtU19fn/r6+rR69erc9lgspkgkomQyqUgkovb29oLWAADOcySE6uvr5fP58t5/ZGRE6XRaoVBIkhQKhZROpzU6OlqQGgDAjHmmG2htbZVt2woEAmppaZHH41Emk1FlZaVcLpckyeVyqaKiQplMRrZtz3jN6/VOqedUKpXXfoFAYErjYnoGBgZmfEzmzjmFmD/MHUZDqLe3Vz6fTxMTE+rq6lJHR4e6u7tNtpSX2tpaLViwwHQb+AyBMbcxfze+8fHxK355NxpCl5bo3G63IpGInn766dz2oaEhWZYll8sly7I0PDwsn88n27ZnvAYAMMPYT7TPnTunsbExSZJt29q7d6/8fr8kqby8XH6/X4lEQpKUSCTk9/vl9XoLUgMAmFFi27Zd6Dfp7OzUvn37dPLkSS1evFhlZWXasWOHNm7cKMuylM1mVVNTo5///OeqqKiQJB05ckTRaFSnT5+Wx+NRPB7XsmXLClbLx6VTyqksx0Xaeqfyp8IU7dzyaMHGHtjSXLCxcVGg7SXTLcABV/vsdCSEbhSE0OxDCM1thFBxuNpnJ3dMAAAYQwgBAIwhhAAAxhBCAABjCCEAgDGEEADAGEIIAGAMIQQAMIYQAgAYQwgBAIwhhAAAxhBCAABjCCEAgDGEEADAGEIIAGAMIQQAMMaREIrH4woGg1qxYoUOHTokSTp16pTWr1+vxsZGrV27Vhs2bNDo6GjumGAwqPvvv19NTU1qamrSgQMHcrXBwUGFw2E1NjYqHA7r6NGj110DADjPkRBqaGhQb2+vlixZkttWUlKi5uZmJZNJ9ff3a+nSperu7p503LZt29TX16e+vj6tXr06tz0WiykSiSiZTCoSiai9vf26awAA5zkSQvX19fL5fJO2lZWVadWqVbnXdXV1OnHixDXHGhkZUTqdVigUkiSFQiGl02mNjo5OuwYAMGOe6QYkKZvNateuXQoGg5O2t7a2yrZtBQIBtbS0yOPxKJPJqLKyUi6XS5LkcrlUUVGhTCYj27anVfN6vVPqN5VK5bVfIBCY0riYnoGBgRkfk7lzTiHmD3PHrAihzZs3a+HChVq3bl1uW29vr3w+nyYmJtTV1aWOjo4vLNeZUltbqwULFphuA58hMOY25u/GNz4+fsUv73kvx7388suX3f7KK69Mr6vPxONxHTt2TC+88IJKS//bzqXlO7fbrUgkooMHD+a2Dw0NybIsSZJlWRoeHpbP55t2DQBgRt4h1NPTc9ntL7744rTffOvWrUqlUurp6ZHb7c5tP3funMbGxiRJtm1r79698vv9kqTy8nL5/X4lEglJUiKRkN/vl9frnXYNAGDGNZfj3n33XUkXr9u89957sm07V/vkk0+0aNGia75JZ2en9u3bp5MnT+qJJ55QWVmZXnjhBe3YsUPV1dV65JFHJElVVVXq6enRyMiINm7cKMuylM1mVVNTo1gslhtv06ZNikaj2r59uzwej+Lx+HXXAADOK7E/nyqXcenHAplMZtLSVUlJiW677TatX79eDQ0Nhe1ylri0rjmVa0KRtt4Cd1Xcdm55tGBjD2xpLtjYuCjQ9pLpFuCAq312XvNMaP/+/ZKktrY2bdmypTAdAgCKUt6/jvt8AGWz2Um1z/+gAACAfOUdQv/617/U0dGhjz/+WOPj45Iu/migpKREH330UcEaBADcuPIOoWg0qu985zt6/vnnddNNNxWyJwBAkcg7hI4fP66f/OQnKikpKWQ/AIAikvfFnPvuu09///vfC9kLAKDI5H0mND4+rg0bNigQCOhLX/rSpBq/mgMATEfeIXTnnXfqzjvvLGQvAIAik3cIbdiwoZB9AACKUN4hdOn2PZfzzW9+c0aaAQAUl7xD6Lnnnpv0+tSpU7pw4YIqKyv1l7/8ZcYbAwDc+PIOoUu377nEsiy9+OKLed3AFACAy5n2/XZcLpeeeuopvfQSNyAEAEzPdd307R//+Af/eRUAMG15L8etWbNmUuCcP39eExMTk57zAwDAVOQdQr/+9a8nvb755pt1xx136JZbbpnxpgAAxSHv5bh77rlH99xzj+rr61VdXa2VK1fmHUDxeFzBYFArVqzQoUOHctsHBwcVDofV2NiocDiso0ePGqsBAJyXdwidOXNGbW1tuvvuu/Xtb39bd999t5599lmNjY1d89iGhgb19vZqyZIlk7bHYjFFIhElk0lFIhG1t7cbqwEAnJd3CHV2dur8+fPq7+/Xhx9+qP7+fp0/f16dnZ3XPLa+vn7So8ElaWRkROl0WqFQSJIUCoWUTqc1OjrqeA0AYEbe14QOHDigd955RzfffLMk6Y477tAvf/lL3XfffdN640wmo8rKSrlcLkkXf/JdUVGhTCYj27YdrXm93in1nkql8tovEAhMaVxMz8DAwIyPydw5pxDzh7kj7xBasGCBRkdHJy2pnTp1Sm63uyCNzWa1tbVasGCB6TbwGQJjbmP+bnzj4+NX/PKedwg9/PDD+uEPf6jHH39ct99+u06cOKFXX31VP/jBD6bVlM/n09DQkCzLksvlkmVZGh4els/nk23bjtYAAGbkfU3o6aef1pNPPqlkMql4PK5kMqnm5mY988wz03rj8vJy+f1+JRIJSVIikZDf75fX63W8BgAwo8S2bTufHTs7O/Xggw/qa1/7Wm7bwYMH9fbbb3/h5qaXO3bfvn06efKkFi9erLKyMr311ls6cuSIotGoTp8+LY/Ho3g8rmXLlkmS47V8XDqlnMpyXKStN+/xMXU7tzxasLEHtjQXbGxcFGjjtl/F4GqfnXmH0De+8Q397W9/m3QNaGJiQmvWrLnqYx5uJITQ7EMIzW2EUHG42mdn3stxJSUlymazk7ZZlvWFbQAA5CvvEKqvr9dvf/vbXOhks1n97ne/U319fcGaAwDc2Kb0ULsf//jHuvfee3X77bcrk8notttu044dOwrZHwDgBpZ3CH35y1/Wnj179OGHHyqTycjn8+nuu+9Wael1PQ0CAFDE8g4hSSotLVVdXZ3q6uoK1Q8AoIhwGgMAMIYQAgAYQwgBAIwhhAAAxhBCAABjCCEAgDGEEADAGEIIAGAMIQQAMIYQAgAYQwgBAIyZ0r3jCuGTTz6Z9IjwsbExnTlzRu+//76CwaDcbnfuIUitra1avXq1JGlwcFDRaFSffvqpysrKFI/HVV1dfV01AICzjIdQVVWV+vr6cq+7urpkWVbu9bZt27R8+fIvHBeLxRSJRNTU1KS+vj61t7frtddeu64aAMBZs2o5bmJiQv39/XrooYeuut/IyIjS6bRCoZAkKRQKKZ1Oa3R0dNo1AIDzjJ8Jfd7+/ftVWVmplStX5ra1trbKtm0FAgG1tLTI4/Eok8mosrJSLpdLkuRyuVRRUaFMJiPbtqdV83q9efeZSqXy2i8QCOQ9JqZvYGBgxsdk7pxTiPnD3DGrQmj37t2TzoJ6e3vl8/k0MTGhrq4udXR0qLu722CHF9XW1uauU8E8AmNuY/5ufOPj41f88j5rluOGhob0wQcfaO3atbltPp9PkuR2uxWJRHTw4MHc9qGhody1I8uyNDw8LJ/PN+0aAMB5syaE9uzZozVr1mjx4sWSpHPnzmlsbEySZNu29u7dK7/fL0kqLy+X3+9XIpGQJCUSCfn9fnm93mnXAADOmzXLcXv27NFzzz2Xez0yMqKNGzfKsixls1nV1NQoFovl6ps2bVI0GtX27dvl8XgUj8evuwYAcNasCaFkMjnp9dKlS/Xmm29ecf+amhq98cYbM1oDADhr1izHAQCKDyEEADCGEAIAGEMIAQCMIYQAAMYQQgAAYwghAIAxhBAAwBhCCABgDCEEADCGEAIAGEMIAQCMIYQAAMYQQgAAYwghAIAxhBAAwJhZEULBYFD333+/mpqa1NTUpAMHDkiSBgcHFQ6H1djYqHA4rKNHj+aOKUQNAOCsWRFCkrRt2zb19fWpr69Pq1evliTFYjFFIhElk0lFIhG1t7fn9i9EDQDgrFkTQv/XyMiI0um0QqGQJCkUCimdTmt0dLQgNQCA8+aZbuCS1tZW2batQCCglpYWZTIZVVZWyuVySZJcLpcqKiqUyWRk2/aM17xeb969plKpvPYLBAJT+RNgmgYGBmZ8TObOOYWYP8wdsyKEent75fP5NDExoa6uLnV0dOjxxx833dYV1dbWasGCBabbwGcIjLmN+bvxjY+PX/HL+6xYjvP5fJIkt9utSCSigwcPyufzaWhoSJZlSZIsy9Lw8LB8Pl9BagAA5xkPoXPnzmlsbEySZNu29u7dK7/fr/Lycvn9fiUSCUlSIpGQ3++X1+stSA0A4Dzjy3EjIyPauHGjLMtSNptVTU2NYrGYJGnTpk2KRqPavn27PB6P4vF47rhC1AAAziqxbds23cRccWldcyrXhCJtvQXuqrjt3PJowcYe2NJcsLFxUaDtJdMtwAFX++w0vhwHAChehBAAwBhCCMANZ+J/L5hu4YY3U39j4z9MAICZ5p43X4+/8j+m27ihvfrEb2dkHM6EAADGEEIAAGMIIQCAMYQQAMAYQggAYAwhBAAwhhACABhDCAEAjCGEAADGEEIAAGMIIQCAMYQQAMAY4zcwPXXqlNra2vTvf/9bbrdbX/nKV9TR0SGv16tgMCi32517CFJra6tWr14tSRocHFQ0GtWnn36qsrIyxeNxVVdXX1cNAOAs42dCJSUlam5uVjKZVH9/v5YuXaru7u5cfdu2berr61NfX18ugCQpFospEokomUwqEomovb39umsAAGcZD6GysjKtWrUq97qurk4nTpy46jEjIyNKp9MKhUKSpFAopHQ6rdHR0WnXAADOM74c93nZbFa7du1SMBjMbWttbZVt2woEAmppaZHH41Emk1FlZaVcLpckyeVyqaKiQplMRrZtT6vm9Xrz7jOVSuW1XyAQyHtMTN/AwMCMj8ncOYf5m7tmYu5mVQht3rxZCxcu1Lp16yRJvb298vl8mpiYUFdXlzo6OiYt1ZlSW1ubu04F8/jAmduYv7kr37kbHx+/4pd348txl8TjcR07dkwvvPCCSksvtuXz+SRJbrdbkUhEBw8ezG0fGhqSZVmSJMuyNDw8LJ/PN+0aAMB5syKEtm7dqlQqpZ6eHrndbknSuXPnNDY2JkmybVt79+6V3++XJJWXl8vv9yuRSEiSEomE/H6/vF7vtGsAAOcZX447fPiwduzYoerqaj3yyCOSpKqqKkWjUW3cuFGWZSmbzaqmpkaxWCx33KZNmxSNRrV9+3Z5PB7F4/HrrgEAnGU8hO666y59/PHHl629+eabVzyupqZGb7zxxozWAADOmhXLcQCA4kQIAQCMIYQAAMYQQgAAYwghAIAxhBAAwBhCCABgDCEEADCGEAIAGEMIAQCMIYQAAMYQQgAAYwghAIAxhBAAwBhCCABgDCEEADCmKENocHBQ4XBYjY2NCofDOnr0qOmWAKAoFWUIxWIxRSIRJZNJRSIRtbe3m24JAIqS8cd7O21kZETpdFqvvPKKJCkUCmnz5s0aHR2V1+u96rG2bUuSJiYm8n4/z8L5028W1zQ+Pl64wW+6tXBjQ1Jh5+/W+YsKNjamNneXPjMvfYZ+XtGFUCaTUWVlpVwulyTJ5XKpoqJCmUzmmiF04cIFSdKhQ4fyfr/1a2um3yyuKZVKFW7w/7eucGNDUmHn73H/QwUbG9ObuwsXLuimm26atK3oQuh6LFq0SMuXL9f8+fNVUlJiuh0AmBNs29aFCxe0aNEXz06LLoR8Pp+GhoZkWZZcLpcsy9Lw8LB8Pt81jy0tLdWtt7JEAwBT9X/PgC4puh8mlJeXy+/3K5FISJISiYT8fv81l+IAADOvxL7claIb3JEjRxSNRnX69Gl5PB7F43EtW7bMdFsAUHSKMoQAALND0S3HAQBmD0IIAGAMIQQAMIYQAgAYQwhBEjd1ncvi8biCwaBWrFgxpbt5wLxTp05p/fr1amxs1Nq1a7VhwwaNjo6abstRhBAkcVPXuayhoUG9vb1asmSJ6VYwRSUlJWpublYymVR/f7+WLl2q7u5u0205ihBC7qauoVBI0sWbuqbT6aL7RjZX1dfX53XHD8w+ZWVlWrVqVe51XV2dTpw4YbAj5xFCuOpNXQE4I5vNateuXQoGg6ZbcRQhBACzwObNm7Vw4UKtW1dcd28vuhuY4ouu56auAK5fPB7XsWPHtGPHDpWWFte5QXH9a3FZ3NQVMGfr1q1KpVLq6emR2+023Y7juHccJHFT17mss7NT+/bt08mTJ7V48WKVlZXprbfeMt0W8nD48GGFQiFVV1fnHnVQVVWlnp4ew505hxACABjDchwAwBhCCABgDCEEADCGEAIAGEMIAQCMIYSAWSgYDOqf//znNfdbsWKFjh07Nq33uJ5jgZlCCAEAjCGEAADGEELALPbhhx8qHA6rvr5e9957rzo6OjQxMTFpn7/+9a9qaGjQqlWrFI/Hlc1mc7U//elPeuCBB/T1r39dP/rRj3T8+HGn/wnAVRFCwCxWWlqqn/3sZ3rvvff0+uuv691339XOnTsn7fPnP/9Zu3fv1p49e7R//37t3r1bkvTOO+/oD3/4g37/+9/r3XffVSAQ0E9/+lMT/wzgigghYBarra1VXV2d5s2bp6qqKoXDYX3wwQeT9lm/fr3Kysp0++2367HHHsvdiPb111/Xk08+qZqaGs2bN09PPfWUPvroI86GMKvwKAdgFhscHNSvfvUrpVIpnT9/XpZlaeXKlZP2+fwjN5YsWaLh4WFJ0okTJ/T8888rHo/n6rZta2hoiEeBY9YghIBZbNOmTfrqV7+q3/zmN7rlllv06quvKplMTtonk8norrvuknQxeCoqKiRdDKennnpK3/3udx3vG8gXy3HALHb27FktWrRIixYt0pEjR7Rr164v7PPyyy/rP//5jzKZjF577TU9+OCDkqRHHnlEf/zjH3X48GFJ0tjYmN5++21H+weuhTMhYBZ79tln9Ytf/EIvv/yy/H6/HnzwQb333nuT9mloaND3v/99nTlzRt/73vf08MMPS5Luu+8+nT17Vi0tLTp+/LhuvfVWfetb39IDDzxg4p8CXBbPEwIAGMNyHADAGEIIAGAMIQQAMIYQAgAYQwgBAIwhhAAAxhBCAABjCCEAgDGEEADAmP8P+y25xvSpKroAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Total Counts of label column: \\n'.format(),data_frame['label'].value_counts())\n",
    "sns.set(style=\"whitegrid\", color_codes=True)\n",
    "sns.countplot(x='label', data=data_frame)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:30.677208Z",
     "iopub.status.busy": "2020-12-13T16:45:30.676328Z",
     "iopub.status.idle": "2020-12-13T16:45:30.725851Z",
     "shell.execute_reply": "2020-12-13T16:45:30.725242Z"
    },
    "papermill": {
     "duration": 0.113687,
     "end_time": "2020-12-13T16:45:30.725978",
     "exception": false,
     "start_time": "2020-12-13T16:45:30.612291",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_bug = data_frame[data_frame['label']==0]\n",
    "df_feature = data_frame[data_frame['label']==1]\n",
    "df_question = data_frame[data_frame['label']==2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:30.855960Z",
     "iopub.status.busy": "2020-12-13T16:45:30.855104Z",
     "iopub.status.idle": "2020-12-13T16:45:30.867631Z",
     "shell.execute_reply": "2020-12-13T16:45:30.867101Z"
    },
    "papermill": {
     "duration": 0.081318,
     "end_time": "2020-12-13T16:45:30.867759",
     "exception": false,
     "start_time": "2020-12-13T16:45:30.786441",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    200481\n",
       "1    207318\n",
       "2     42201\n",
       "Name: label, dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "label_counts = data_frame.label.value_counts().sort_index()\n",
    "label_counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:30.993191Z",
     "iopub.status.busy": "2020-12-13T16:45:30.992271Z",
     "iopub.status.idle": "2020-12-13T16:45:51.192586Z",
     "shell.execute_reply": "2020-12-13T16:45:51.192001Z"
    },
    "papermill": {
     "duration": 20.264659,
     "end_time": "2020-12-13T16:45:51.192706",
     "exception": false,
     "start_time": "2020-12-13T16:45:30.928047",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def fx(x):\n",
    "    return x['title'] + \" \" + x['body']   \n",
    "data_frame['text']= data_frame.apply(lambda x : fx(x),axis=1)\n",
    "data_frame['text']= data_frame.apply(lambda x : fx(x),axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.058262,
     "end_time": "2020-12-13T16:45:51.314866",
     "exception": false,
     "start_time": "2020-12-13T16:45:51.256604",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Count the Lables :  Bug, Feature abd Questions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:51.443577Z",
     "iopub.status.busy": "2020-12-13T16:45:51.442601Z",
     "iopub.status.idle": "2020-12-13T16:45:51.448090Z",
     "shell.execute_reply": "2020-12-13T16:45:51.447342Z"
    },
    "papermill": {
     "duration": 0.072687,
     "end_time": "2020-12-13T16:45:51.448230",
     "exception": false,
     "start_time": "2020-12-13T16:45:51.375543",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of datapoints with label as Bug : 200481\n",
      "Number of datapoints with label as Feature : 207318\n",
      "Number of datapoints with label as Question : 42201\n"
     ]
    }
   ],
   "source": [
    "print('Number of datapoints with label as Bug :',label_counts[0])\n",
    "print('Number of datapoints with label as Feature :',label_counts[1])\n",
    "print('Number of datapoints with label as Question :',label_counts[2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.059004,
     "end_time": "2020-12-13T16:45:51.566838",
     "exception": false,
     "start_time": "2020-12-13T16:45:51.507834",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's ensure we make text lowercase, remove text in square brackets, remove links, remove punctuation and remove words containing numbers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.080133,
     "end_time": "2020-12-13T16:45:51.705506",
     "exception": false,
     "start_time": "2020-12-13T16:45:51.625373",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Cleaning the Text Field"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:51.888637Z",
     "iopub.status.busy": "2020-12-13T16:45:51.887847Z",
     "iopub.status.idle": "2020-12-13T16:45:52.413203Z",
     "shell.execute_reply": "2020-12-13T16:45:52.413788Z"
    },
    "papermill": {
     "duration": 0.613859,
     "end_time": "2020-12-13T16:45:52.413939",
     "exception": false,
     "start_time": "2020-12-13T16:45:51.800080",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import nltk\n",
    "import re\n",
    "import string\n",
    "\n",
    "\n",
    "def clean_text(text):\n",
    "    '''Make text lowercase, remove text in square brackets,remove links,remove punctuation\n",
    "    and remove words containing numbers'''\n",
    "    text = text.lower()\n",
    "    text = re.sub('\\[.*?\\]', '', text)\n",
    "    text = re.sub('https?://\\S+|www\\.\\S+', '', text)\n",
    "    text = re.sub('<.*?>+', '', text)\n",
    "    text = re.sub('[%s]' % re.escape(string.punctuation), '', text)\n",
    "    text = re.sub('\\n', '', text)\n",
    "    text = re.sub('\\w*\\d\\w*', '', text)\n",
    "\n",
    "    return text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.08541,
     "end_time": "2020-12-13T16:45:52.587950",
     "exception": false,
     "start_time": "2020-12-13T16:45:52.502540",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Removing Stopwords & Punctuations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:52.775535Z",
     "iopub.status.busy": "2020-12-13T16:45:52.774599Z",
     "iopub.status.idle": "2020-12-13T16:45:52.777543Z",
     "shell.execute_reply": "2020-12-13T16:45:52.778198Z"
    },
    "papermill": {
     "duration": 0.101102,
     "end_time": "2020-12-13T16:45:52.778372",
     "exception": false,
     "start_time": "2020-12-13T16:45:52.677270",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from string import punctuation\n",
    "from nltk.corpus import stopwords\n",
    "\n",
    "def remove_stopwords(git_text):\n",
    "    # filters charecter-by-charecter : ['h', 'e', 'e', 'l', 'o', 'o', ' ', 'm', 'y', ' ', 'n', 'a', 'm', 'e', ' ', 'i', 's', ' ', 'p', 'u', 'r', 'v', 'a']\n",
    "    remove_punctuation = [ch for ch in git_text if ch not in punctuation]\n",
    "    # convert them back to sentences and split into words\n",
    "    remove_punctuation = \"\".join(remove_punctuation).split()\n",
    "    filtered_git_text = [word.lower() for word in remove_punctuation if word.lower() not in stopwords.words('english')]\n",
    "    return filtered_git_text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.105295,
     "end_time": "2020-12-13T16:45:53.006351",
     "exception": false,
     "start_time": "2020-12-13T16:45:52.901056",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Visulaize the Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:53.196228Z",
     "iopub.status.busy": "2020-12-13T16:45:53.195303Z",
     "iopub.status.idle": "2020-12-13T16:45:54.535124Z",
     "shell.execute_reply": "2020-12-13T16:45:54.536039Z"
    },
    "papermill": {
     "duration": 1.435682,
     "end_time": "2020-12-13T16:45:54.536265",
     "exception": false,
     "start_time": "2020-12-13T16:45:53.100583",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "import plotly.express as px\n",
    "\n",
    "def visulize_dataset(data_frame, category):\n",
    "    \n",
    "    # Let's apply the above two functions 'clean_text' and 'remove_stopwords' to the whole dataset\n",
    "\n",
    "    data_frame['text'] = data_frame['text'].apply(lambda x: x.replace(\"\\\\r\", \"\"))\n",
    "    data_frame['text'] = data_frame['text'].apply(lambda x: clean_text(x))\n",
    "    data_frame[\"text\"] = data_frame[\"text\"].apply(remove_stopwords)\n",
    "    \n",
    "    \n",
    "    word_list = []\n",
    "    \n",
    "    for i, j in data_frame.iterrows():\n",
    "        for word in j['text']:\n",
    "            word_list.append(word)\n",
    "        \n",
    "    count_dict = Counter(word_list)\n",
    "    most_common_words_df = pd.DataFrame(count_dict.most_common(20), columns=['word', 'count'])\n",
    "    fig = px.histogram(most_common_words_df,\n",
    "                       x='word', \n",
    "                       y='count',\n",
    "                       title='Most common terms used while refering to a GitHub {}'.format(category),\n",
    "                       color_discrete_sequence=['#843B62'] )\n",
    "    fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.063457,
     "end_time": "2020-12-13T16:45:54.684646",
     "exception": false,
     "start_time": "2020-12-13T16:45:54.621189",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Printing the text feild after Cleaning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:45:54.812381Z",
     "iopub.status.busy": "2020-12-13T16:45:54.811425Z",
     "iopub.status.idle": "2020-12-13T16:47:32.074504Z",
     "shell.execute_reply": "2020-12-13T16:47:32.075019Z"
    },
    "papermill": {
     "duration": 97.329618,
     "end_time": "2020-12-13T16:47:32.075164",
     "exception": false,
     "start_time": "2020-12-13T16:45:54.745546",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>body</th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>y-zoom piano roll</td>\n",
       "      <td>a y-zoom on the piano roll would be useful.</td>\n",
       "      <td>1</td>\n",
       "      <td>yzoom piano roll a yzoom on the piano roll wou...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>buggy behavior in selection</td>\n",
       "      <td>! screenshot from 2016-02-23 21 27 40  https:/...</td>\n",
       "      <td>0</td>\n",
       "      <td>buggy behavior in selection  screenshot from  ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>auto update feature</td>\n",
       "      <td>hi,\\r \\r great job so far, @saenzramiro ! : \\r...</td>\n",
       "      <td>1</td>\n",
       "      <td>auto update feature hi  great job so far saenz...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>filter out noisy endpoints in logs</td>\n",
       "      <td>i think we should stop logging requests to:\\r ...</td>\n",
       "      <td>1</td>\n",
       "      <td>filter out noisy endpoints in logs i think we ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>enable pid on / pid off alarm actions for ardu...</td>\n",
       "      <td>expected behavior\\r alarm actions pid on and p...</td>\n",
       "      <td>0</td>\n",
       "      <td>enable pid on  pid off alarm actions for  expe...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title  \\\n",
       "0                                  y-zoom piano roll   \n",
       "1                        buggy behavior in selection   \n",
       "2                                auto update feature   \n",
       "3                 filter out noisy endpoints in logs   \n",
       "4  enable pid on / pid off alarm actions for ardu...   \n",
       "\n",
       "                                                body  label  \\\n",
       "0        a y-zoom on the piano roll would be useful.      1   \n",
       "1  ! screenshot from 2016-02-23 21 27 40  https:/...      0   \n",
       "2  hi,\\r \\r great job so far, @saenzramiro ! : \\r...      1   \n",
       "3  i think we should stop logging requests to:\\r ...      1   \n",
       "4  expected behavior\\r alarm actions pid on and p...      0   \n",
       "\n",
       "                                                text  \n",
       "0  yzoom piano roll a yzoom on the piano roll wou...  \n",
       "1  buggy behavior in selection  screenshot from  ...  \n",
       "2  auto update feature hi  great job so far saenz...  \n",
       "3  filter out noisy endpoints in logs i think we ...  \n",
       "4  enable pid on  pid off alarm actions for  expe...  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"\n",
    "label 0: Bug\n",
    "label 1: Feature\n",
    "label 2: Question\n",
    "\"\"\"\n",
    "data_frame['text'] = data_frame['text'].apply(lambda x: x.replace(\"\\\\r\", \"\"))\n",
    "data_frame['text'] = data_frame['text'].apply(lambda x: clean_text(x))\n",
    "data_frame.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:47:32.210101Z",
     "iopub.status.busy": "2020-12-13T16:47:32.209329Z",
     "iopub.status.idle": "2020-12-13T16:47:32.214999Z",
     "shell.execute_reply": "2020-12-13T16:47:32.214484Z"
    },
    "papermill": {
     "duration": 0.078143,
     "end_time": "2020-12-13T16:47:32.215104",
     "exception": false,
     "start_time": "2020-12-13T16:47:32.136961",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bound method NDFrame.describe of                                                     title  \\\n",
       "0                                       y-zoom piano roll   \n",
       "1                             buggy behavior in selection   \n",
       "2                                     auto update feature   \n",
       "3                      filter out noisy endpoints in logs   \n",
       "4       enable pid on / pid off alarm actions for ardu...   \n",
       "...                                                   ...   \n",
       "449995                       shield against invalid dates   \n",
       "449996                                     fix the knight   \n",
       "449997  portaudio playback binding crashes often, ubun...   \n",
       "449998  add --db-path option for allowing user to spec...   \n",
       "449999                 staff users cannot log in to admin   \n",
       "\n",
       "                                                     body  label  \\\n",
       "0             a y-zoom on the piano roll would be useful.      1   \n",
       "1       ! screenshot from 2016-02-23 21 27 40  https:/...      0   \n",
       "2       hi,\\r \\r great job so far, @saenzramiro ! : \\r...      1   \n",
       "3       i think we should stop logging requests to:\\r ...      1   \n",
       "4       expected behavior\\r alarm actions pid on and p...      0   \n",
       "...                                                   ...    ...   \n",
       "449995  makes'em crash the renderer for now :|\\r \\r da...      0   \n",
       "449996   seriously, if you do just one thing this week...      0   \n",
       "449997         appear to be very unstable. relates to  17      0   \n",
       "449998  add  --db-path  option with default value as  ...      1   \n",
       "449999  i added admire as a staff user but he cannot l...      0   \n",
       "\n",
       "                                                     text  \n",
       "0       yzoom piano roll a yzoom on the piano roll wou...  \n",
       "1       buggy behavior in selection  screenshot from  ...  \n",
       "2       auto update feature hi  great job so far saenz...  \n",
       "3       filter out noisy endpoints in logs i think we ...  \n",
       "4       enable pid on  pid off alarm actions for  expe...  \n",
       "...                                                   ...  \n",
       "449995  shield against invalid dates makesem crash the...  \n",
       "449996  fix the knight seriously if you do just one th...  \n",
       "449997  portaudio playback binding crashes often ubunt...  \n",
       "449998  add dbpath option for allowing user to specify...  \n",
       "449999  staff users cannot log in to admin i added adm...  \n",
       "\n",
       "[450000 rows x 4 columns]>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_frame.head()\n",
    "data_frame.describe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:47:32.343575Z",
     "iopub.status.busy": "2020-12-13T16:47:32.342801Z",
     "iopub.status.idle": "2020-12-13T16:47:32.351029Z",
     "shell.execute_reply": "2020-12-13T16:47:32.351541Z"
    },
    "papermill": {
     "duration": 0.074605,
     "end_time": "2020-12-13T16:47:32.351662",
     "exception": false,
     "start_time": "2020-12-13T16:47:32.277057",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7fdad00469b0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RANDOM_SEED = 42\n",
    "np.random.seed(RANDOM_SEED)\n",
    "torch.manual_seed(RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.06244,
     "end_time": "2020-12-13T16:47:32.476382",
     "exception": false,
     "start_time": "2020-12-13T16:47:32.413942",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# The Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:47:32.605777Z",
     "iopub.status.busy": "2020-12-13T16:47:32.605000Z",
     "iopub.status.idle": "2020-12-13T16:47:32.657964Z",
     "shell.execute_reply": "2020-12-13T16:47:32.657368Z"
    },
    "papermill": {
     "duration": 0.119468,
     "end_time": "2020-12-13T16:47:32.658086",
     "exception": false,
     "start_time": "2020-12-13T16:47:32.538618",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "class config:\n",
    "    MAX_LEN = 128\n",
    "    TRAIN_BATCH_SIZE = 64\n",
    "    VALID_BATCH_SIZE = 16\n",
    "    EPOCHS = 5\n",
    "    BERT_PATH = \"/kaggle/input/bert-base-uncased\"\n",
    "    MODEL_PATH = \"../input/bert-base-uncased/pytorch_model.bin\"\n",
    "    tokenizer = BertTokenizer.from_pretrained(BERT_PATH)\n",
    "    truncation=True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:47:32.790571Z",
     "iopub.status.busy": "2020-12-13T16:47:32.789675Z",
     "iopub.status.idle": "2020-12-13T16:48:11.389198Z",
     "shell.execute_reply": "2020-12-13T16:48:11.389854Z"
    },
    "papermill": {
     "duration": 38.668971,
     "end_time": "2020-12-13T16:48:11.390039",
     "exception": false,
     "start_time": "2020-12-13T16:47:32.721068",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ce6847e4e60d443bbb755316fd7efa72",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=433.0, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f9e7726d0af64c46afee749c8b22d6ed",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=440473133.0, style=ProgressStyle(descri…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "BCEWithLogitsLoss()"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "\n",
    "model = BertModel.from_pretrained(\"bert-base-uncased\")\n",
    "# optimizer = torch.optim.SGD(model.parameters(), lr=1e-3, momentum=0.99, weight_decay=0.005)\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)\n",
    "criterion = nn.BCEWithLogitsLoss()\n",
    "\n",
    "model.to(device)\n",
    "criterion.to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:11.526603Z",
     "iopub.status.busy": "2020-12-13T16:48:11.525890Z",
     "iopub.status.idle": "2020-12-13T16:48:11.530450Z",
     "shell.execute_reply": "2020-12-13T16:48:11.529964Z"
    },
    "papermill": {
     "duration": 0.07362,
     "end_time": "2020-12-13T16:48:11.530550",
     "exception": false,
     "start_time": "2020-12-13T16:48:11.456930",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#loading our BERT model\n",
    "BERT_PATH = '/kaggle/input/bert-base-uncased'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:11.662529Z",
     "iopub.status.busy": "2020-12-13T16:48:11.661530Z",
     "iopub.status.idle": "2020-12-13T16:48:11.703123Z",
     "shell.execute_reply": "2020-12-13T16:48:11.702519Z"
    },
    "papermill": {
     "duration": 0.108963,
     "end_time": "2020-12-13T16:48:11.703231",
     "exception": false,
     "start_time": "2020-12-13T16:48:11.594268",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#loading the pre-trained BertTokenizer\n",
    "tokenizer = BertTokenizer.from_pretrained(BERT_PATH)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.067601,
     "end_time": "2020-12-13T16:48:11.836016",
     "exception": false,
     "start_time": "2020-12-13T16:48:11.768415",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# BERT conversion of sentences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:11.985511Z",
     "iopub.status.busy": "2020-12-13T16:48:11.984695Z",
     "iopub.status.idle": "2020-12-13T16:48:11.992327Z",
     "shell.execute_reply": "2020-12-13T16:48:11.991757Z"
    },
    "papermill": {
     "duration": 0.084737,
     "end_time": "2020-12-13T16:48:11.992449",
     "exception": false,
     "start_time": "2020-12-13T16:48:11.907712",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Sentence: script stopped adding videos saenzramiro abc xyz\n",
      "   Tokens: ['script', 'stopped', 'adding', 'videos', 'sa', '##en', '##z', '##ram', '##iro', 'abc', 'x', '##y', '##z']\n",
      "Token IDs: [5896, 3030, 5815, 6876, 7842, 2368, 2480, 6444, 9711, 5925, 1060, 2100, 2480]\n"
     ]
    }
   ],
   "source": [
    "# some basic operations to understand how BERT converts a sentence into tokens and then into IDs\n",
    "sample_body = 'script stopped adding videos saenzramiro abc xyz'\n",
    "tokens = tokenizer.tokenize(sample_body)\n",
    "token_ids = tokenizer.convert_tokens_to_ids(tokens)\n",
    "\n",
    "print(f' Sentence: {sample_body}')\n",
    "print(f'   Tokens: {tokens}')\n",
    "print(f'Token IDs: {token_ids}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.066143,
     "end_time": "2020-12-13T16:48:12.125485",
     "exception": false,
     "start_time": "2020-12-13T16:48:12.059342",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Adding [CLS]:101, [SEP]:102, [PAD]:0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:12.267290Z",
     "iopub.status.busy": "2020-12-13T16:48:12.266350Z",
     "iopub.status.idle": "2020-12-13T16:48:12.276733Z",
     "shell.execute_reply": "2020-12-13T16:48:12.276086Z"
    },
    "papermill": {
     "duration": 0.085611,
     "end_time": "2020-12-13T16:48:12.276840",
     "exception": false,
     "start_time": "2020-12-13T16:48:12.191229",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "dict_keys(['input_ids', 'attention_mask'])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# using encode_plus to add special tokens : [CLS]:101, [SEP]:102, [PAD]:0\n",
    "encodings = tokenizer.encode_plus(\n",
    "            sample_body,\n",
    "            max_length=32,\n",
    "            add_special_tokens=True,\n",
    "            return_token_type_ids=False,\n",
    "            pad_to_max_length=True,\n",
    "            return_attention_mask=True,\n",
    "            return_tensors='pt'\n",
    ")\n",
    "\n",
    "encodings.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:12.420627Z",
     "iopub.status.busy": "2020-12-13T16:48:12.419748Z",
     "iopub.status.idle": "2020-12-13T16:48:12.429771Z",
     "shell.execute_reply": "2020-12-13T16:48:12.430269Z"
    },
    "papermill": {
     "duration": 0.085113,
     "end_time": "2020-12-13T16:48:12.430406",
     "exception": false,
     "start_time": "2020-12-13T16:48:12.345293",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input IDs : tensor([ 101, 5896, 3030, 5815, 6876, 7842, 2368, 2480, 6444, 9711, 5925, 1060,\n",
      "        2100, 2480,  102,    0,    0,    0,    0,    0,    0,    0,    0,    0,\n",
      "           0,    0,    0,    0,    0,    0,    0,    0])\n",
      "\n",
      "Attention Mask : tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "        0, 0, 0, 0, 0, 0, 0, 0])\n"
     ]
    }
   ],
   "source": [
    "print('Input IDs : {}'.format(encodings['input_ids'][0]))\n",
    "print('\\nAttention Mask : {}'.format(encodings['attention_mask'][0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:12.572854Z",
     "iopub.status.busy": "2020-12-13T16:48:12.570993Z",
     "iopub.status.idle": "2020-12-13T16:48:12.573605Z",
     "shell.execute_reply": "2020-12-13T16:48:12.574133Z"
    },
    "papermill": {
     "duration": 0.073757,
     "end_time": "2020-12-13T16:48:12.574260",
     "exception": false,
     "start_time": "2020-12-13T16:48:12.500503",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "MAX_LENGTH = 512"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.066807,
     "end_time": "2020-12-13T16:48:12.710560",
     "exception": false,
     "start_time": "2020-12-13T16:48:12.643753",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Construct a Class for GitHub Messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:12.852005Z",
     "iopub.status.busy": "2020-12-13T16:48:12.851028Z",
     "iopub.status.idle": "2020-12-13T16:48:12.862675Z",
     "shell.execute_reply": "2020-12-13T16:48:12.862113Z"
    },
    "papermill": {
     "duration": 0.084704,
     "end_time": "2020-12-13T16:48:12.862788",
     "exception": false,
     "start_time": "2020-12-13T16:48:12.778084",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "class Git_Message(Dataset):\n",
    "    def __init__(self, git_messages, label, tokenizer, max_len):\n",
    "        self.git_messages = git_messages\n",
    "        self.label = label\n",
    "        self.tokenizer = config.tokenizer\n",
    "        self.max_len = config.MAX_LEN\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.git_messages)\n",
    "\n",
    "    def __getitem__(self, item):\n",
    "        git_messages = str(self.git_messages[item])\n",
    "        label = self.label[item]\n",
    "        \n",
    "        encoding = self.tokenizer.encode_plus(\n",
    "        git_messages,\n",
    "        add_special_tokens=True,\n",
    "        max_length=self.max_len,\n",
    "        return_token_type_ids=False,\n",
    "        pad_to_max_length=True,\n",
    "        return_attention_mask=True,\n",
    "        return_tensors='pt')\n",
    "        return {\n",
    "        'git_messages': git_messages,\n",
    "         'input_ids': encoding['input_ids'],\n",
    "         'attention_mask': encoding['attention_mask'],\n",
    "         'label': torch.tensor(label, dtype=torch.long)\n",
    "          }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.070252,
     "end_time": "2020-12-13T16:48:13.003701",
     "exception": false,
     "start_time": "2020-12-13T16:48:12.933449",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Creating Train, Test Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:13.144551Z",
     "iopub.status.busy": "2020-12-13T16:48:13.143640Z",
     "iopub.status.idle": "2020-12-13T16:48:13.158431Z",
     "shell.execute_reply": "2020-12-13T16:48:13.157633Z"
    },
    "papermill": {
     "duration": 0.08811,
     "end_time": "2020-12-13T16:48:13.158789",
     "exception": false,
     "start_time": "2020-12-13T16:48:13.070679",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:13.310315Z",
     "iopub.status.busy": "2020-12-13T16:48:13.309335Z",
     "iopub.status.idle": "2020-12-13T16:48:13.313068Z",
     "shell.execute_reply": "2020-12-13T16:48:13.313567Z"
    },
    "papermill": {
     "duration": 0.085082,
     "end_time": "2020-12-13T16:48:13.313687",
     "exception": false,
     "start_time": "2020-12-13T16:48:13.228605",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>body</th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>y-zoom piano roll</td>\n",
       "      <td>a y-zoom on the piano roll would be useful.</td>\n",
       "      <td>1</td>\n",
       "      <td>yzoom piano roll a yzoom on the piano roll wou...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>buggy behavior in selection</td>\n",
       "      <td>! screenshot from 2016-02-23 21 27 40  https:/...</td>\n",
       "      <td>0</td>\n",
       "      <td>buggy behavior in selection  screenshot from  ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>auto update feature</td>\n",
       "      <td>hi,\\r \\r great job so far, @saenzramiro ! : \\r...</td>\n",
       "      <td>1</td>\n",
       "      <td>auto update feature hi  great job so far saenz...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>filter out noisy endpoints in logs</td>\n",
       "      <td>i think we should stop logging requests to:\\r ...</td>\n",
       "      <td>1</td>\n",
       "      <td>filter out noisy endpoints in logs i think we ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>enable pid on / pid off alarm actions for ardu...</td>\n",
       "      <td>expected behavior\\r alarm actions pid on and p...</td>\n",
       "      <td>0</td>\n",
       "      <td>enable pid on  pid off alarm actions for  expe...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title  \\\n",
       "0                                  y-zoom piano roll   \n",
       "1                        buggy behavior in selection   \n",
       "2                                auto update feature   \n",
       "3                 filter out noisy endpoints in logs   \n",
       "4  enable pid on / pid off alarm actions for ardu...   \n",
       "\n",
       "                                                body  label  \\\n",
       "0        a y-zoom on the piano roll would be useful.      1   \n",
       "1  ! screenshot from 2016-02-23 21 27 40  https:/...      0   \n",
       "2  hi,\\r \\r great job so far, @saenzramiro ! : \\r...      1   \n",
       "3  i think we should stop logging requests to:\\r ...      1   \n",
       "4  expected behavior\\r alarm actions pid on and p...      0   \n",
       "\n",
       "                                                text  \n",
       "0  yzoom piano roll a yzoom on the piano roll wou...  \n",
       "1  buggy behavior in selection  screenshot from  ...  \n",
       "2  auto update feature hi  great job so far saenz...  \n",
       "3  filter out noisy endpoints in logs i think we ...  \n",
       "4  enable pid on  pid off alarm actions for  expe...  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_frame.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:13.455254Z",
     "iopub.status.busy": "2020-12-13T16:48:13.454445Z",
     "iopub.status.idle": "2020-12-13T16:48:13.457370Z",
     "shell.execute_reply": "2020-12-13T16:48:13.456849Z"
    },
    "papermill": {
     "duration": 0.075353,
     "end_time": "2020-12-13T16:48:13.457482",
     "exception": false,
     "start_time": "2020-12-13T16:48:13.382129",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "data_frame = data_frame[:2000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:13.602151Z",
     "iopub.status.busy": "2020-12-13T16:48:13.601217Z",
     "iopub.status.idle": "2020-12-13T16:48:13.606492Z",
     "shell.execute_reply": "2020-12-13T16:48:13.607090Z"
    },
    "papermill": {
     "duration": 0.080526,
     "end_time": "2020-12-13T16:48:13.607206",
     "exception": false,
     "start_time": "2020-12-13T16:48:13.526680",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "training_data, testing_data = train_test_split(\n",
    "    data_frame,\n",
    "    test_size=0.1,\n",
    "    random_state=RANDOM_SEED\n",
    ")\n",
    "\n",
    "testing_data, validation_data = train_test_split(\n",
    "    testing_data,\n",
    "    test_size=0.5,\n",
    "    random_state=RANDOM_SEED\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:13.750377Z",
     "iopub.status.busy": "2020-12-13T16:48:13.749420Z",
     "iopub.status.idle": "2020-12-13T16:48:13.753631Z",
     "shell.execute_reply": "2020-12-13T16:48:13.753123Z"
    },
    "papermill": {
     "duration": 0.078637,
     "end_time": "2020-12-13T16:48:13.753753",
     "exception": false,
     "start_time": "2020-12-13T16:48:13.675116",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1800, 4), (100, 4), (100, 4))"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_data.shape, testing_data.shape, validation_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.073187,
     "end_time": "2020-12-13T16:48:13.897781",
     "exception": false,
     "start_time": "2020-12-13T16:48:13.824594",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Creating Data Loader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:14.048489Z",
     "iopub.status.busy": "2020-12-13T16:48:14.047550Z",
     "iopub.status.idle": "2020-12-13T16:48:14.050567Z",
     "shell.execute_reply": "2020-12-13T16:48:14.050026Z"
    },
    "papermill": {
     "duration": 0.082842,
     "end_time": "2020-12-13T16:48:14.050676",
     "exception": false,
     "start_time": "2020-12-13T16:48:13.967834",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def create_data_loader(data, tokenizer, max_len, batch_size):\n",
    "    \n",
    "    ds = Git_Message(git_messages=data.text.to_numpy(),\n",
    "    label=data.label.to_numpy(),\n",
    "    tokenizer=tokenizer,\n",
    "    max_len=max_len)\n",
    "    \n",
    "    return DataLoader(\n",
    "    ds,\n",
    "    batch_size=batch_size,\n",
    "    num_workers=4)\n",
    "\n",
    "\n",
    "BATCH_SIZE = 16\n",
    "train_data_loader = create_data_loader(training_data, tokenizer, MAX_LENGTH, BATCH_SIZE)\n",
    "testing_data_loader = create_data_loader(testing_data, tokenizer, MAX_LENGTH, BATCH_SIZE)\n",
    "val_data_loader = create_data_loader(validation_data, tokenizer, MAX_LENGTH, BATCH_SIZE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:14.194329Z",
     "iopub.status.busy": "2020-12-13T16:48:14.193357Z",
     "iopub.status.idle": "2020-12-13T16:48:14.756021Z",
     "shell.execute_reply": "2020-12-13T16:48:14.755495Z"
    },
    "papermill": {
     "duration": 0.636209,
     "end_time": "2020-12-13T16:48:14.756135",
     "exception": false,
     "start_time": "2020-12-13T16:48:14.119926",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "dict_keys(['git_messages', 'input_ids', 'attention_mask', 'label'])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_frame = next(iter(train_data_loader))\n",
    "data_frame.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:14.912124Z",
     "iopub.status.busy": "2020-12-13T16:48:14.911325Z",
     "iopub.status.idle": "2020-12-13T16:48:14.916519Z",
     "shell.execute_reply": "2020-12-13T16:48:14.917046Z"
    },
    "papermill": {
     "duration": 0.087903,
     "end_time": "2020-12-13T16:48:14.917169",
     "exception": false,
     "start_time": "2020-12-13T16:48:14.829266",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([16, 128]), torch.Size([16, 128]), torch.Size([16]))"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_frame['input_ids'].squeeze().shape, data_frame['attention_mask'].squeeze().shape, data_frame['label'].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:15.066424Z",
     "iopub.status.busy": "2020-12-13T16:48:15.065604Z",
     "iopub.status.idle": "2020-12-13T16:48:15.077504Z",
     "shell.execute_reply": "2020-12-13T16:48:15.078411Z"
    },
    "papermill": {
     "duration": 0.089743,
     "end_time": "2020-12-13T16:48:15.078608",
     "exception": false,
     "start_time": "2020-12-13T16:48:14.988865",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "git_messages  :  how to use html instead of twig  for a template file extension  thanks  jason\n",
      "input_ids :  tensor([  101,  2129,  2000,  2224, 16129,  2612,  1997,  1056, 16279,  2005,\n",
      "         1037, 23561,  5371,  5331,  4283,  4463,   102,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
      "            0,     0,     0,     0,     0,     0,     0,     0])\n",
      "attention_mask :  tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0,\n",
      "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
      "        0, 0, 0, 0, 0, 0, 0, 0])\n",
      "label :  tensor(2)\n"
     ]
    }
   ],
   "source": [
    "print('git_messages  : ', data_frame['git_messages'][0])\n",
    "print('input_ids : ', data_frame['input_ids'].squeeze()[0])\n",
    "print('attention_mask : ', data_frame['attention_mask'].squeeze()[0])\n",
    "print('label : ', data_frame['label'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:15.227407Z",
     "iopub.status.busy": "2020-12-13T16:48:15.226812Z",
     "iopub.status.idle": "2020-12-13T16:48:23.698914Z",
     "shell.execute_reply": "2020-12-13T16:48:23.697503Z"
    },
    "papermill": {
     "duration": 8.548246,
     "end_time": "2020-12-13T16:48:23.699032",
     "exception": false,
     "start_time": "2020-12-13T16:48:15.150786",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "bert_model = BertModel.from_pretrained(BERT_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:23.847701Z",
     "iopub.status.busy": "2020-12-13T16:48:23.846811Z",
     "iopub.status.idle": "2020-12-13T16:48:24.044973Z",
     "shell.execute_reply": "2020-12-13T16:48:24.044331Z"
    },
    "papermill": {
     "duration": 0.27487,
     "end_time": "2020-12-13T16:48:24.045096",
     "exception": false,
     "start_time": "2020-12-13T16:48:23.770226",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "last_hidden_state, pooled_output = bert_model(\n",
    "  input_ids=encodings['input_ids'],\n",
    "  attention_mask=encodings['attention_mask']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:24.198497Z",
     "iopub.status.busy": "2020-12-13T16:48:24.197572Z",
     "iopub.status.idle": "2020-12-13T16:48:24.201521Z",
     "shell.execute_reply": "2020-12-13T16:48:24.202034Z"
    },
    "papermill": {
     "duration": 0.083097,
     "end_time": "2020-12-13T16:48:24.202172",
     "exception": false,
     "start_time": "2020-12-13T16:48:24.119075",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1, 32, 768]), torch.Size([1, 768]))"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_hidden_state.shape, pooled_output.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.072693,
     "end_time": "2020-12-13T16:48:24.350214",
     "exception": false,
     "start_time": "2020-12-13T16:48:24.277521",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## **Predictor Model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:24.505248Z",
     "iopub.status.busy": "2020-12-13T16:48:24.503343Z",
     "iopub.status.idle": "2020-12-13T16:48:24.506022Z",
     "shell.execute_reply": "2020-12-13T16:48:24.506526Z"
    },
    "papermill": {
     "duration": 0.084541,
     "end_time": "2020-12-13T16:48:24.506647",
     "exception": false,
     "start_time": "2020-12-13T16:48:24.422106",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "class BugPredictor(nn.Module):\n",
    "    \n",
    "    def __init__(self, n_classes):\n",
    "        super(BugPredictor, self).__init__()\n",
    "        self.bert_model = BertModel.from_pretrained(BERT_PATH)\n",
    "        self.dropout = nn.Dropout(p=0.0)\n",
    "        self.out = nn.Linear(self.bert_model.config.hidden_size, n_classes)\n",
    "        \n",
    "    def forward(self, input_ids, attention_mask):\n",
    "        _, pooled_output = self.bert_model(\n",
    "        input_ids=input_ids,\n",
    "        attention_mask = attention_mask\n",
    "        )\n",
    "        output = self.dropout(pooled_output)\n",
    "        return self.out(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.071553,
     "end_time": "2020-12-13T16:48:24.649391",
     "exception": false,
     "start_time": "2020-12-13T16:48:24.577838",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Let's Start Training the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:24.800179Z",
     "iopub.status.busy": "2020-12-13T16:48:24.799433Z",
     "iopub.status.idle": "2020-12-13T16:48:27.528432Z",
     "shell.execute_reply": "2020-12-13T16:48:27.527289Z"
    },
    "papermill": {
     "duration": 2.806113,
     "end_time": "2020-12-13T16:48:27.528553",
     "exception": false,
     "start_time": "2020-12-13T16:48:24.722440",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "label 0: Bug\n",
    "label 1: Feature\n",
    "label 2: Question\n",
    "\"\"\"\n",
    "class_names = [0, 1, 2]\n",
    "bug_predictor_model = BugPredictor(len(class_names))\n",
    "bug_predictor_model = bug_predictor_model.to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:27.684874Z",
     "iopub.status.busy": "2020-12-13T16:48:27.684120Z",
     "iopub.status.idle": "2020-12-13T16:48:27.687388Z",
     "shell.execute_reply": "2020-12-13T16:48:27.687879Z"
    },
    "papermill": {
     "duration": 0.086499,
     "end_time": "2020-12-13T16:48:27.688005",
     "exception": false,
     "start_time": "2020-12-13T16:48:27.601506",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "EPOCHS = 10\n",
    "\n",
    "optimizer = AdamW(bug_predictor_model.parameters(), lr=2e-5, correct_bias=False)\n",
    "total_steps = len(train_data_loader) * EPOCHS\n",
    "\n",
    "scheduler = get_linear_schedule_with_warmup(\n",
    "    optimizer,\n",
    "    num_warmup_steps = 0,\n",
    "    num_training_steps = total_steps\n",
    ")\n",
    "\n",
    "loss_fn = nn.CrossEntropyLoss().to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:27.849899Z",
     "iopub.status.busy": "2020-12-13T16:48:27.848133Z",
     "iopub.status.idle": "2020-12-13T16:48:27.851245Z",
     "shell.execute_reply": "2020-12-13T16:48:27.851825Z"
    },
    "papermill": {
     "duration": 0.091786,
     "end_time": "2020-12-13T16:48:27.851958",
     "exception": false,
     "start_time": "2020-12-13T16:48:27.760172",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def train_model(model, data_loader, loss_fn, optimizer, device, scheduler, n_examples):\n",
    "    model = model.train()\n",
    "    \n",
    "    losses = []\n",
    "    correct_predictions = 0\n",
    "    \n",
    "    for d in data_loader:\n",
    "        input_ids = d['input_ids'].squeeze().to(device)\n",
    "        attention_mask = d['attention_mask'].squeeze().to(device)\n",
    "        targets = d['label'].to(device)\n",
    "\n",
    "        outputs = model(input_ids = input_ids, attention_mask = attention_mask)\n",
    "        _, preds = torch.max(outputs, dim=1)\n",
    "        loss = loss_fn(outputs, targets)\n",
    "        \n",
    "        correct_predictions += torch.sum(preds==targets)\n",
    "        losses.append(loss.item())\n",
    "        \n",
    "        loss.backward()\n",
    "        nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n",
    "        optimizer.step()\n",
    "        scheduler.step()\n",
    "        optimizer.zero_grad()\n",
    "    \n",
    "    return correct_predictions.double()/n_examples, np.mean(losses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:28.013819Z",
     "iopub.status.busy": "2020-12-13T16:48:28.012895Z",
     "iopub.status.idle": "2020-12-13T16:48:28.015644Z",
     "shell.execute_reply": "2020-12-13T16:48:28.015128Z"
    },
    "papermill": {
     "duration": 0.087874,
     "end_time": "2020-12-13T16:48:28.015775",
     "exception": false,
     "start_time": "2020-12-13T16:48:27.927901",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def eval_model(model, data_loader, loss_fn, device, n_examples):\n",
    "    model = model.eval()\n",
    "    \n",
    "    losses = []\n",
    "    correct_predictions = 0\n",
    "    \n",
    "    with torch.no_grad():\n",
    "        for d in data_loader:\n",
    "            input_ids = d['input_ids'].squeeze().to(device)\n",
    "            attention_mask = d['attention_mask'].squeeze().to(device)\n",
    "            targets = d['label'].to(device)\n",
    "\n",
    "            outputs = model(input_ids = input_ids, attention_mask = attention_mask)\n",
    "            _, preds = torch.max(outputs, dim=1)\n",
    "            loss = loss_fn(outputs, targets)\n",
    "\n",
    "            correct_predictions += torch.sum(preds==targets)\n",
    "            losses.append(loss.item())\n",
    "    \n",
    "    return correct_predictions.double()/n_examples, np.mean(losses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:48:28.170392Z",
     "iopub.status.busy": "2020-12-13T16:48:28.169461Z",
     "iopub.status.idle": "2020-12-13T16:53:19.031341Z",
     "shell.execute_reply": "2020-12-13T16:53:19.030683Z"
    },
    "papermill": {
     "duration": 290.943388,
     "end_time": "2020-12-13T16:53:19.031468",
     "exception": false,
     "start_time": "2020-12-13T16:48:28.088080",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "EPOCH 1/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.6944666371672554 accuracy : 0.7138888888888889\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.5042440614530018 accuracy : 0.81\n",
      "Saving the best model ...\n",
      "EPOCH 2/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.4178736749871642 accuracy : 0.8505555555555555\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.493165733558791 accuracy : 0.8300000000000001\n",
      "Saving the best model ...\n",
      "EPOCH 3/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.216841061426475 accuracy : 0.9338888888888889\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.4432843237050942 accuracy : 0.85\n",
      "Saving the best model ...\n",
      "EPOCH 4/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.09865674041870949 accuracy : 0.9738888888888889\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.6207537371665239 accuracy : 0.85\n",
      "EPOCH 5/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.04824518742023316 accuracy : 0.9872222222222222\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.6582272543206013 accuracy : 0.85\n",
      "EPOCH 6/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.02082199550623206 accuracy : 0.9944444444444445\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.7303051552735269 accuracy : 0.84\n",
      "EPOCH 7/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.005806491606708089 accuracy : 0.9988888888888889\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.7917869131197222 accuracy : 0.85\n",
      "EPOCH 8/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.0033428893392118504 accuracy : 0.9988888888888889\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.7805440685021624 accuracy : 0.84\n",
      "EPOCH 9/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.000884617515015457 accuracy : 1.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.8214602688551947 accuracy : 0.85\n",
      "EPOCH 10/10\n",
      "----------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss : 0.000495937797374842 accuracy : 1.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation loss : 0.8179555840823534 accuracy : 0.85\n",
      "CPU times: user 4min 20s, sys: 6.92 s, total: 4min 27s\n",
      "Wall time: 4min 50s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "from collections import defaultdict\n",
    "\n",
    "history = defaultdict(list)\n",
    "best_accuracy = 0\n",
    "\n",
    "for epoch in range(EPOCHS):\n",
    "    print('EPOCH {}/{}'.format(epoch+1,EPOCHS))\n",
    "    print('-' * 10)\n",
    "  \n",
    "    train_acc, train_loss = train_model(bug_predictor_model, train_data_loader, loss_fn, optimizer, device, scheduler, len(training_data))\n",
    "    \n",
    "    print('Train loss : {} accuracy : {}'.format(train_loss, train_acc))\n",
    "    \n",
    "    val_acc, val_loss = eval_model(bug_predictor_model, val_data_loader, loss_fn, device, len(validation_data))\n",
    "    \n",
    "    print('Validation loss : {} accuracy : {}'.format(val_loss, val_acc))\n",
    "    \n",
    "    history['train_acc'].append(train_acc)\n",
    "    history['train_loss'].append(train_loss)\n",
    "    history['val_acc'].append(val_acc)\n",
    "    history['val_loss'].append(val_loss)\n",
    "    \n",
    "    if val_acc > best_accuracy:\n",
    "        print('Saving the best model ...')\n",
    "        torch.save(bug_predictor_model.state_dict(), 'best_model.bin')\n",
    "        best_accuracy = val_acc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.118948,
     "end_time": "2020-12-13T16:53:19.268292",
     "exception": false,
     "start_time": "2020-12-13T16:53:19.149344",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:53:19.510231Z",
     "iopub.status.busy": "2020-12-13T16:53:19.508258Z",
     "iopub.status.idle": "2020-12-13T16:53:19.511071Z",
     "shell.execute_reply": "2020-12-13T16:53:19.511559Z"
    },
    "papermill": {
     "duration": 0.130726,
     "end_time": "2020-12-13T16:53:19.511699",
     "exception": false,
     "start_time": "2020-12-13T16:53:19.380973",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "sample_bug_message = \"Script stopped adding video's. A recent change in the youtube layout broke the script. Probably caused by element names being altered.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:53:19.744486Z",
     "iopub.status.busy": "2020-12-13T16:53:19.743667Z",
     "iopub.status.idle": "2020-12-13T16:53:19.748016Z",
     "shell.execute_reply": "2020-12-13T16:53:19.747376Z"
    },
    "papermill": {
     "duration": 0.123831,
     "end_time": "2020-12-13T16:53:19.748126",
     "exception": false,
     "start_time": "2020-12-13T16:53:19.624295",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "class_names = ['bug', 'feature', 'question']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:53:20.106910Z",
     "iopub.status.busy": "2020-12-13T16:53:20.104634Z",
     "iopub.status.idle": "2020-12-13T16:53:20.107988Z",
     "shell.execute_reply": "2020-12-13T16:53:20.108766Z"
    },
    "papermill": {
     "duration": 0.192341,
     "end_time": "2020-12-13T16:53:20.108949",
     "exception": false,
     "start_time": "2020-12-13T16:53:19.916608",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def predict_git_category(sample_message, model):\n",
    "    encoded_message = tokenizer.encode_plus(sample_bug_message, max_length=MAX_LENGTH, add_special_tokens=True, return_token_type_ids=False, pad_to_max_length=True, return_attention_mask=True, return_tensors='pt')\n",
    "    input_ids = encoded_message['input_ids'].to(device)\n",
    "    attention_mask = encoded_message['attention_mask'].to(device)\n",
    "    \n",
    "    output = model(input_ids=input_ids, attention_mask=attention_mask)\n",
    "    _, prediction_idx = torch.max(output, dim=1)\n",
    "        \n",
    "    return class_names[prediction_idx]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-13T16:53:20.455798Z",
     "iopub.status.busy": "2020-12-13T16:53:20.454853Z",
     "iopub.status.idle": "2020-12-13T16:53:20.490924Z",
     "shell.execute_reply": "2020-12-13T16:53:20.490205Z"
    },
    "papermill": {
     "duration": 0.214127,
     "end_time": "2020-12-13T16:53:20.491081",
     "exception": false,
     "start_time": "2020-12-13T16:53:20.276954",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample bug message :  Script stopped adding video's. A recent change in the youtube layout broke the script. Probably caused by element names being altered.\n",
      "Predicted GitHub Category :  bug\n"
     ]
    }
   ],
   "source": [
    "print('Sample bug message : ', sample_bug_message)\n",
    "print('Predicted GitHub Category : ', predict_git_category(sample_bug_message, bug_predictor_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.168235,
     "end_time": "2020-12-13T16:53:20.829450",
     "exception": false,
     "start_time": "2020-12-13T16:53:20.661215",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Would like to thank and give credits to the Kaggle Kernel by  [ Purva Singh](http://https://www.kaggle.com/purvasingh/github-bug-prediction-via-bert). I have a taken the ideas and workflow as a guide to bhild my first Kaggle Notebook. 😊👏👏👏👏👏👏👏👏"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.153961,
     "end_time": "2020-12-13T16:53:21.156863",
     "exception": false,
     "start_time": "2020-12-13T16:53:21.002902",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "papermill": {
   "duration": 499.974064,
   "end_time": "2020-12-13T16:53:21.989640",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2020-12-13T16:45:02.015576",
   "version": "2.1.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "0a18c659b7ca4ba68bcc0b8d01067530": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1a754c04f07a46cb94f18be34677cefd": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "2cd9f19500cf4b67af0ca9f5c1494ca5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_554fc2f94f214242b1a49ee2e0c18d5e",
       "placeholder": "​",
       "style": "IPY_MODEL_ae7d30b25b71472f8fdbea49e868d642",
       "value": " 440M/440M [00:29&lt;00:00, 15.1MB/s]"
      }
     },
     "36e41377ce9e403d82f1fa285bec07ea": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_0a18c659b7ca4ba68bcc0b8d01067530",
       "max": 440473133.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_b150f667c2db41c3b1e6b5a3cef0df9d",
       "value": 440473133.0
      }
     },
     "554fc2f94f214242b1a49ee2e0c18d5e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "5673453c7b7b43ce80904849cf35da02": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "5ee1524fb4c2482e9bb36496385f6861": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "69201bc6f446494cb71752406bc2e2bd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "8a0550c7fe314a59b39e5152da44fd31": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "8aea9437bd87477a9155de5bb2734855": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_5ee1524fb4c2482e9bb36496385f6861",
       "placeholder": "​",
       "style": "IPY_MODEL_8a0550c7fe314a59b39e5152da44fd31",
       "value": " 433/433 [00:00&lt;00:00, 5.68kB/s]"
      }
     },
     "ae7d30b25b71472f8fdbea49e868d642": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "b150f667c2db41c3b1e6b5a3cef0df9d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "ce6847e4e60d443bbb755316fd7efa72": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_eb7af668da474af7b9a1f6b00040bb75",
        "IPY_MODEL_8aea9437bd87477a9155de5bb2734855"
       ],
       "layout": "IPY_MODEL_dc2afa561c364b049d68637559fa0da2"
      }
     },
     "dc2afa561c364b049d68637559fa0da2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "eb7af668da474af7b9a1f6b00040bb75": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_1a754c04f07a46cb94f18be34677cefd",
       "max": 433.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_69201bc6f446494cb71752406bc2e2bd",
       "value": 433.0
      }
     },
     "f9e7726d0af64c46afee749c8b22d6ed": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_36e41377ce9e403d82f1fa285bec07ea",
        "IPY_MODEL_2cd9f19500cf4b67af0ca9f5c1494ca5"
       ],
       "layout": "IPY_MODEL_5673453c7b7b43ce80904849cf35da02"
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}