XGBoost_and_Hyperparameter_Search.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# CS-6600 Homework 5 - XGBoost and Hyperparameter Search\n",
        "\n",
        "*Weber State University*\n",
        "\n",
        "**Your Name**: Belal Glab\n",
        "\n"
      ],
      "metadata": {
        "id": "pokRtVGyet8U"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "For this assignment we'll be exploring another *boosting* algorithm like AdaBoost and Gradient Boost - XGBoost. XGBoost is similar to gradient boost in its approach, but a bit more mathematically complicated. More mathematically complicated that we'll need to get into here, but if you're curious you can check out:\n",
        "\n",
        "* [XGBoost: A Scalable Tree Boosting System](https://arxiv.org/abs/1603.02754)\n",
        "* [XGBoost Video Series](https://youtu.be/OtD8wVaFm6E?si=M8SAhD0eMU6an8a7)\n",
        "* [XGBoost Docs](https://xgboost.readthedocs.io/en/stable/)\n",
        "\n"
      ],
      "metadata": {
        "id": "RhF5yT-OgGEN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "XGBoost was all the rage back in 2016, where it was the method behind a bunch of Kaggle competiton winners."
      ],
      "metadata": {
        "id": "jJ9UEbJRiTSS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's grab the libraries we'll need:"
      ],
      "metadata": {
        "id": "oRV4ovHrn-L9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#The usual suspects\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "#Our synthetic data creator\n",
        "from sklearn.datasets import make_classification\n",
        "\n",
        "#For creating the validation set\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "#For creating an XGBoost model\n",
        "import xgboost as xgb\n",
        "\n",
        "#For inspecting model performance\n",
        "from sklearn.metrics import roc_auc_score\n",
        "\n",
        "#For random sampling\n",
        "from scipy.stats import uniform\n",
        "from sklearn.model_selection import ParameterSampler"
      ],
      "metadata": {
        "id": "gcZWzLLuoBu1"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Generating The Data"
      ],
      "metadata": {
        "id": "d1gpzNk5u3ZM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "For this assignment we'll use a synthetic dataset created with scikit-learn's [make_classification](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_classification.html) method."
      ],
      "metadata": {
        "id": "FNzuRSmGoluj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X,y = make_classification(n_samples = 5000, n_features = 40, n_informative = 3, n_redundant = 2, n_clusters_per_class = 3, weights = None, flip_y = .05, class_sep = 0.1, hypercube = True, shift = 0.0, scale = 1.0, shuffle = True, random_state = 6600)"
      ],
      "metadata": {
        "id": "Wg-PtKi5pAOE"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "This creates a synthetic dataset with 5000 samples and 40 features. However, most of these features are useless - only 3 are informative, and another 2 are redundant (we can derive their values from the informative ones). So, only 5 features should be useful in making predictions, and if we can learn the relation between the informative ones and the redundant ones, we could get by just fine with 3."
      ],
      "metadata": {
        "id": "GFqLqUUzpWF0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll want to use both a training and a validation set for this assignment, so let's split the data placing 80% in the training set and 20% in the validation set."
      ],
      "metadata": {
        "id": "hUFAGopPpzID"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_val, y_train, y_val = train_test_split(X, y, test_size = .2, random_state = 6600)"
      ],
      "metadata": {
        "id": "WJJ9vKoxqJjB"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Creating an XGBoost Model"
      ],
      "metadata": {
        "id": "QJ3lkArmu-qU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Alright, let's cretae a XGBoost model and train it. Note that XGBoost is its own library separate from scikit-learn."
      ],
      "metadata": {
        "id": "vng6o-uPqXP6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "First, XGBoost uses its own internal data structure called a DMatrix, which is optimized for both memory efficiency and training speed with XGBoost. We'll encode both our training data and validation data as DMatrix structures."
      ],
      "metadata": {
        "id": "tJlQnCNKThhj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dtrain = xgb.DMatrix(X_train, label = y_train)\n",
        "dval = xgb.DMatrix(X_val, label = y_val)"
      ],
      "metadata": {
        "id": "iRE4rNo0xsqB"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we're going to create a dictionary of parameters for the model. We'll use the logistic regression objective function (we'll get into this later), and ROC AUC as the evaluation metric. We'll also set the learning rate to $0.3$."
      ],
      "metadata": {
        "id": "DxibFh15T02E"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "params = {'objective': 'binary:logistic', 'eval_metric': 'auc', 'learning_rate': 0.3}"
      ],
      "metadata": {
        "id": "p1hRtltDxxMy"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Alright. Now, we're going to create an XGBoost model. Unlike with scikit-learn, we won't first create the model object and then train (fit) it. Rather, we'll call the *train* method in xgb, and that will produce a trained model object.\n",
        "\n",
        "We'll pass the *train* method the params, training data, number of boost rounds, and the datasets we should use for evaluation (we'll want to see the ROC AUC scores for both the training data and testing data)."
      ],
      "metadata": {
        "id": "pC3TgKH6ULHy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "xgb_model_1 = xgb.train(params, dtrain, num_boost_round = 1000, evals = [(dtrain, 'train'), (dval, 'val')])"
      ],
      "metadata": {
        "id": "14hca5U9x6Qn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fc3f6139-ba58-4675-c587-ac6910990823"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[0]\ttrain-auc:0.79531\tval-auc:0.73119\n",
            "[1]\ttrain-auc:0.80636\tval-auc:0.73976\n",
            "[2]\ttrain-auc:0.82250\tval-auc:0.73920\n",
            "[3]\ttrain-auc:0.83829\tval-auc:0.73583\n",
            "[4]\ttrain-auc:0.85144\tval-auc:0.74157\n",
            "[5]\ttrain-auc:0.87664\tval-auc:0.73214\n",
            "[6]\ttrain-auc:0.88978\tval-auc:0.73201\n",
            "[7]\ttrain-auc:0.89280\tval-auc:0.73452\n",
            "[8]\ttrain-auc:0.90171\tval-auc:0.72859\n",
            "[9]\ttrain-auc:0.90602\tval-auc:0.73272\n",
            "[10]\ttrain-auc:0.92157\tval-auc:0.73137\n",
            "[11]\ttrain-auc:0.93475\tval-auc:0.72668\n",
            "[12]\ttrain-auc:0.94233\tval-auc:0.72230\n",
            "[13]\ttrain-auc:0.95055\tval-auc:0.72127\n",
            "[14]\ttrain-auc:0.95533\tval-auc:0.71564\n",
            "[15]\ttrain-auc:0.95822\tval-auc:0.71433\n",
            "[16]\ttrain-auc:0.96280\tval-auc:0.71521\n",
            "[17]\ttrain-auc:0.96772\tval-auc:0.71352\n",
            "[18]\ttrain-auc:0.97143\tval-auc:0.72707\n",
            "[19]\ttrain-auc:0.97464\tval-auc:0.72644\n",
            "[20]\ttrain-auc:0.97718\tval-auc:0.72580\n",
            "[21]\ttrain-auc:0.97893\tval-auc:0.72509\n",
            "[22]\ttrain-auc:0.98167\tval-auc:0.72142\n",
            "[23]\ttrain-auc:0.98256\tval-auc:0.72197\n",
            "[24]\ttrain-auc:0.98553\tval-auc:0.72124\n",
            "[25]\ttrain-auc:0.98676\tval-auc:0.72233\n",
            "[26]\ttrain-auc:0.98976\tval-auc:0.71914\n",
            "[27]\ttrain-auc:0.99136\tval-auc:0.71955\n",
            "[28]\ttrain-auc:0.99207\tval-auc:0.71662\n",
            "[29]\ttrain-auc:0.99281\tval-auc:0.71585\n",
            "[30]\ttrain-auc:0.99367\tval-auc:0.71515\n",
            "[31]\ttrain-auc:0.99423\tval-auc:0.71510\n",
            "[32]\ttrain-auc:0.99499\tval-auc:0.71714\n",
            "[33]\ttrain-auc:0.99579\tval-auc:0.71353\n",
            "[34]\ttrain-auc:0.99647\tval-auc:0.71266\n",
            "[35]\ttrain-auc:0.99705\tval-auc:0.71214\n",
            "[36]\ttrain-auc:0.99774\tval-auc:0.71002\n",
            "[37]\ttrain-auc:0.99842\tval-auc:0.70773\n",
            "[38]\ttrain-auc:0.99882\tval-auc:0.70448\n",
            "[39]\ttrain-auc:0.99900\tval-auc:0.70465\n",
            "[40]\ttrain-auc:0.99900\tval-auc:0.71135\n",
            "[41]\ttrain-auc:0.99926\tval-auc:0.71195\n",
            "[42]\ttrain-auc:0.99941\tval-auc:0.71073\n",
            "[43]\ttrain-auc:0.99948\tval-auc:0.71603\n",
            "[44]\ttrain-auc:0.99957\tval-auc:0.71544\n",
            "[45]\ttrain-auc:0.99962\tval-auc:0.71299\n",
            "[46]\ttrain-auc:0.99966\tval-auc:0.71295\n",
            "[47]\ttrain-auc:0.99973\tval-auc:0.71015\n",
            "[48]\ttrain-auc:0.99976\tval-auc:0.71113\n",
            "[49]\ttrain-auc:0.99983\tval-auc:0.70911\n",
            "[50]\ttrain-auc:0.99989\tval-auc:0.70748\n",
            "[51]\ttrain-auc:0.99991\tval-auc:0.70729\n",
            "[52]\ttrain-auc:0.99991\tval-auc:0.70750\n",
            "[53]\ttrain-auc:0.99990\tval-auc:0.71041\n",
            "[54]\ttrain-auc:0.99991\tval-auc:0.70865\n",
            "[55]\ttrain-auc:0.99992\tval-auc:0.70999\n",
            "[56]\ttrain-auc:0.99992\tval-auc:0.71010\n",
            "[57]\ttrain-auc:0.99993\tval-auc:0.71086\n",
            "[58]\ttrain-auc:0.99996\tval-auc:0.70984\n",
            "[59]\ttrain-auc:0.99997\tval-auc:0.70970\n",
            "[60]\ttrain-auc:0.99998\tval-auc:0.70962\n",
            "[61]\ttrain-auc:0.99998\tval-auc:0.71577\n",
            "[62]\ttrain-auc:0.99999\tval-auc:0.71602\n",
            "[63]\ttrain-auc:0.99999\tval-auc:0.71536\n",
            "[64]\ttrain-auc:0.99999\tval-auc:0.71551\n",
            "[65]\ttrain-auc:1.00000\tval-auc:0.71620\n",
            "[66]\ttrain-auc:1.00000\tval-auc:0.71569\n",
            "[67]\ttrain-auc:1.00000\tval-auc:0.71603\n",
            "[68]\ttrain-auc:1.00000\tval-auc:0.72203\n",
            "[69]\ttrain-auc:1.00000\tval-auc:0.72131\n",
            "[70]\ttrain-auc:1.00000\tval-auc:0.72181\n",
            "[71]\ttrain-auc:1.00000\tval-auc:0.72103\n",
            "[72]\ttrain-auc:1.00000\tval-auc:0.72589\n",
            "[73]\ttrain-auc:1.00000\tval-auc:0.72528\n",
            "[74]\ttrain-auc:1.00000\tval-auc:0.72573\n",
            "[75]\ttrain-auc:1.00000\tval-auc:0.72413\n",
            "[76]\ttrain-auc:1.00000\tval-auc:0.72445\n",
            "[77]\ttrain-auc:1.00000\tval-auc:0.72510\n",
            "[78]\ttrain-auc:1.00000\tval-auc:0.72343\n",
            "[79]\ttrain-auc:1.00000\tval-auc:0.72318\n",
            "[80]\ttrain-auc:1.00000\tval-auc:0.72262\n",
            "[81]\ttrain-auc:1.00000\tval-auc:0.72177\n",
            "[82]\ttrain-auc:1.00000\tval-auc:0.72408\n",
            "[83]\ttrain-auc:1.00000\tval-auc:0.72402\n",
            "[84]\ttrain-auc:1.00000\tval-auc:0.72283\n",
            "[85]\ttrain-auc:1.00000\tval-auc:0.72195\n",
            "[86]\ttrain-auc:1.00000\tval-auc:0.72234\n",
            "[87]\ttrain-auc:1.00000\tval-auc:0.72333\n",
            "[88]\ttrain-auc:1.00000\tval-auc:0.72310\n",
            "[89]\ttrain-auc:1.00000\tval-auc:0.72406\n",
            "[90]\ttrain-auc:1.00000\tval-auc:0.72282\n",
            "[91]\ttrain-auc:1.00000\tval-auc:0.72329\n",
            "[92]\ttrain-auc:1.00000\tval-auc:0.72317\n",
            "[93]\ttrain-auc:1.00000\tval-auc:0.72339\n",
            "[94]\ttrain-auc:1.00000\tval-auc:0.72267\n",
            "[95]\ttrain-auc:1.00000\tval-auc:0.72200\n",
            "[96]\ttrain-auc:1.00000\tval-auc:0.72345\n",
            "[97]\ttrain-auc:1.00000\tval-auc:0.72327\n",
            "[98]\ttrain-auc:1.00000\tval-auc:0.72277\n",
            "[99]\ttrain-auc:1.00000\tval-auc:0.72378\n",
            "[100]\ttrain-auc:1.00000\tval-auc:0.72420\n",
            "[101]\ttrain-auc:1.00000\tval-auc:0.72418\n",
            "[102]\ttrain-auc:1.00000\tval-auc:0.72418\n",
            "[103]\ttrain-auc:1.00000\tval-auc:0.72380\n",
            "[104]\ttrain-auc:1.00000\tval-auc:0.72447\n",
            "[105]\ttrain-auc:1.00000\tval-auc:0.72534\n",
            "[106]\ttrain-auc:1.00000\tval-auc:0.72532\n",
            "[107]\ttrain-auc:1.00000\tval-auc:0.72476\n",
            "[108]\ttrain-auc:1.00000\tval-auc:0.72492\n",
            "[109]\ttrain-auc:1.00000\tval-auc:0.72399\n",
            "[110]\ttrain-auc:1.00000\tval-auc:0.72489\n",
            "[111]\ttrain-auc:1.00000\tval-auc:0.72375\n",
            "[112]\ttrain-auc:1.00000\tval-auc:0.72438\n",
            "[113]\ttrain-auc:1.00000\tval-auc:0.72685\n",
            "[114]\ttrain-auc:1.00000\tval-auc:0.72761\n",
            "[115]\ttrain-auc:1.00000\tval-auc:0.72791\n",
            "[116]\ttrain-auc:1.00000\tval-auc:0.72773\n",
            "[117]\ttrain-auc:1.00000\tval-auc:0.73064\n",
            "[118]\ttrain-auc:1.00000\tval-auc:0.73011\n",
            "[119]\ttrain-auc:1.00000\tval-auc:0.72936\n",
            "[120]\ttrain-auc:1.00000\tval-auc:0.72817\n",
            "[121]\ttrain-auc:1.00000\tval-auc:0.72920\n",
            "[122]\ttrain-auc:1.00000\tval-auc:0.72997\n",
            "[123]\ttrain-auc:1.00000\tval-auc:0.72943\n",
            "[124]\ttrain-auc:1.00000\tval-auc:0.73128\n",
            "[125]\ttrain-auc:1.00000\tval-auc:0.73115\n",
            "[126]\ttrain-auc:1.00000\tval-auc:0.73301\n",
            "[127]\ttrain-auc:1.00000\tval-auc:0.73306\n",
            "[128]\ttrain-auc:1.00000\tval-auc:0.73226\n",
            "[129]\ttrain-auc:1.00000\tval-auc:0.73214\n",
            "[130]\ttrain-auc:1.00000\tval-auc:0.73161\n",
            "[131]\ttrain-auc:1.00000\tval-auc:0.73217\n",
            "[132]\ttrain-auc:1.00000\tval-auc:0.73206\n",
            "[133]\ttrain-auc:1.00000\tval-auc:0.73213\n",
            "[134]\ttrain-auc:1.00000\tval-auc:0.73240\n",
            "[135]\ttrain-auc:1.00000\tval-auc:0.73247\n",
            "[136]\ttrain-auc:1.00000\tval-auc:0.73173\n",
            "[137]\ttrain-auc:1.00000\tval-auc:0.73091\n",
            "[138]\ttrain-auc:1.00000\tval-auc:0.72979\n",
            "[139]\ttrain-auc:1.00000\tval-auc:0.73081\n",
            "[140]\ttrain-auc:1.00000\tval-auc:0.73063\n",
            "[141]\ttrain-auc:1.00000\tval-auc:0.73016\n",
            "[142]\ttrain-auc:1.00000\tval-auc:0.72942\n",
            "[143]\ttrain-auc:1.00000\tval-auc:0.72907\n",
            "[144]\ttrain-auc:1.00000\tval-auc:0.72947\n",
            "[145]\ttrain-auc:1.00000\tval-auc:0.72993\n",
            "[146]\ttrain-auc:1.00000\tval-auc:0.72967\n",
            "[147]\ttrain-auc:1.00000\tval-auc:0.72916\n",
            "[148]\ttrain-auc:1.00000\tval-auc:0.72841\n",
            "[149]\ttrain-auc:1.00000\tval-auc:0.72828\n",
            "[150]\ttrain-auc:1.00000\tval-auc:0.72745\n",
            "[151]\ttrain-auc:1.00000\tval-auc:0.72743\n",
            "[152]\ttrain-auc:1.00000\tval-auc:0.72791\n",
            "[153]\ttrain-auc:1.00000\tval-auc:0.72807\n",
            "[154]\ttrain-auc:1.00000\tval-auc:0.72854\n",
            "[155]\ttrain-auc:1.00000\tval-auc:0.72970\n",
            "[156]\ttrain-auc:1.00000\tval-auc:0.73049\n",
            "[157]\ttrain-auc:1.00000\tval-auc:0.73200\n",
            "[158]\ttrain-auc:1.00000\tval-auc:0.73042\n",
            "[159]\ttrain-auc:1.00000\tval-auc:0.73026\n",
            "[160]\ttrain-auc:1.00000\tval-auc:0.73112\n",
            "[161]\ttrain-auc:1.00000\tval-auc:0.73121\n",
            "[162]\ttrain-auc:1.00000\tval-auc:0.73160\n",
            "[163]\ttrain-auc:1.00000\tval-auc:0.73139\n",
            "[164]\ttrain-auc:1.00000\tval-auc:0.73228\n",
            "[165]\ttrain-auc:1.00000\tval-auc:0.73193\n",
            "[166]\ttrain-auc:1.00000\tval-auc:0.73239\n",
            "[167]\ttrain-auc:1.00000\tval-auc:0.73232\n",
            "[168]\ttrain-auc:1.00000\tval-auc:0.73199\n",
            "[169]\ttrain-auc:1.00000\tval-auc:0.73295\n",
            "[170]\ttrain-auc:1.00000\tval-auc:0.73348\n",
            "[171]\ttrain-auc:1.00000\tval-auc:0.73333\n",
            "[172]\ttrain-auc:1.00000\tval-auc:0.73317\n",
            "[173]\ttrain-auc:1.00000\tval-auc:0.73328\n",
            "[174]\ttrain-auc:1.00000\tval-auc:0.73337\n",
            "[175]\ttrain-auc:1.00000\tval-auc:0.73367\n",
            "[176]\ttrain-auc:1.00000\tval-auc:0.73406\n",
            "[177]\ttrain-auc:1.00000\tval-auc:0.73547\n",
            "[178]\ttrain-auc:1.00000\tval-auc:0.73563\n",
            "[179]\ttrain-auc:1.00000\tval-auc:0.73523\n",
            "[180]\ttrain-auc:1.00000\tval-auc:0.73701\n",
            "[181]\ttrain-auc:1.00000\tval-auc:0.73711\n",
            "[182]\ttrain-auc:1.00000\tval-auc:0.73721\n",
            "[183]\ttrain-auc:1.00000\tval-auc:0.73720\n",
            "[184]\ttrain-auc:1.00000\tval-auc:0.73770\n",
            "[185]\ttrain-auc:1.00000\tval-auc:0.73795\n",
            "[186]\ttrain-auc:1.00000\tval-auc:0.73776\n",
            "[187]\ttrain-auc:1.00000\tval-auc:0.73732\n",
            "[188]\ttrain-auc:1.00000\tval-auc:0.73745\n",
            "[189]\ttrain-auc:1.00000\tval-auc:0.73757\n",
            "[190]\ttrain-auc:1.00000\tval-auc:0.73717\n",
            "[191]\ttrain-auc:1.00000\tval-auc:0.73608\n",
            "[192]\ttrain-auc:1.00000\tval-auc:0.73693\n",
            "[193]\ttrain-auc:1.00000\tval-auc:0.73711\n",
            "[194]\ttrain-auc:1.00000\tval-auc:0.73675\n",
            "[195]\ttrain-auc:1.00000\tval-auc:0.73718\n",
            "[196]\ttrain-auc:1.00000\tval-auc:0.73695\n",
            "[197]\ttrain-auc:1.00000\tval-auc:0.73694\n",
            "[198]\ttrain-auc:1.00000\tval-auc:0.73657\n",
            "[199]\ttrain-auc:1.00000\tval-auc:0.73583\n",
            "[200]\ttrain-auc:1.00000\tval-auc:0.73529\n",
            "[201]\ttrain-auc:1.00000\tval-auc:0.73546\n",
            "[202]\ttrain-auc:1.00000\tval-auc:0.73491\n",
            "[203]\ttrain-auc:1.00000\tval-auc:0.73502\n",
            "[204]\ttrain-auc:1.00000\tval-auc:0.73533\n",
            "[205]\ttrain-auc:1.00000\tval-auc:0.73555\n",
            "[206]\ttrain-auc:1.00000\tval-auc:0.73585\n",
            "[207]\ttrain-auc:1.00000\tval-auc:0.73623\n",
            "[208]\ttrain-auc:1.00000\tval-auc:0.73641\n",
            "[209]\ttrain-auc:1.00000\tval-auc:0.73588\n",
            "[210]\ttrain-auc:1.00000\tval-auc:0.73592\n",
            "[211]\ttrain-auc:1.00000\tval-auc:0.73598\n",
            "[212]\ttrain-auc:1.00000\tval-auc:0.73599\n",
            "[213]\ttrain-auc:1.00000\tval-auc:0.73647\n",
            "[214]\ttrain-auc:1.00000\tval-auc:0.73634\n",
            "[215]\ttrain-auc:1.00000\tval-auc:0.73650\n",
            "[216]\ttrain-auc:1.00000\tval-auc:0.73598\n",
            "[217]\ttrain-auc:1.00000\tval-auc:0.73568\n",
            "[218]\ttrain-auc:1.00000\tval-auc:0.73545\n",
            "[219]\ttrain-auc:1.00000\tval-auc:0.73482\n",
            "[220]\ttrain-auc:1.00000\tval-auc:0.73477\n",
            "[221]\ttrain-auc:1.00000\tval-auc:0.73497\n",
            "[222]\ttrain-auc:1.00000\tval-auc:0.73449\n",
            "[223]\ttrain-auc:1.00000\tval-auc:0.73393\n",
            "[224]\ttrain-auc:1.00000\tval-auc:0.73386\n",
            "[225]\ttrain-auc:1.00000\tval-auc:0.73352\n",
            "[226]\ttrain-auc:1.00000\tval-auc:0.73341\n",
            "[227]\ttrain-auc:1.00000\tval-auc:0.73318\n",
            "[228]\ttrain-auc:1.00000\tval-auc:0.73323\n",
            "[229]\ttrain-auc:1.00000\tval-auc:0.73271\n",
            "[230]\ttrain-auc:1.00000\tval-auc:0.73307\n",
            "[231]\ttrain-auc:1.00000\tval-auc:0.73296\n",
            "[232]\ttrain-auc:1.00000\tval-auc:0.73489\n",
            "[233]\ttrain-auc:1.00000\tval-auc:0.73527\n",
            "[234]\ttrain-auc:1.00000\tval-auc:0.73495\n",
            "[235]\ttrain-auc:1.00000\tval-auc:0.73584\n",
            "[236]\ttrain-auc:1.00000\tval-auc:0.73564\n",
            "[237]\ttrain-auc:1.00000\tval-auc:0.73546\n",
            "[238]\ttrain-auc:1.00000\tval-auc:0.73575\n",
            "[239]\ttrain-auc:1.00000\tval-auc:0.73680\n",
            "[240]\ttrain-auc:1.00000\tval-auc:0.73668\n",
            "[241]\ttrain-auc:1.00000\tval-auc:0.73648\n",
            "[242]\ttrain-auc:1.00000\tval-auc:0.73620\n",
            "[243]\ttrain-auc:1.00000\tval-auc:0.73586\n",
            "[244]\ttrain-auc:1.00000\tval-auc:0.73650\n",
            "[245]\ttrain-auc:1.00000\tval-auc:0.73631\n",
            "[246]\ttrain-auc:1.00000\tval-auc:0.73590\n",
            "[247]\ttrain-auc:1.00000\tval-auc:0.73586\n",
            "[248]\ttrain-auc:1.00000\tval-auc:0.73533\n",
            "[249]\ttrain-auc:1.00000\tval-auc:0.73544\n",
            "[250]\ttrain-auc:1.00000\tval-auc:0.73512\n",
            "[251]\ttrain-auc:1.00000\tval-auc:0.73557\n",
            "[252]\ttrain-auc:1.00000\tval-auc:0.73604\n",
            "[253]\ttrain-auc:1.00000\tval-auc:0.73620\n",
            "[254]\ttrain-auc:1.00000\tval-auc:0.73645\n",
            "[255]\ttrain-auc:1.00000\tval-auc:0.73632\n",
            "[256]\ttrain-auc:1.00000\tval-auc:0.73687\n",
            "[257]\ttrain-auc:1.00000\tval-auc:0.73653\n",
            "[258]\ttrain-auc:1.00000\tval-auc:0.73668\n",
            "[259]\ttrain-auc:1.00000\tval-auc:0.73667\n",
            "[260]\ttrain-auc:1.00000\tval-auc:0.73697\n",
            "[261]\ttrain-auc:1.00000\tval-auc:0.73712\n",
            "[262]\ttrain-auc:1.00000\tval-auc:0.73692\n",
            "[263]\ttrain-auc:1.00000\tval-auc:0.73634\n",
            "[264]\ttrain-auc:1.00000\tval-auc:0.73611\n",
            "[265]\ttrain-auc:1.00000\tval-auc:0.73626\n",
            "[266]\ttrain-auc:1.00000\tval-auc:0.73628\n",
            "[267]\ttrain-auc:1.00000\tval-auc:0.73582\n",
            "[268]\ttrain-auc:1.00000\tval-auc:0.73545\n",
            "[269]\ttrain-auc:1.00000\tval-auc:0.73561\n",
            "[270]\ttrain-auc:1.00000\tval-auc:0.73531\n",
            "[271]\ttrain-auc:1.00000\tval-auc:0.73547\n",
            "[272]\ttrain-auc:1.00000\tval-auc:0.73505\n",
            "[273]\ttrain-auc:1.00000\tval-auc:0.73531\n",
            "[274]\ttrain-auc:1.00000\tval-auc:0.73503\n",
            "[275]\ttrain-auc:1.00000\tval-auc:0.73513\n",
            "[276]\ttrain-auc:1.00000\tval-auc:0.73455\n",
            "[277]\ttrain-auc:1.00000\tval-auc:0.73414\n",
            "[278]\ttrain-auc:1.00000\tval-auc:0.73417\n",
            "[279]\ttrain-auc:1.00000\tval-auc:0.73460\n",
            "[280]\ttrain-auc:1.00000\tval-auc:0.73493\n",
            "[281]\ttrain-auc:1.00000\tval-auc:0.73446\n",
            "[282]\ttrain-auc:1.00000\tval-auc:0.73409\n",
            "[283]\ttrain-auc:1.00000\tval-auc:0.73418\n",
            "[284]\ttrain-auc:1.00000\tval-auc:0.73321\n",
            "[285]\ttrain-auc:1.00000\tval-auc:0.73354\n",
            "[286]\ttrain-auc:1.00000\tval-auc:0.73349\n",
            "[287]\ttrain-auc:1.00000\tval-auc:0.73329\n",
            "[288]\ttrain-auc:1.00000\tval-auc:0.73316\n",
            "[289]\ttrain-auc:1.00000\tval-auc:0.73398\n",
            "[290]\ttrain-auc:1.00000\tval-auc:0.73432\n",
            "[291]\ttrain-auc:1.00000\tval-auc:0.73522\n",
            "[292]\ttrain-auc:1.00000\tval-auc:0.73545\n",
            "[293]\ttrain-auc:1.00000\tval-auc:0.73728\n",
            "[294]\ttrain-auc:1.00000\tval-auc:0.73722\n",
            "[295]\ttrain-auc:1.00000\tval-auc:0.73685\n",
            "[296]\ttrain-auc:1.00000\tval-auc:0.73687\n",
            "[297]\ttrain-auc:1.00000\tval-auc:0.73632\n",
            "[298]\ttrain-auc:1.00000\tval-auc:0.73716\n",
            "[299]\ttrain-auc:1.00000\tval-auc:0.73721\n",
            "[300]\ttrain-auc:1.00000\tval-auc:0.73702\n",
            "[301]\ttrain-auc:1.00000\tval-auc:0.73690\n",
            "[302]\ttrain-auc:1.00000\tval-auc:0.73725\n",
            "[303]\ttrain-auc:1.00000\tval-auc:0.73731\n",
            "[304]\ttrain-auc:1.00000\tval-auc:0.73731\n",
            "[305]\ttrain-auc:1.00000\tval-auc:0.73742\n",
            "[306]\ttrain-auc:1.00000\tval-auc:0.73823\n",
            "[307]\ttrain-auc:1.00000\tval-auc:0.73810\n",
            "[308]\ttrain-auc:1.00000\tval-auc:0.73775\n",
            "[309]\ttrain-auc:1.00000\tval-auc:0.73753\n",
            "[310]\ttrain-auc:1.00000\tval-auc:0.73780\n",
            "[311]\ttrain-auc:1.00000\tval-auc:0.73787\n",
            "[312]\ttrain-auc:1.00000\tval-auc:0.73778\n",
            "[313]\ttrain-auc:1.00000\tval-auc:0.73812\n",
            "[314]\ttrain-auc:1.00000\tval-auc:0.73757\n",
            "[315]\ttrain-auc:1.00000\tval-auc:0.73772\n",
            "[316]\ttrain-auc:1.00000\tval-auc:0.73792\n",
            "[317]\ttrain-auc:1.00000\tval-auc:0.73784\n",
            "[318]\ttrain-auc:1.00000\tval-auc:0.73766\n",
            "[319]\ttrain-auc:1.00000\tval-auc:0.73764\n",
            "[320]\ttrain-auc:1.00000\tval-auc:0.73709\n",
            "[321]\ttrain-auc:1.00000\tval-auc:0.73699\n",
            "[322]\ttrain-auc:1.00000\tval-auc:0.73709\n",
            "[323]\ttrain-auc:1.00000\tval-auc:0.73675\n",
            "[324]\ttrain-auc:1.00000\tval-auc:0.73680\n",
            "[325]\ttrain-auc:1.00000\tval-auc:0.73686\n",
            "[326]\ttrain-auc:1.00000\tval-auc:0.73712\n",
            "[327]\ttrain-auc:1.00000\tval-auc:0.73694\n",
            "[328]\ttrain-auc:1.00000\tval-auc:0.73678\n",
            "[329]\ttrain-auc:1.00000\tval-auc:0.73668\n",
            "[330]\ttrain-auc:1.00000\tval-auc:0.73675\n",
            "[331]\ttrain-auc:1.00000\tval-auc:0.73724\n",
            "[332]\ttrain-auc:1.00000\tval-auc:0.73709\n",
            "[333]\ttrain-auc:1.00000\tval-auc:0.73689\n",
            "[334]\ttrain-auc:1.00000\tval-auc:0.73677\n",
            "[335]\ttrain-auc:1.00000\tval-auc:0.73701\n",
            "[336]\ttrain-auc:1.00000\tval-auc:0.73681\n",
            "[337]\ttrain-auc:1.00000\tval-auc:0.73659\n",
            "[338]\ttrain-auc:1.00000\tval-auc:0.73681\n",
            "[339]\ttrain-auc:1.00000\tval-auc:0.73661\n",
            "[340]\ttrain-auc:1.00000\tval-auc:0.73703\n",
            "[341]\ttrain-auc:1.00000\tval-auc:0.73690\n",
            "[342]\ttrain-auc:1.00000\tval-auc:0.73690\n",
            "[343]\ttrain-auc:1.00000\tval-auc:0.73741\n",
            "[344]\ttrain-auc:1.00000\tval-auc:0.73677\n",
            "[345]\ttrain-auc:1.00000\tval-auc:0.73662\n",
            "[346]\ttrain-auc:1.00000\tval-auc:0.73667\n",
            "[347]\ttrain-auc:1.00000\tval-auc:0.73679\n",
            "[348]\ttrain-auc:1.00000\tval-auc:0.73672\n",
            "[349]\ttrain-auc:1.00000\tval-auc:0.73677\n",
            "[350]\ttrain-auc:1.00000\tval-auc:0.73618\n",
            "[351]\ttrain-auc:1.00000\tval-auc:0.73668\n",
            "[352]\ttrain-auc:1.00000\tval-auc:0.73663\n",
            "[353]\ttrain-auc:1.00000\tval-auc:0.73677\n",
            "[354]\ttrain-auc:1.00000\tval-auc:0.73731\n",
            "[355]\ttrain-auc:1.00000\tval-auc:0.73752\n",
            "[356]\ttrain-auc:1.00000\tval-auc:0.73776\n",
            "[357]\ttrain-auc:1.00000\tval-auc:0.73800\n",
            "[358]\ttrain-auc:1.00000\tval-auc:0.73795\n",
            "[359]\ttrain-auc:1.00000\tval-auc:0.73789\n",
            "[360]\ttrain-auc:1.00000\tval-auc:0.73791\n",
            "[361]\ttrain-auc:1.00000\tval-auc:0.73780\n",
            "[362]\ttrain-auc:1.00000\tval-auc:0.73763\n",
            "[363]\ttrain-auc:1.00000\tval-auc:0.73743\n",
            "[364]\ttrain-auc:1.00000\tval-auc:0.73748\n",
            "[365]\ttrain-auc:1.00000\tval-auc:0.73786\n",
            "[366]\ttrain-auc:1.00000\tval-auc:0.73800\n",
            "[367]\ttrain-auc:1.00000\tval-auc:0.73802\n",
            "[368]\ttrain-auc:1.00000\tval-auc:0.73814\n",
            "[369]\ttrain-auc:1.00000\tval-auc:0.73822\n",
            "[370]\ttrain-auc:1.00000\tval-auc:0.73872\n",
            "[371]\ttrain-auc:1.00000\tval-auc:0.73812\n",
            "[372]\ttrain-auc:1.00000\tval-auc:0.73780\n",
            "[373]\ttrain-auc:1.00000\tval-auc:0.73783\n",
            "[374]\ttrain-auc:1.00000\tval-auc:0.73796\n",
            "[375]\ttrain-auc:1.00000\tval-auc:0.73809\n",
            "[376]\ttrain-auc:1.00000\tval-auc:0.73765\n",
            "[377]\ttrain-auc:1.00000\tval-auc:0.73735\n",
            "[378]\ttrain-auc:1.00000\tval-auc:0.73739\n",
            "[379]\ttrain-auc:1.00000\tval-auc:0.73807\n",
            "[380]\ttrain-auc:1.00000\tval-auc:0.73821\n",
            "[381]\ttrain-auc:1.00000\tval-auc:0.73778\n",
            "[382]\ttrain-auc:1.00000\tval-auc:0.73829\n",
            "[383]\ttrain-auc:1.00000\tval-auc:0.73846\n",
            "[384]\ttrain-auc:1.00000\tval-auc:0.73865\n",
            "[385]\ttrain-auc:1.00000\tval-auc:0.73855\n",
            "[386]\ttrain-auc:1.00000\tval-auc:0.73883\n",
            "[387]\ttrain-auc:1.00000\tval-auc:0.73867\n",
            "[388]\ttrain-auc:1.00000\tval-auc:0.73853\n",
            "[389]\ttrain-auc:1.00000\tval-auc:0.73849\n",
            "[390]\ttrain-auc:1.00000\tval-auc:0.73831\n",
            "[391]\ttrain-auc:1.00000\tval-auc:0.73836\n",
            "[392]\ttrain-auc:1.00000\tval-auc:0.73841\n",
            "[393]\ttrain-auc:1.00000\tval-auc:0.73866\n",
            "[394]\ttrain-auc:1.00000\tval-auc:0.73878\n",
            "[395]\ttrain-auc:1.00000\tval-auc:0.73876\n",
            "[396]\ttrain-auc:1.00000\tval-auc:0.73869\n",
            "[397]\ttrain-auc:1.00000\tval-auc:0.73835\n",
            "[398]\ttrain-auc:1.00000\tval-auc:0.73826\n",
            "[399]\ttrain-auc:1.00000\tval-auc:0.73836\n",
            "[400]\ttrain-auc:1.00000\tval-auc:0.73821\n",
            "[401]\ttrain-auc:1.00000\tval-auc:0.73821\n",
            "[402]\ttrain-auc:1.00000\tval-auc:0.73829\n",
            "[403]\ttrain-auc:1.00000\tval-auc:0.73840\n",
            "[404]\ttrain-auc:1.00000\tval-auc:0.73858\n",
            "[405]\ttrain-auc:1.00000\tval-auc:0.73843\n",
            "[406]\ttrain-auc:1.00000\tval-auc:0.73813\n",
            "[407]\ttrain-auc:1.00000\tval-auc:0.73802\n",
            "[408]\ttrain-auc:1.00000\tval-auc:0.73799\n",
            "[409]\ttrain-auc:1.00000\tval-auc:0.73812\n",
            "[410]\ttrain-auc:1.00000\tval-auc:0.73839\n",
            "[411]\ttrain-auc:1.00000\tval-auc:0.73852\n",
            "[412]\ttrain-auc:1.00000\tval-auc:0.73860\n",
            "[413]\ttrain-auc:1.00000\tval-auc:0.73836\n",
            "[414]\ttrain-auc:1.00000\tval-auc:0.73843\n",
            "[415]\ttrain-auc:1.00000\tval-auc:0.73851\n",
            "[416]\ttrain-auc:1.00000\tval-auc:0.73867\n",
            "[417]\ttrain-auc:1.00000\tval-auc:0.73899\n",
            "[418]\ttrain-auc:1.00000\tval-auc:0.73917\n",
            "[419]\ttrain-auc:1.00000\tval-auc:0.73910\n",
            "[420]\ttrain-auc:1.00000\tval-auc:0.73894\n",
            "[421]\ttrain-auc:1.00000\tval-auc:0.73922\n",
            "[422]\ttrain-auc:1.00000\tval-auc:0.73939\n",
            "[423]\ttrain-auc:1.00000\tval-auc:0.73916\n",
            "[424]\ttrain-auc:1.00000\tval-auc:0.73936\n",
            "[425]\ttrain-auc:1.00000\tval-auc:0.73911\n",
            "[426]\ttrain-auc:1.00000\tval-auc:0.73900\n",
            "[427]\ttrain-auc:1.00000\tval-auc:0.73954\n",
            "[428]\ttrain-auc:1.00000\tval-auc:0.73921\n",
            "[429]\ttrain-auc:1.00000\tval-auc:0.73910\n",
            "[430]\ttrain-auc:1.00000\tval-auc:0.73889\n",
            "[431]\ttrain-auc:1.00000\tval-auc:0.73875\n",
            "[432]\ttrain-auc:1.00000\tval-auc:0.73840\n",
            "[433]\ttrain-auc:1.00000\tval-auc:0.73840\n",
            "[434]\ttrain-auc:1.00000\tval-auc:0.73815\n",
            "[435]\ttrain-auc:1.00000\tval-auc:0.73794\n",
            "[436]\ttrain-auc:1.00000\tval-auc:0.73839\n",
            "[437]\ttrain-auc:1.00000\tval-auc:0.73861\n",
            "[438]\ttrain-auc:1.00000\tval-auc:0.73852\n",
            "[439]\ttrain-auc:1.00000\tval-auc:0.73880\n",
            "[440]\ttrain-auc:1.00000\tval-auc:0.73875\n",
            "[441]\ttrain-auc:1.00000\tval-auc:0.73847\n",
            "[442]\ttrain-auc:1.00000\tval-auc:0.73849\n",
            "[443]\ttrain-auc:1.00000\tval-auc:0.73871\n",
            "[444]\ttrain-auc:1.00000\tval-auc:0.73874\n",
            "[445]\ttrain-auc:1.00000\tval-auc:0.73892\n",
            "[446]\ttrain-auc:1.00000\tval-auc:0.73904\n",
            "[447]\ttrain-auc:1.00000\tval-auc:0.73915\n",
            "[448]\ttrain-auc:1.00000\tval-auc:0.73913\n",
            "[449]\ttrain-auc:1.00000\tval-auc:0.73924\n",
            "[450]\ttrain-auc:1.00000\tval-auc:0.73968\n",
            "[451]\ttrain-auc:1.00000\tval-auc:0.73991\n",
            "[452]\ttrain-auc:1.00000\tval-auc:0.74009\n",
            "[453]\ttrain-auc:1.00000\tval-auc:0.73995\n",
            "[454]\ttrain-auc:1.00000\tval-auc:0.73977\n",
            "[455]\ttrain-auc:1.00000\tval-auc:0.73966\n",
            "[456]\ttrain-auc:1.00000\tval-auc:0.74006\n",
            "[457]\ttrain-auc:1.00000\tval-auc:0.73965\n",
            "[458]\ttrain-auc:1.00000\tval-auc:0.73966\n",
            "[459]\ttrain-auc:1.00000\tval-auc:0.73974\n",
            "[460]\ttrain-auc:1.00000\tval-auc:0.73991\n",
            "[461]\ttrain-auc:1.00000\tval-auc:0.73959\n",
            "[462]\ttrain-auc:1.00000\tval-auc:0.73920\n",
            "[463]\ttrain-auc:1.00000\tval-auc:0.73913\n",
            "[464]\ttrain-auc:1.00000\tval-auc:0.73916\n",
            "[465]\ttrain-auc:1.00000\tval-auc:0.73918\n",
            "[466]\ttrain-auc:1.00000\tval-auc:0.73968\n",
            "[467]\ttrain-auc:1.00000\tval-auc:0.73951\n",
            "[468]\ttrain-auc:1.00000\tval-auc:0.73941\n",
            "[469]\ttrain-auc:1.00000\tval-auc:0.73927\n",
            "[470]\ttrain-auc:1.00000\tval-auc:0.73990\n",
            "[471]\ttrain-auc:1.00000\tval-auc:0.73996\n",
            "[472]\ttrain-auc:1.00000\tval-auc:0.74017\n",
            "[473]\ttrain-auc:1.00000\tval-auc:0.74041\n",
            "[474]\ttrain-auc:1.00000\tval-auc:0.74029\n",
            "[475]\ttrain-auc:1.00000\tval-auc:0.74021\n",
            "[476]\ttrain-auc:1.00000\tval-auc:0.74011\n",
            "[477]\ttrain-auc:1.00000\tval-auc:0.73984\n",
            "[478]\ttrain-auc:1.00000\tval-auc:0.73991\n",
            "[479]\ttrain-auc:1.00000\tval-auc:0.73970\n",
            "[480]\ttrain-auc:1.00000\tval-auc:0.73970\n",
            "[481]\ttrain-auc:1.00000\tval-auc:0.73955\n",
            "[482]\ttrain-auc:1.00000\tval-auc:0.73977\n",
            "[483]\ttrain-auc:1.00000\tval-auc:0.73994\n",
            "[484]\ttrain-auc:1.00000\tval-auc:0.73992\n",
            "[485]\ttrain-auc:1.00000\tval-auc:0.73972\n",
            "[486]\ttrain-auc:1.00000\tval-auc:0.73966\n",
            "[487]\ttrain-auc:1.00000\tval-auc:0.73959\n",
            "[488]\ttrain-auc:1.00000\tval-auc:0.73922\n",
            "[489]\ttrain-auc:1.00000\tval-auc:0.73940\n",
            "[490]\ttrain-auc:1.00000\tval-auc:0.73938\n",
            "[491]\ttrain-auc:1.00000\tval-auc:0.73926\n",
            "[492]\ttrain-auc:1.00000\tval-auc:0.73904\n",
            "[493]\ttrain-auc:1.00000\tval-auc:0.73895\n",
            "[494]\ttrain-auc:1.00000\tval-auc:0.73896\n",
            "[495]\ttrain-auc:1.00000\tval-auc:0.73892\n",
            "[496]\ttrain-auc:1.00000\tval-auc:0.73863\n",
            "[497]\ttrain-auc:1.00000\tval-auc:0.73863\n",
            "[498]\ttrain-auc:1.00000\tval-auc:0.73842\n",
            "[499]\ttrain-auc:1.00000\tval-auc:0.73841\n",
            "[500]\ttrain-auc:1.00000\tval-auc:0.73854\n",
            "[501]\ttrain-auc:1.00000\tval-auc:0.73839\n",
            "[502]\ttrain-auc:1.00000\tval-auc:0.73847\n",
            "[503]\ttrain-auc:1.00000\tval-auc:0.73860\n",
            "[504]\ttrain-auc:1.00000\tval-auc:0.73856\n",
            "[505]\ttrain-auc:1.00000\tval-auc:0.73833\n",
            "[506]\ttrain-auc:1.00000\tval-auc:0.73801\n",
            "[507]\ttrain-auc:1.00000\tval-auc:0.73802\n",
            "[508]\ttrain-auc:1.00000\tval-auc:0.73798\n",
            "[509]\ttrain-auc:1.00000\tval-auc:0.73792\n",
            "[510]\ttrain-auc:1.00000\tval-auc:0.73814\n",
            "[511]\ttrain-auc:1.00000\tval-auc:0.73806\n",
            "[512]\ttrain-auc:1.00000\tval-auc:0.73825\n",
            "[513]\ttrain-auc:1.00000\tval-auc:0.73810\n",
            "[514]\ttrain-auc:1.00000\tval-auc:0.73818\n",
            "[515]\ttrain-auc:1.00000\tval-auc:0.73824\n",
            "[516]\ttrain-auc:1.00000\tval-auc:0.73786\n",
            "[517]\ttrain-auc:1.00000\tval-auc:0.73769\n",
            "[518]\ttrain-auc:1.00000\tval-auc:0.73760\n",
            "[519]\ttrain-auc:1.00000\tval-auc:0.73765\n",
            "[520]\ttrain-auc:1.00000\tval-auc:0.73734\n",
            "[521]\ttrain-auc:1.00000\tval-auc:0.73739\n",
            "[522]\ttrain-auc:1.00000\tval-auc:0.73739\n",
            "[523]\ttrain-auc:1.00000\tval-auc:0.73719\n",
            "[524]\ttrain-auc:1.00000\tval-auc:0.73765\n",
            "[525]\ttrain-auc:1.00000\tval-auc:0.73784\n",
            "[526]\ttrain-auc:1.00000\tval-auc:0.73793\n",
            "[527]\ttrain-auc:1.00000\tval-auc:0.73802\n",
            "[528]\ttrain-auc:1.00000\tval-auc:0.73755\n",
            "[529]\ttrain-auc:1.00000\tval-auc:0.73775\n",
            "[530]\ttrain-auc:1.00000\tval-auc:0.73783\n",
            "[531]\ttrain-auc:1.00000\tval-auc:0.73776\n",
            "[532]\ttrain-auc:1.00000\tval-auc:0.73768\n",
            "[533]\ttrain-auc:1.00000\tval-auc:0.73761\n",
            "[534]\ttrain-auc:1.00000\tval-auc:0.73786\n",
            "[535]\ttrain-auc:1.00000\tval-auc:0.73802\n",
            "[536]\ttrain-auc:1.00000\tval-auc:0.73790\n",
            "[537]\ttrain-auc:1.00000\tval-auc:0.73778\n",
            "[538]\ttrain-auc:1.00000\tval-auc:0.73781\n",
            "[539]\ttrain-auc:1.00000\tval-auc:0.73790\n",
            "[540]\ttrain-auc:1.00000\tval-auc:0.73776\n",
            "[541]\ttrain-auc:1.00000\tval-auc:0.73771\n",
            "[542]\ttrain-auc:1.00000\tval-auc:0.73769\n",
            "[543]\ttrain-auc:1.00000\tval-auc:0.73794\n",
            "[544]\ttrain-auc:1.00000\tval-auc:0.73765\n",
            "[545]\ttrain-auc:1.00000\tval-auc:0.73769\n",
            "[546]\ttrain-auc:1.00000\tval-auc:0.73757\n",
            "[547]\ttrain-auc:1.00000\tval-auc:0.73729\n",
            "[548]\ttrain-auc:1.00000\tval-auc:0.73717\n",
            "[549]\ttrain-auc:1.00000\tval-auc:0.73712\n",
            "[550]\ttrain-auc:1.00000\tval-auc:0.73707\n",
            "[551]\ttrain-auc:1.00000\tval-auc:0.73725\n",
            "[552]\ttrain-auc:1.00000\tval-auc:0.73709\n",
            "[553]\ttrain-auc:1.00000\tval-auc:0.73705\n",
            "[554]\ttrain-auc:1.00000\tval-auc:0.73679\n",
            "[555]\ttrain-auc:1.00000\tval-auc:0.73695\n",
            "[556]\ttrain-auc:1.00000\tval-auc:0.73677\n",
            "[557]\ttrain-auc:1.00000\tval-auc:0.73681\n",
            "[558]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[559]\ttrain-auc:1.00000\tval-auc:0.73661\n",
            "[560]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[561]\ttrain-auc:1.00000\tval-auc:0.73666\n",
            "[562]\ttrain-auc:1.00000\tval-auc:0.73677\n",
            "[563]\ttrain-auc:1.00000\tval-auc:0.73707\n",
            "[564]\ttrain-auc:1.00000\tval-auc:0.73723\n",
            "[565]\ttrain-auc:1.00000\tval-auc:0.73734\n",
            "[566]\ttrain-auc:1.00000\tval-auc:0.73740\n",
            "[567]\ttrain-auc:1.00000\tval-auc:0.73744\n",
            "[568]\ttrain-auc:1.00000\tval-auc:0.73731\n",
            "[569]\ttrain-auc:1.00000\tval-auc:0.73741\n",
            "[570]\ttrain-auc:1.00000\tval-auc:0.73734\n",
            "[571]\ttrain-auc:1.00000\tval-auc:0.73775\n",
            "[572]\ttrain-auc:1.00000\tval-auc:0.73773\n",
            "[573]\ttrain-auc:1.00000\tval-auc:0.73774\n",
            "[574]\ttrain-auc:1.00000\tval-auc:0.73789\n",
            "[575]\ttrain-auc:1.00000\tval-auc:0.73786\n",
            "[576]\ttrain-auc:1.00000\tval-auc:0.73775\n",
            "[577]\ttrain-auc:1.00000\tval-auc:0.73768\n",
            "[578]\ttrain-auc:1.00000\tval-auc:0.73771\n",
            "[579]\ttrain-auc:1.00000\tval-auc:0.73775\n",
            "[580]\ttrain-auc:1.00000\tval-auc:0.73770\n",
            "[581]\ttrain-auc:1.00000\tval-auc:0.73769\n",
            "[582]\ttrain-auc:1.00000\tval-auc:0.73758\n",
            "[583]\ttrain-auc:1.00000\tval-auc:0.73767\n",
            "[584]\ttrain-auc:1.00000\tval-auc:0.73749\n",
            "[585]\ttrain-auc:1.00000\tval-auc:0.73765\n",
            "[586]\ttrain-auc:1.00000\tval-auc:0.73763\n",
            "[587]\ttrain-auc:1.00000\tval-auc:0.73758\n",
            "[588]\ttrain-auc:1.00000\tval-auc:0.73763\n",
            "[589]\ttrain-auc:1.00000\tval-auc:0.73772\n",
            "[590]\ttrain-auc:1.00000\tval-auc:0.73752\n",
            "[591]\ttrain-auc:1.00000\tval-auc:0.73754\n",
            "[592]\ttrain-auc:1.00000\tval-auc:0.73759\n",
            "[593]\ttrain-auc:1.00000\tval-auc:0.73764\n",
            "[594]\ttrain-auc:1.00000\tval-auc:0.73735\n",
            "[595]\ttrain-auc:1.00000\tval-auc:0.73723\n",
            "[596]\ttrain-auc:1.00000\tval-auc:0.73731\n",
            "[597]\ttrain-auc:1.00000\tval-auc:0.73749\n",
            "[598]\ttrain-auc:1.00000\tval-auc:0.73748\n",
            "[599]\ttrain-auc:1.00000\tval-auc:0.73749\n",
            "[600]\ttrain-auc:1.00000\tval-auc:0.73773\n",
            "[601]\ttrain-auc:1.00000\tval-auc:0.73764\n",
            "[602]\ttrain-auc:1.00000\tval-auc:0.73778\n",
            "[603]\ttrain-auc:1.00000\tval-auc:0.73777\n",
            "[604]\ttrain-auc:1.00000\tval-auc:0.73778\n",
            "[605]\ttrain-auc:1.00000\tval-auc:0.73772\n",
            "[606]\ttrain-auc:1.00000\tval-auc:0.73774\n",
            "[607]\ttrain-auc:1.00000\tval-auc:0.73786\n",
            "[608]\ttrain-auc:1.00000\tval-auc:0.73793\n",
            "[609]\ttrain-auc:1.00000\tval-auc:0.73816\n",
            "[610]\ttrain-auc:1.00000\tval-auc:0.73810\n",
            "[611]\ttrain-auc:1.00000\tval-auc:0.73797\n",
            "[612]\ttrain-auc:1.00000\tval-auc:0.73804\n",
            "[613]\ttrain-auc:1.00000\tval-auc:0.73794\n",
            "[614]\ttrain-auc:1.00000\tval-auc:0.73804\n",
            "[615]\ttrain-auc:1.00000\tval-auc:0.73809\n",
            "[616]\ttrain-auc:1.00000\tval-auc:0.73792\n",
            "[617]\ttrain-auc:1.00000\tval-auc:0.73792\n",
            "[618]\ttrain-auc:1.00000\tval-auc:0.73780\n",
            "[619]\ttrain-auc:1.00000\tval-auc:0.73768\n",
            "[620]\ttrain-auc:1.00000\tval-auc:0.73789\n",
            "[621]\ttrain-auc:1.00000\tval-auc:0.73797\n",
            "[622]\ttrain-auc:1.00000\tval-auc:0.73791\n",
            "[623]\ttrain-auc:1.00000\tval-auc:0.73798\n",
            "[624]\ttrain-auc:1.00000\tval-auc:0.73783\n",
            "[625]\ttrain-auc:1.00000\tval-auc:0.73784\n",
            "[626]\ttrain-auc:1.00000\tval-auc:0.73814\n",
            "[627]\ttrain-auc:1.00000\tval-auc:0.73799\n",
            "[628]\ttrain-auc:1.00000\tval-auc:0.73810\n",
            "[629]\ttrain-auc:1.00000\tval-auc:0.73813\n",
            "[630]\ttrain-auc:1.00000\tval-auc:0.73798\n",
            "[631]\ttrain-auc:1.00000\tval-auc:0.73814\n",
            "[632]\ttrain-auc:1.00000\tval-auc:0.73817\n",
            "[633]\ttrain-auc:1.00000\tval-auc:0.73805\n",
            "[634]\ttrain-auc:1.00000\tval-auc:0.73810\n",
            "[635]\ttrain-auc:1.00000\tval-auc:0.73803\n",
            "[636]\ttrain-auc:1.00000\tval-auc:0.73798\n",
            "[637]\ttrain-auc:1.00000\tval-auc:0.73801\n",
            "[638]\ttrain-auc:1.00000\tval-auc:0.73803\n",
            "[639]\ttrain-auc:1.00000\tval-auc:0.73795\n",
            "[640]\ttrain-auc:1.00000\tval-auc:0.73808\n",
            "[641]\ttrain-auc:1.00000\tval-auc:0.73791\n",
            "[642]\ttrain-auc:1.00000\tval-auc:0.73791\n",
            "[643]\ttrain-auc:1.00000\tval-auc:0.73774\n",
            "[644]\ttrain-auc:1.00000\tval-auc:0.73805\n",
            "[645]\ttrain-auc:1.00000\tval-auc:0.73793\n",
            "[646]\ttrain-auc:1.00000\tval-auc:0.73792\n",
            "[647]\ttrain-auc:1.00000\tval-auc:0.73798\n",
            "[648]\ttrain-auc:1.00000\tval-auc:0.73786\n",
            "[649]\ttrain-auc:1.00000\tval-auc:0.73769\n",
            "[650]\ttrain-auc:1.00000\tval-auc:0.73743\n",
            "[651]\ttrain-auc:1.00000\tval-auc:0.73756\n",
            "[652]\ttrain-auc:1.00000\tval-auc:0.73738\n",
            "[653]\ttrain-auc:1.00000\tval-auc:0.73754\n",
            "[654]\ttrain-auc:1.00000\tval-auc:0.73762\n",
            "[655]\ttrain-auc:1.00000\tval-auc:0.73772\n",
            "[656]\ttrain-auc:1.00000\tval-auc:0.73771\n",
            "[657]\ttrain-auc:1.00000\tval-auc:0.73749\n",
            "[658]\ttrain-auc:1.00000\tval-auc:0.73736\n",
            "[659]\ttrain-auc:1.00000\tval-auc:0.73756\n",
            "[660]\ttrain-auc:1.00000\tval-auc:0.73760\n",
            "[661]\ttrain-auc:1.00000\tval-auc:0.73765\n",
            "[662]\ttrain-auc:1.00000\tval-auc:0.73780\n",
            "[663]\ttrain-auc:1.00000\tval-auc:0.73769\n",
            "[664]\ttrain-auc:1.00000\tval-auc:0.73766\n",
            "[665]\ttrain-auc:1.00000\tval-auc:0.73771\n",
            "[666]\ttrain-auc:1.00000\tval-auc:0.73757\n",
            "[667]\ttrain-auc:1.00000\tval-auc:0.73734\n",
            "[668]\ttrain-auc:1.00000\tval-auc:0.73744\n",
            "[669]\ttrain-auc:1.00000\tval-auc:0.73749\n",
            "[670]\ttrain-auc:1.00000\tval-auc:0.73757\n",
            "[671]\ttrain-auc:1.00000\tval-auc:0.73746\n",
            "[672]\ttrain-auc:1.00000\tval-auc:0.73750\n",
            "[673]\ttrain-auc:1.00000\tval-auc:0.73752\n",
            "[674]\ttrain-auc:1.00000\tval-auc:0.73747\n",
            "[675]\ttrain-auc:1.00000\tval-auc:0.73753\n",
            "[676]\ttrain-auc:1.00000\tval-auc:0.73748\n",
            "[677]\ttrain-auc:1.00000\tval-auc:0.73763\n",
            "[678]\ttrain-auc:1.00000\tval-auc:0.73748\n",
            "[679]\ttrain-auc:1.00000\tval-auc:0.73766\n",
            "[680]\ttrain-auc:1.00000\tval-auc:0.73764\n",
            "[681]\ttrain-auc:1.00000\tval-auc:0.73764\n",
            "[682]\ttrain-auc:1.00000\tval-auc:0.73750\n",
            "[683]\ttrain-auc:1.00000\tval-auc:0.73754\n",
            "[684]\ttrain-auc:1.00000\tval-auc:0.73749\n",
            "[685]\ttrain-auc:1.00000\tval-auc:0.73738\n",
            "[686]\ttrain-auc:1.00000\tval-auc:0.73735\n",
            "[687]\ttrain-auc:1.00000\tval-auc:0.73733\n",
            "[688]\ttrain-auc:1.00000\tval-auc:0.73745\n",
            "[689]\ttrain-auc:1.00000\tval-auc:0.73742\n",
            "[690]\ttrain-auc:1.00000\tval-auc:0.73729\n",
            "[691]\ttrain-auc:1.00000\tval-auc:0.73733\n",
            "[692]\ttrain-auc:1.00000\tval-auc:0.73720\n",
            "[693]\ttrain-auc:1.00000\tval-auc:0.73732\n",
            "[694]\ttrain-auc:1.00000\tval-auc:0.73713\n",
            "[695]\ttrain-auc:1.00000\tval-auc:0.73706\n",
            "[696]\ttrain-auc:1.00000\tval-auc:0.73715\n",
            "[697]\ttrain-auc:1.00000\tval-auc:0.73730\n",
            "[698]\ttrain-auc:1.00000\tval-auc:0.73723\n",
            "[699]\ttrain-auc:1.00000\tval-auc:0.73736\n",
            "[700]\ttrain-auc:1.00000\tval-auc:0.73749\n",
            "[701]\ttrain-auc:1.00000\tval-auc:0.73745\n",
            "[702]\ttrain-auc:1.00000\tval-auc:0.73741\n",
            "[703]\ttrain-auc:1.00000\tval-auc:0.73755\n",
            "[704]\ttrain-auc:1.00000\tval-auc:0.73755\n",
            "[705]\ttrain-auc:1.00000\tval-auc:0.73756\n",
            "[706]\ttrain-auc:1.00000\tval-auc:0.73758\n",
            "[707]\ttrain-auc:1.00000\tval-auc:0.73755\n",
            "[708]\ttrain-auc:1.00000\tval-auc:0.73753\n",
            "[709]\ttrain-auc:1.00000\tval-auc:0.73750\n",
            "[710]\ttrain-auc:1.00000\tval-auc:0.73745\n",
            "[711]\ttrain-auc:1.00000\tval-auc:0.73721\n",
            "[712]\ttrain-auc:1.00000\tval-auc:0.73714\n",
            "[713]\ttrain-auc:1.00000\tval-auc:0.73720\n",
            "[714]\ttrain-auc:1.00000\tval-auc:0.73699\n",
            "[715]\ttrain-auc:1.00000\tval-auc:0.73699\n",
            "[716]\ttrain-auc:1.00000\tval-auc:0.73707\n",
            "[717]\ttrain-auc:1.00000\tval-auc:0.73738\n",
            "[718]\ttrain-auc:1.00000\tval-auc:0.73740\n",
            "[719]\ttrain-auc:1.00000\tval-auc:0.73742\n",
            "[720]\ttrain-auc:1.00000\tval-auc:0.73750\n",
            "[721]\ttrain-auc:1.00000\tval-auc:0.73738\n",
            "[722]\ttrain-auc:1.00000\tval-auc:0.73746\n",
            "[723]\ttrain-auc:1.00000\tval-auc:0.73748\n",
            "[724]\ttrain-auc:1.00000\tval-auc:0.73729\n",
            "[725]\ttrain-auc:1.00000\tval-auc:0.73719\n",
            "[726]\ttrain-auc:1.00000\tval-auc:0.73708\n",
            "[727]\ttrain-auc:1.00000\tval-auc:0.73707\n",
            "[728]\ttrain-auc:1.00000\tval-auc:0.73715\n",
            "[729]\ttrain-auc:1.00000\tval-auc:0.73746\n",
            "[730]\ttrain-auc:1.00000\tval-auc:0.73754\n",
            "[731]\ttrain-auc:1.00000\tval-auc:0.73748\n",
            "[732]\ttrain-auc:1.00000\tval-auc:0.73736\n",
            "[733]\ttrain-auc:1.00000\tval-auc:0.73741\n",
            "[734]\ttrain-auc:1.00000\tval-auc:0.73750\n",
            "[735]\ttrain-auc:1.00000\tval-auc:0.73784\n",
            "[736]\ttrain-auc:1.00000\tval-auc:0.73776\n",
            "[737]\ttrain-auc:1.00000\tval-auc:0.73767\n",
            "[738]\ttrain-auc:1.00000\tval-auc:0.73766\n",
            "[739]\ttrain-auc:1.00000\tval-auc:0.73768\n",
            "[740]\ttrain-auc:1.00000\tval-auc:0.73767\n",
            "[741]\ttrain-auc:1.00000\tval-auc:0.73767\n",
            "[742]\ttrain-auc:1.00000\tval-auc:0.73752\n",
            "[743]\ttrain-auc:1.00000\tval-auc:0.73756\n",
            "[744]\ttrain-auc:1.00000\tval-auc:0.73749\n",
            "[745]\ttrain-auc:1.00000\tval-auc:0.73731\n",
            "[746]\ttrain-auc:1.00000\tval-auc:0.73750\n",
            "[747]\ttrain-auc:1.00000\tval-auc:0.73751\n",
            "[748]\ttrain-auc:1.00000\tval-auc:0.73742\n",
            "[749]\ttrain-auc:1.00000\tval-auc:0.73730\n",
            "[750]\ttrain-auc:1.00000\tval-auc:0.73723\n",
            "[751]\ttrain-auc:1.00000\tval-auc:0.73719\n",
            "[752]\ttrain-auc:1.00000\tval-auc:0.73713\n",
            "[753]\ttrain-auc:1.00000\tval-auc:0.73725\n",
            "[754]\ttrain-auc:1.00000\tval-auc:0.73732\n",
            "[755]\ttrain-auc:1.00000\tval-auc:0.73731\n",
            "[756]\ttrain-auc:1.00000\tval-auc:0.73722\n",
            "[757]\ttrain-auc:1.00000\tval-auc:0.73724\n",
            "[758]\ttrain-auc:1.00000\tval-auc:0.73707\n",
            "[759]\ttrain-auc:1.00000\tval-auc:0.73708\n",
            "[760]\ttrain-auc:1.00000\tval-auc:0.73697\n",
            "[761]\ttrain-auc:1.00000\tval-auc:0.73673\n",
            "[762]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[763]\ttrain-auc:1.00000\tval-auc:0.73688\n",
            "[764]\ttrain-auc:1.00000\tval-auc:0.73663\n",
            "[765]\ttrain-auc:1.00000\tval-auc:0.73658\n",
            "[766]\ttrain-auc:1.00000\tval-auc:0.73662\n",
            "[767]\ttrain-auc:1.00000\tval-auc:0.73665\n",
            "[768]\ttrain-auc:1.00000\tval-auc:0.73660\n",
            "[769]\ttrain-auc:1.00000\tval-auc:0.73663\n",
            "[770]\ttrain-auc:1.00000\tval-auc:0.73676\n",
            "[771]\ttrain-auc:1.00000\tval-auc:0.73660\n",
            "[772]\ttrain-auc:1.00000\tval-auc:0.73682\n",
            "[773]\ttrain-auc:1.00000\tval-auc:0.73677\n",
            "[774]\ttrain-auc:1.00000\tval-auc:0.73670\n",
            "[775]\ttrain-auc:1.00000\tval-auc:0.73661\n",
            "[776]\ttrain-auc:1.00000\tval-auc:0.73660\n",
            "[777]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[778]\ttrain-auc:1.00000\tval-auc:0.73691\n",
            "[779]\ttrain-auc:1.00000\tval-auc:0.73678\n",
            "[780]\ttrain-auc:1.00000\tval-auc:0.73694\n",
            "[781]\ttrain-auc:1.00000\tval-auc:0.73679\n",
            "[782]\ttrain-auc:1.00000\tval-auc:0.73673\n",
            "[783]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[784]\ttrain-auc:1.00000\tval-auc:0.73670\n",
            "[785]\ttrain-auc:1.00000\tval-auc:0.73654\n",
            "[786]\ttrain-auc:1.00000\tval-auc:0.73675\n",
            "[787]\ttrain-auc:1.00000\tval-auc:0.73694\n",
            "[788]\ttrain-auc:1.00000\tval-auc:0.73682\n",
            "[789]\ttrain-auc:1.00000\tval-auc:0.73678\n",
            "[790]\ttrain-auc:1.00000\tval-auc:0.73665\n",
            "[791]\ttrain-auc:1.00000\tval-auc:0.73693\n",
            "[792]\ttrain-auc:1.00000\tval-auc:0.73687\n",
            "[793]\ttrain-auc:1.00000\tval-auc:0.73686\n",
            "[794]\ttrain-auc:1.00000\tval-auc:0.73674\n",
            "[795]\ttrain-auc:1.00000\tval-auc:0.73674\n",
            "[796]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[797]\ttrain-auc:1.00000\tval-auc:0.73667\n",
            "[798]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[799]\ttrain-auc:1.00000\tval-auc:0.73688\n",
            "[800]\ttrain-auc:1.00000\tval-auc:0.73680\n",
            "[801]\ttrain-auc:1.00000\tval-auc:0.73677\n",
            "[802]\ttrain-auc:1.00000\tval-auc:0.73662\n",
            "[803]\ttrain-auc:1.00000\tval-auc:0.73650\n",
            "[804]\ttrain-auc:1.00000\tval-auc:0.73653\n",
            "[805]\ttrain-auc:1.00000\tval-auc:0.73653\n",
            "[806]\ttrain-auc:1.00000\tval-auc:0.73653\n",
            "[807]\ttrain-auc:1.00000\tval-auc:0.73644\n",
            "[808]\ttrain-auc:1.00000\tval-auc:0.73647\n",
            "[809]\ttrain-auc:1.00000\tval-auc:0.73639\n",
            "[810]\ttrain-auc:1.00000\tval-auc:0.73659\n",
            "[811]\ttrain-auc:1.00000\tval-auc:0.73665\n",
            "[812]\ttrain-auc:1.00000\tval-auc:0.73661\n",
            "[813]\ttrain-auc:1.00000\tval-auc:0.73636\n",
            "[814]\ttrain-auc:1.00000\tval-auc:0.73629\n",
            "[815]\ttrain-auc:1.00000\tval-auc:0.73621\n",
            "[816]\ttrain-auc:1.00000\tval-auc:0.73613\n",
            "[817]\ttrain-auc:1.00000\tval-auc:0.73617\n",
            "[818]\ttrain-auc:1.00000\tval-auc:0.73620\n",
            "[819]\ttrain-auc:1.00000\tval-auc:0.73641\n",
            "[820]\ttrain-auc:1.00000\tval-auc:0.73638\n",
            "[821]\ttrain-auc:1.00000\tval-auc:0.73631\n",
            "[822]\ttrain-auc:1.00000\tval-auc:0.73644\n",
            "[823]\ttrain-auc:1.00000\tval-auc:0.73618\n",
            "[824]\ttrain-auc:1.00000\tval-auc:0.73608\n",
            "[825]\ttrain-auc:1.00000\tval-auc:0.73618\n",
            "[826]\ttrain-auc:1.00000\tval-auc:0.73613\n",
            "[827]\ttrain-auc:1.00000\tval-auc:0.73635\n",
            "[828]\ttrain-auc:1.00000\tval-auc:0.73647\n",
            "[829]\ttrain-auc:1.00000\tval-auc:0.73648\n",
            "[830]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[831]\ttrain-auc:1.00000\tval-auc:0.73669\n",
            "[832]\ttrain-auc:1.00000\tval-auc:0.73675\n",
            "[833]\ttrain-auc:1.00000\tval-auc:0.73663\n",
            "[834]\ttrain-auc:1.00000\tval-auc:0.73666\n",
            "[835]\ttrain-auc:1.00000\tval-auc:0.73654\n",
            "[836]\ttrain-auc:1.00000\tval-auc:0.73656\n",
            "[837]\ttrain-auc:1.00000\tval-auc:0.73648\n",
            "[838]\ttrain-auc:1.00000\tval-auc:0.73633\n",
            "[839]\ttrain-auc:1.00000\tval-auc:0.73640\n",
            "[840]\ttrain-auc:1.00000\tval-auc:0.73632\n",
            "[841]\ttrain-auc:1.00000\tval-auc:0.73610\n",
            "[842]\ttrain-auc:1.00000\tval-auc:0.73620\n",
            "[843]\ttrain-auc:1.00000\tval-auc:0.73622\n",
            "[844]\ttrain-auc:1.00000\tval-auc:0.73616\n",
            "[845]\ttrain-auc:1.00000\tval-auc:0.73626\n",
            "[846]\ttrain-auc:1.00000\tval-auc:0.73619\n",
            "[847]\ttrain-auc:1.00000\tval-auc:0.73629\n",
            "[848]\ttrain-auc:1.00000\tval-auc:0.73612\n",
            "[849]\ttrain-auc:1.00000\tval-auc:0.73635\n",
            "[850]\ttrain-auc:1.00000\tval-auc:0.73632\n",
            "[851]\ttrain-auc:1.00000\tval-auc:0.73642\n",
            "[852]\ttrain-auc:1.00000\tval-auc:0.73622\n",
            "[853]\ttrain-auc:1.00000\tval-auc:0.73617\n",
            "[854]\ttrain-auc:1.00000\tval-auc:0.73615\n",
            "[855]\ttrain-auc:1.00000\tval-auc:0.73598\n",
            "[856]\ttrain-auc:1.00000\tval-auc:0.73595\n",
            "[857]\ttrain-auc:1.00000\tval-auc:0.73602\n",
            "[858]\ttrain-auc:1.00000\tval-auc:0.73595\n",
            "[859]\ttrain-auc:1.00000\tval-auc:0.73604\n",
            "[860]\ttrain-auc:1.00000\tval-auc:0.73606\n",
            "[861]\ttrain-auc:1.00000\tval-auc:0.73591\n",
            "[862]\ttrain-auc:1.00000\tval-auc:0.73592\n",
            "[863]\ttrain-auc:1.00000\tval-auc:0.73583\n",
            "[864]\ttrain-auc:1.00000\tval-auc:0.73576\n",
            "[865]\ttrain-auc:1.00000\tval-auc:0.73570\n",
            "[866]\ttrain-auc:1.00000\tval-auc:0.73572\n",
            "[867]\ttrain-auc:1.00000\tval-auc:0.73567\n",
            "[868]\ttrain-auc:1.00000\tval-auc:0.73570\n",
            "[869]\ttrain-auc:1.00000\tval-auc:0.73559\n",
            "[870]\ttrain-auc:1.00000\tval-auc:0.73556\n",
            "[871]\ttrain-auc:1.00000\tval-auc:0.73560\n",
            "[872]\ttrain-auc:1.00000\tval-auc:0.73570\n",
            "[873]\ttrain-auc:1.00000\tval-auc:0.73567\n",
            "[874]\ttrain-auc:1.00000\tval-auc:0.73566\n",
            "[875]\ttrain-auc:1.00000\tval-auc:0.73557\n",
            "[876]\ttrain-auc:1.00000\tval-auc:0.73579\n",
            "[877]\ttrain-auc:1.00000\tval-auc:0.73572\n",
            "[878]\ttrain-auc:1.00000\tval-auc:0.73569\n",
            "[879]\ttrain-auc:1.00000\tval-auc:0.73583\n",
            "[880]\ttrain-auc:1.00000\tval-auc:0.73593\n",
            "[881]\ttrain-auc:1.00000\tval-auc:0.73588\n",
            "[882]\ttrain-auc:1.00000\tval-auc:0.73603\n",
            "[883]\ttrain-auc:1.00000\tval-auc:0.73601\n",
            "[884]\ttrain-auc:1.00000\tval-auc:0.73607\n",
            "[885]\ttrain-auc:1.00000\tval-auc:0.73588\n",
            "[886]\ttrain-auc:1.00000\tval-auc:0.73582\n",
            "[887]\ttrain-auc:1.00000\tval-auc:0.73578\n",
            "[888]\ttrain-auc:1.00000\tval-auc:0.73598\n",
            "[889]\ttrain-auc:1.00000\tval-auc:0.73584\n",
            "[890]\ttrain-auc:1.00000\tval-auc:0.73583\n",
            "[891]\ttrain-auc:1.00000\tval-auc:0.73581\n",
            "[892]\ttrain-auc:1.00000\tval-auc:0.73581\n",
            "[893]\ttrain-auc:1.00000\tval-auc:0.73596\n",
            "[894]\ttrain-auc:1.00000\tval-auc:0.73586\n",
            "[895]\ttrain-auc:1.00000\tval-auc:0.73597\n",
            "[896]\ttrain-auc:1.00000\tval-auc:0.73598\n",
            "[897]\ttrain-auc:1.00000\tval-auc:0.73606\n",
            "[898]\ttrain-auc:1.00000\tval-auc:0.73620\n",
            "[899]\ttrain-auc:1.00000\tval-auc:0.73621\n",
            "[900]\ttrain-auc:1.00000\tval-auc:0.73604\n",
            "[901]\ttrain-auc:1.00000\tval-auc:0.73596\n",
            "[902]\ttrain-auc:1.00000\tval-auc:0.73602\n",
            "[903]\ttrain-auc:1.00000\tval-auc:0.73583\n",
            "[904]\ttrain-auc:1.00000\tval-auc:0.73581\n",
            "[905]\ttrain-auc:1.00000\tval-auc:0.73570\n",
            "[906]\ttrain-auc:1.00000\tval-auc:0.73559\n",
            "[907]\ttrain-auc:1.00000\tval-auc:0.73553\n",
            "[908]\ttrain-auc:1.00000\tval-auc:0.73565\n",
            "[909]\ttrain-auc:1.00000\tval-auc:0.73563\n",
            "[910]\ttrain-auc:1.00000\tval-auc:0.73579\n",
            "[911]\ttrain-auc:1.00000\tval-auc:0.73581\n",
            "[912]\ttrain-auc:1.00000\tval-auc:0.73565\n",
            "[913]\ttrain-auc:1.00000\tval-auc:0.73561\n",
            "[914]\ttrain-auc:1.00000\tval-auc:0.73555\n",
            "[915]\ttrain-auc:1.00000\tval-auc:0.73549\n",
            "[916]\ttrain-auc:1.00000\tval-auc:0.73578\n",
            "[917]\ttrain-auc:1.00000\tval-auc:0.73579\n",
            "[918]\ttrain-auc:1.00000\tval-auc:0.73586\n",
            "[919]\ttrain-auc:1.00000\tval-auc:0.73580\n",
            "[920]\ttrain-auc:1.00000\tval-auc:0.73570\n",
            "[921]\ttrain-auc:1.00000\tval-auc:0.73569\n",
            "[922]\ttrain-auc:1.00000\tval-auc:0.73550\n",
            "[923]\ttrain-auc:1.00000\tval-auc:0.73582\n",
            "[924]\ttrain-auc:1.00000\tval-auc:0.73582\n",
            "[925]\ttrain-auc:1.00000\tval-auc:0.73586\n",
            "[926]\ttrain-auc:1.00000\tval-auc:0.73572\n",
            "[927]\ttrain-auc:1.00000\tval-auc:0.73582\n",
            "[928]\ttrain-auc:1.00000\tval-auc:0.73582\n",
            "[929]\ttrain-auc:1.00000\tval-auc:0.73584\n",
            "[930]\ttrain-auc:1.00000\tval-auc:0.73594\n",
            "[931]\ttrain-auc:1.00000\tval-auc:0.73591\n",
            "[932]\ttrain-auc:1.00000\tval-auc:0.73569\n",
            "[933]\ttrain-auc:1.00000\tval-auc:0.73585\n",
            "[934]\ttrain-auc:1.00000\tval-auc:0.73576\n",
            "[935]\ttrain-auc:1.00000\tval-auc:0.73573\n",
            "[936]\ttrain-auc:1.00000\tval-auc:0.73578\n",
            "[937]\ttrain-auc:1.00000\tval-auc:0.73577\n",
            "[938]\ttrain-auc:1.00000\tval-auc:0.73579\n",
            "[939]\ttrain-auc:1.00000\tval-auc:0.73564\n",
            "[940]\ttrain-auc:1.00000\tval-auc:0.73559\n",
            "[941]\ttrain-auc:1.00000\tval-auc:0.73569\n",
            "[942]\ttrain-auc:1.00000\tval-auc:0.73586\n",
            "[943]\ttrain-auc:1.00000\tval-auc:0.73583\n",
            "[944]\ttrain-auc:1.00000\tval-auc:0.73571\n",
            "[945]\ttrain-auc:1.00000\tval-auc:0.73568\n",
            "[946]\ttrain-auc:1.00000\tval-auc:0.73553\n",
            "[947]\ttrain-auc:1.00000\tval-auc:0.73561\n",
            "[948]\ttrain-auc:1.00000\tval-auc:0.73551\n",
            "[949]\ttrain-auc:1.00000\tval-auc:0.73554\n",
            "[950]\ttrain-auc:1.00000\tval-auc:0.73574\n",
            "[951]\ttrain-auc:1.00000\tval-auc:0.73563\n",
            "[952]\ttrain-auc:1.00000\tval-auc:0.73568\n",
            "[953]\ttrain-auc:1.00000\tval-auc:0.73576\n",
            "[954]\ttrain-auc:1.00000\tval-auc:0.73586\n",
            "[955]\ttrain-auc:1.00000\tval-auc:0.73580\n",
            "[956]\ttrain-auc:1.00000\tval-auc:0.73583\n",
            "[957]\ttrain-auc:1.00000\tval-auc:0.73574\n",
            "[958]\ttrain-auc:1.00000\tval-auc:0.73580\n",
            "[959]\ttrain-auc:1.00000\tval-auc:0.73581\n",
            "[960]\ttrain-auc:1.00000\tval-auc:0.73590\n",
            "[961]\ttrain-auc:1.00000\tval-auc:0.73586\n",
            "[962]\ttrain-auc:1.00000\tval-auc:0.73587\n",
            "[963]\ttrain-auc:1.00000\tval-auc:0.73566\n",
            "[964]\ttrain-auc:1.00000\tval-auc:0.73567\n",
            "[965]\ttrain-auc:1.00000\tval-auc:0.73557\n",
            "[966]\ttrain-auc:1.00000\tval-auc:0.73555\n",
            "[967]\ttrain-auc:1.00000\tval-auc:0.73555\n",
            "[968]\ttrain-auc:1.00000\tval-auc:0.73553\n",
            "[969]\ttrain-auc:1.00000\tval-auc:0.73547\n",
            "[970]\ttrain-auc:1.00000\tval-auc:0.73545\n",
            "[971]\ttrain-auc:1.00000\tval-auc:0.73551\n",
            "[972]\ttrain-auc:1.00000\tval-auc:0.73539\n",
            "[973]\ttrain-auc:1.00000\tval-auc:0.73539\n",
            "[974]\ttrain-auc:1.00000\tval-auc:0.73547\n",
            "[975]\ttrain-auc:1.00000\tval-auc:0.73535\n",
            "[976]\ttrain-auc:1.00000\tval-auc:0.73529\n",
            "[977]\ttrain-auc:1.00000\tval-auc:0.73525\n",
            "[978]\ttrain-auc:1.00000\tval-auc:0.73523\n",
            "[979]\ttrain-auc:1.00000\tval-auc:0.73509\n",
            "[980]\ttrain-auc:1.00000\tval-auc:0.73509\n",
            "[981]\ttrain-auc:1.00000\tval-auc:0.73507\n",
            "[982]\ttrain-auc:1.00000\tval-auc:0.73497\n",
            "[983]\ttrain-auc:1.00000\tval-auc:0.73497\n",
            "[984]\ttrain-auc:1.00000\tval-auc:0.73508\n",
            "[985]\ttrain-auc:1.00000\tval-auc:0.73499\n",
            "[986]\ttrain-auc:1.00000\tval-auc:0.73500\n",
            "[987]\ttrain-auc:1.00000\tval-auc:0.73499\n",
            "[988]\ttrain-auc:1.00000\tval-auc:0.73504\n",
            "[989]\ttrain-auc:1.00000\tval-auc:0.73503\n",
            "[990]\ttrain-auc:1.00000\tval-auc:0.73507\n",
            "[991]\ttrain-auc:1.00000\tval-auc:0.73500\n",
            "[992]\ttrain-auc:1.00000\tval-auc:0.73517\n",
            "[993]\ttrain-auc:1.00000\tval-auc:0.73522\n",
            "[994]\ttrain-auc:1.00000\tval-auc:0.73525\n",
            "[995]\ttrain-auc:1.00000\tval-auc:0.73519\n",
            "[996]\ttrain-auc:1.00000\tval-auc:0.73521\n",
            "[997]\ttrain-auc:1.00000\tval-auc:0.73519\n",
            "[998]\ttrain-auc:1.00000\tval-auc:0.73540\n",
            "[999]\ttrain-auc:1.00000\tval-auc:0.73540\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Whoa! So if we create a model with 1,000 trees, it looks like it just keeps getting better and better on the training set (not surprising), but certainly hits a point of diminishing (or negative) returns on the validation set. The reason for this is the model is overfitting."
      ],
      "metadata": {
        "id": "xTB3MD2Ht9u-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Our final validation ROC AUC score is $.7354$. I bet we can do better."
      ],
      "metadata": {
        "id": "1Ln4PNxxuSqk"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Early Stopping"
      ],
      "metadata": {
        "id": "OSpI51dKuwLc"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "One way we might be able to improve the performance is by stopping training much earlier to prevent overfitting.\n",
        "\n",
        "To do this, we use the validation set, which means the choice of the model depends not just on the performance on the training data, but also on the validation data. This is fine - in fact this is what the validation data is for. But, please note there's a *big* difference between the validation dataset and the test dataset, and you would never want to do something like this with the test dataset."
      ],
      "metadata": {
        "id": "In8uDH-WvGAb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We note that in our model specification above we included both the training data and the valuation data in the *evals* parameter. This is because we're curious about the model performance on both. However, only the last tuple of data will be used when determining early stopping."
      ],
      "metadata": {
        "id": "yrmBP1hXvrYT"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Early stopping works by stopping if, after a set number of rounds, the model performance on the validation data has not improved beyond a certain amount (the tolerance). In our case, we'll stop the model fit process if it goes 30 rounds (adds 30 trees) without any notable improvement. We do this with the *early_stopping_rounds* parameter in the *train* method."
      ],
      "metadata": {
        "id": "bjLteV5vwh4i"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "xgb_model_1 = xgb.train(params, dtrain, num_boost_round = 1000, early_stopping_rounds = 30, evals = [(dtrain, 'train'), (dval, 'val')])"
      ],
      "metadata": {
        "id": "zCBFLmQEw74o",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1f484fd3-0ad4-4803-8d63-86587cbca668"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[0]\ttrain-auc:0.79531\tval-auc:0.73119\n",
            "[1]\ttrain-auc:0.80636\tval-auc:0.73976\n",
            "[2]\ttrain-auc:0.82250\tval-auc:0.73920\n",
            "[3]\ttrain-auc:0.83829\tval-auc:0.73583\n",
            "[4]\ttrain-auc:0.85144\tval-auc:0.74157\n",
            "[5]\ttrain-auc:0.87664\tval-auc:0.73214\n",
            "[6]\ttrain-auc:0.88978\tval-auc:0.73201\n",
            "[7]\ttrain-auc:0.89280\tval-auc:0.73452\n",
            "[8]\ttrain-auc:0.90171\tval-auc:0.72859\n",
            "[9]\ttrain-auc:0.90602\tval-auc:0.73272\n",
            "[10]\ttrain-auc:0.92157\tval-auc:0.73137\n",
            "[11]\ttrain-auc:0.93475\tval-auc:0.72668\n",
            "[12]\ttrain-auc:0.94233\tval-auc:0.72230\n",
            "[13]\ttrain-auc:0.95055\tval-auc:0.72127\n",
            "[14]\ttrain-auc:0.95533\tval-auc:0.71564\n",
            "[15]\ttrain-auc:0.95822\tval-auc:0.71433\n",
            "[16]\ttrain-auc:0.96280\tval-auc:0.71521\n",
            "[17]\ttrain-auc:0.96772\tval-auc:0.71352\n",
            "[18]\ttrain-auc:0.97143\tval-auc:0.72707\n",
            "[19]\ttrain-auc:0.97464\tval-auc:0.72644\n",
            "[20]\ttrain-auc:0.97718\tval-auc:0.72580\n",
            "[21]\ttrain-auc:0.97893\tval-auc:0.72509\n",
            "[22]\ttrain-auc:0.98167\tval-auc:0.72142\n",
            "[23]\ttrain-auc:0.98256\tval-auc:0.72197\n",
            "[24]\ttrain-auc:0.98553\tval-auc:0.72124\n",
            "[25]\ttrain-auc:0.98676\tval-auc:0.72233\n",
            "[26]\ttrain-auc:0.98976\tval-auc:0.71914\n",
            "[27]\ttrain-auc:0.99136\tval-auc:0.71955\n",
            "[28]\ttrain-auc:0.99207\tval-auc:0.71662\n",
            "[29]\ttrain-auc:0.99281\tval-auc:0.71585\n",
            "[30]\ttrain-auc:0.99367\tval-auc:0.71515\n",
            "[31]\ttrain-auc:0.99423\tval-auc:0.71510\n",
            "[32]\ttrain-auc:0.99499\tval-auc:0.71714\n",
            "[33]\ttrain-auc:0.99579\tval-auc:0.71353\n",
            "[34]\ttrain-auc:0.99647\tval-auc:0.71266\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "So, it looks like instead of using 1,000 trees in the model, we can do better with... 5. We can check out the best iteration and its corresponding score on the valuation data with the *attributes* method."
      ],
      "metadata": {
        "id": "v0_rCEb_XvZl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "xgb_model_1.attributes()"
      ],
      "metadata": {
        "id": "DTdqdGWdz2aE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "02efb8d9-5b48-425d-ba69-72e5b6b7983c"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'best_iteration': '4', 'best_score': '0.7415745878504073'}"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "And, indeed, we can calculate this ROC AUC score directly with the code below. Note that for the prediction if we only want it to use up until the best iteration (the iteration that does the best on the valuation data), you need to specify this with the *iteration_range* parameter."
      ],
      "metadata": {
        "id": "Ye5f5Mh8YAF0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "val_set_pred_prob = xgb_model_1.predict(dval, iteration_range=(0, xgb_model_1.best_iteration + 1))\n",
        "roc_auc_score(y_val, val_set_pred_prob)"
      ],
      "metadata": {
        "id": "NcobaPWqwJxD",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c8bf1ee3-ccf5-4a4a-92e3-a440a482c1f8"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.7415745878504072"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Learning Rate Optimization"
      ],
      "metadata": {
        "id": "hEo0OFmVYYe1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We've set a default learning rate of $0.3$. Could we do better?\n",
        "\n",
        "Let's investigate. First, let's create an array of 25 learning rates, starting at .01, going up to 1, equally spaced."
      ],
      "metadata": {
        "id": "qBSk6jFBYiPs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "learning_rates = np.linspace(start=0.01, stop=1, num=25)"
      ],
      "metadata": {
        "id": "FYYd3Lzn4rBf"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, let's write some code that trains a XGBoost model for each of these learning rates. It should use the same parameters (except the learning rate), training data, number of rounds, and early stopping rounds we've used before. Only the learning rate will be different.\n",
        "\n",
        "For each learning rate, calculate the ROC AUC score of the optimal stopping model on the validation data, and record the number of trees required to get to that optimal stopping model.\n",
        "\n",
        "*NOTE* - Within the training method, you'll want to set *verbose_eval* = **False** or else the for loop will produce a *lot* of text."
      ],
      "metadata": {
        "id": "rZnvmPIoY7eD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "XQ61MJiQY_1m"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "val_aucs = [] #The ROC AUC score on the validation data for the optimal number of iterations (trees)\n",
        "best_iters = [] #The optimal number of iterations (trees) for the given learning rate\n",
        "for learning_rate in learning_rates: #Fit the model to the training data for the different learning rates, and record the optimal ROC AUC and iteration for each learning rate.\n",
        "### YOUR CODE HERE ###\n",
        "    params = {'objective': 'binary:logistic', 'eval_metric': 'auc'}\n",
        "\n",
        "    params['learning_rate'] = learning_rate\n",
        "\n",
        "    xgb_model = xgb.train(params, dtrain, num_boost_round = 1000, early_stopping_rounds = 30, evals = [(dtrain, 'train'), (dval, 'val')], verbose_eval=False)\n",
        "\n",
        "    val_set_pred_prob = xgb_model.predict(dval, iteration_range=(0, xgb_model.best_iteration + 1))\n",
        "\n",
        "    val_auc = roc_auc_score(y_val, val_set_pred_prob)\n",
        "\n",
        "    val_aucs.append(val_auc)\n",
        "    best_iters.append(xgb_model.best_iteration)"
      ],
      "metadata": {
        "id": "W1Nd7e2rZYas"
      },
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's create a DataFrame will our learning rates, best ROC AUC scores, and ideal number of iterations (trees)."
      ],
      "metadata": {
        "id": "Qp9WL3GjcA0B"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "learning_rate_df = pd.DataFrame({'learning_rate': learning_rates, 'val_auc': val_aucs, 'best_iter': best_iters})\n",
        "learning_rate_df.head()"
      ],
      "metadata": {
        "id": "zRkNXFjHC3Jy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "outputId": "db27d39f-1147-4122-af28-8c46f9deaa87"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   learning_rate   val_auc  best_iter\n",
              "0        0.01000  0.743611          4\n",
              "1        0.05125  0.760701         41\n",
              "2        0.09250  0.771297         15\n",
              "3        0.13375  0.755579         10\n",
              "4        0.17500  0.766310         10"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-92116758-0418-4561-ad04-a51d7d2c96ec\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>learning_rate</th>\n",
              "      <th>val_auc</th>\n",
              "      <th>best_iter</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.01000</td>\n",
              "      <td>0.743611</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.05125</td>\n",
              "      <td>0.760701</td>\n",
              "      <td>41</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.09250</td>\n",
              "      <td>0.771297</td>\n",
              "      <td>15</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.13375</td>\n",
              "      <td>0.755579</td>\n",
              "      <td>10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.17500</td>\n",
              "      <td>0.766310</td>\n",
              "      <td>10</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-92116758-0418-4561-ad04-a51d7d2c96ec')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-92116758-0418-4561-ad04-a51d7d2c96ec button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-92116758-0418-4561-ad04-a51d7d2c96ec');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-b3c2370d-7f92-4ea7-bd94-03d728210505\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b3c2370d-7f92-4ea7-bd94-03d728210505')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-b3c2370d-7f92-4ea7-bd94-03d728210505 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "learning_rate_df",
              "summary": "{\n  \"name\": \"learning_rate_df\",\n  \"rows\": 25,\n  \"fields\": [\n    {\n      \"column\": \"learning_rate\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.3035917797800198,\n        \"min\": 0.01,\n        \"max\": 1.0,\n        \"num_unique_values\": 25,\n        \"samples\": [\n          0.34,\n          0.67,\n          0.01\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"val_auc\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.01127616386523276,\n        \"min\": 0.7311886586695747,\n        \"max\": 0.7712970684456989,\n        \"num_unique_values\": 22,\n        \"samples\": [\n          0.7436112803900187,\n          0.7505051638976201,\n          0.7469028641991147\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"best_iter\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 8,\n        \"min\": 0,\n        \"max\": 41,\n        \"num_unique_values\": 9,\n        \"samples\": [\n          6,\n          41,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's plot both the best ROC AUC score and the requires number of trees to obtain it for the various learning rates."
      ],
      "metadata": {
        "id": "vjDooJe4cLYw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "learning_rate_df.set_index('learning_rate').plot(secondary_y='best_iter', style=['-o', '--o'], xlabel='Learning Rate');"
      ],
      "metadata": {
        "id": "vBnf1SOfC6xD",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 449
        },
        "outputId": "63c37b1b-182a-4f4f-a131-cfc490bd5915"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAGwCAYAAABmTltaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACnaElEQVR4nOzdeVzUdf7A8dfMcN8CcogoHnngfUGWlamlHZblVm6aZWblZofudri//WW2h7aHa4flr5bMNku3zbbcijTL0jwoDRUxMkVR5BBQbgaYmd8f35mBAQZmYC7g/Xw85gHzne98vx+I4O3n8/683yqDwWBACCGEEELYTO3uAQghhBBCdDYSQAkhhBBC2EkCKCGEEEIIO0kAJYQQQghhJwmghBBCCCHsJAGUEEIIIYSdJIASQgghhLCTl7sH4Cr19fX88MMPREdHo1ZL3CiEEEJ0Bnq9noKCAsaMGYOXl+eELZ4zEif74YcfSEpKcvcwhBBCCNEOaWlpTJgwwd3DMOs2AVR0dDSg/AeIjY1182iEEEIIYYu8vDySkpLMf8c9RbcJoEzLdrGxsfTu3dvNoxFCCCGEPTwt/cazRiOEEEII0QlIACWEEEIIYScJoIQQQggh7NRtcqCEEELYTq/XU1tb6+5hiG7A29sbjUbj7mHYTQIoIYQQFmpra8nOzkav17t7KKKbCAsLIyYmBpVK5e6h2EwCKCGEEGYGg4G8vDw0Gg3x8fEet/NJdC0Gg4GqqioKCwsBOlWZIQmghBBCmNXX11NVVUWvXr0ICAhw93BEN+Dv7w9AYWEhUVFRnWY5T/5pIYQQwkyn0wHg4+Pj5pGI7sQUrNfV1bl5JLaTAEoIIUQznSkXRXR+nfHnTZbwPIleB2f2QkUBBEVD3ytAbf9Upk5vIC27hMLyGqKC/UjqF45G3fl+OIUQQghPJQGUp8j8GFKfhrLzDcdCesGMFyDxFpsvk5qRx8ptmeSV1piPxYb6sWJmIjOGd57kPCGEEMKTyRKeJ8j8GP413zJ4AijLU45nfmzTZVIz8lj8ziGL4Akgv7SGxe8cIjUjz1EjFkKIVun0BvadLOaj9Fz2nSxGpze4e0htSkhIYO3ate4ehugkZAbK3fQ6ZeaJln65GAAVpD4DQ25qdTlPpzewcltma1dh5bZMrkuMkeU8IYRTyUy46A5kBsrdzuxtPvNkwQBlucp5rUjLLmk289TkKuSV1pCWXdK+cQohhA1kJlx0FxJAuVtFgUPOKyy3Hjy15zwhhABjocPaepse5TV1rPj4mNWZcIDnPs6kvKbOpusZDLYv+73++uv06tWrWfX0W2+9lfvvv5+TJ09y6623Eh0dTVBQEBMmTOCLL75o9/dlzZo1jBgxgsDAQOLj4/nVr35FRUWF+fXnnnuO0aNHW7xn7dq1JCQkWBx78803GTZsGL6+vsTGxrJkyZJ2j0m4lizhuVtQtEPOiwr2s+kytp4nhBAA1XU6Ep/93CHXMgD5ZTWMeG67TednPj+dAB/b/kzdcccdPProo3z11VdMnToVgJKSElJTU/n000+pqKjgxhtv5I9//CO+vr68/fbbzJw5k6ysLPr06WP316JWq3nppZfo168fp06d4le/+hVPPfUUr776qs3XeO2111i2bBmrV6/mhhtuoLS0lG+//dbusQj3kADK3fpeoey2K8uj5TwolfJ63ytavUxSv3BiQ/3IL62xdhViQpWSBkII0dX06NGDG264gXfffdccQP373/8mMjKSa6+9FrVazahRo8zn//73v+fDDz/k448/bteszxNPPGH+PCEhgT/84Q88/PDDdgVQf/jDH/j1r3/N448/bj42YcIEu8ci3EMCKHdTa5RSBf+a38KLxmTvGavbrAelUatYMTORxe8csnYVVsxMlARyIYRd/L01ZD4/3aZz07JLuG/Dd22e99aCCTb9Y87f2746eHPnzmXRokW8+uqr+Pr6smnTJubMmYNaraaiooLnnnuOTz75hLy8POrr66muriYnJ8eue5h88cUXrFq1ih9//JGysjLq6+upqamhqqrKphY4hYWFnD9/3hzsic5HcqA8QeItcOfboGnSOiGkl3LcxjpQM4bH8tq8sfh6Wf5njQ7x47V5Y2X3ixDCbiqVigAfL5seV13Wk9hQP6z9M02Fshvvqst62nQ9e6tTz5w5E4PBwCeffMLZs2fZvXs3c+fOBeA3v/kNH374IX/605/YvXs36enpjBgxgtraWru/J6dPn+bmm29m5MiRfPDBBxw8eJB169YBmK+nVqub5XA1blNi6v8mnG/16tWoVCqLWcOamhoeeeQRIiIiCAoKYvbs2RQU2JiTbCQBlKdIvAWu/R/lc40v3PkOPHHUriKaANOHxRDkazmx+H/3jJPgSQjhdKaZcKBZEOWKmXA/Pz9uv/12Nm3axHvvvcfgwYMZO3YsAN9++y333Xcft912GyNGjCAmJobTp0+36z4HDx5Er9fzt7/9jcsvv5xBgwZx/rzlbuqePXuSn59vEUSlp6ebPw8ODiYhIYGdO3e2awzCNt999x3/93//x8iRIy2OL126lG3btvH+++/z9ddfc/78eW6//Xa7ri0BlCeZ9AQERIJOC+EJ7Wrjcr60huLKWjRqFYmxIQCcu1jt2HEKIYQVppnwmFDLDSsxoa6ZCZ87dy6ffPIJb775pnn2CeCyyy5j69atpKenc/jwYe6+++5mO/ZsNXDgQOrq6nj55Zc5deoU//znP1m/fr3FOZMnT+bChQv8+c9/5uTJk6xbt47PPvvM4pznnnuOv/3tb7z00kucOHGCQ4cO8fLLL7drTKK5iooK5s6dyxtvvEGPHj3Mx0tLS0lJSWHNmjVMmTKFcePGsWHDBvbu3cv+/fttvn67Aqh169aRkJCAn58fycnJpKWlWT138uTJqFSqZo+bbrrJfE5Lr6tUKv7yl7+YzykpKWHu3LmEhIQQFhbGwoULLbaMdhlBUcrHygvtevvhs5cAGBITzJDYYABOF1c6YmRCCGGTGcNj2fP0FN5bdDkvzhnNe4suZ8/TU1wyEz5lyhTCw8PJysri7rvvNh9fs2YNPXr04IorrmDmzJlMnz7dPDtlr1GjRrFmzRpeeOEFhg8fzqZNm1i1apXFOUOHDuXVV19l3bp1jBo1irS0NH7zm99YnHPvvfeydu1aXn31VYYNG8bNN9/MiRMn2jWm7qC8vJyysjLzQ6vVtnr+I488wk033cS0adMsjh88eJC6ujqL40OGDKFPnz7s27fP5vHYnUS+ZcsWli1bxvr160lOTmbt2rVMnz6drKwsoqKimp2/detWizXm4uJiRo0axR133GE+lpdnWVjts88+Y+HChcyePdt8bO7cueTl5bFjxw7q6upYsGABDz74IO+++669X4Jn0uuVgplevsrzio4FUKPiw4gJUf4FeLpIAighhGtp1ComDohw+X3VanWz5TRQdsp9+eWXFsceeeQRi+f2LOktXbqUpUuXWhy75557LJ4//PDDPPzwwxbHfvvb31o8f+ihh3jooYdsvm93lpiYaPF8xYoVPPfccy2eu3nzZg4dOsR33zXf1JCfn4+Pjw9hYWEWx6Ojo8nPz7d5PHYHUGvWrGHRokUsWLAAgPXr15unS5955plm54eHW+602Lx5MwEBARYBVExMjMU5H330Eddeey39+/cH4Pjx46SmpvLdd98xfvx4AF5++WVuvPFG/vrXv9KrVy97vwzPU1UMa4c3PG/nDFS6MYAaHR+Gn3EHi8xACSGE6OwyMzOJi4szP/f19W3xvLNnz/L444+zY8cO/PycV/vQriW82tpaDh48aDHtpVarmTZtms3TXikpKcyZM4fAwMAWXy8oKOCTTz5h4cKF5mP79u0jLCzMHDwBTJs2DbVazYEDB1q8jlartZjqKy8vt2l8blNTavm8stDuS+j0Bo7mKtcZHR9GQoSylTa7qKrDwxNCiO5i06ZNBAUFtfgYNmyYu4fXbQUHBxMSEmJ+WAugDh48SGFhIWPHjsXLywsvLy++/vprXnrpJby8vIiOjqa2tpZLly5ZvK+goKDZhE5r7JqBKioqQqfTER1tWRU7OjqaH3/8sc33p6WlkZGRQUpKitVzNm7cSHBwsEU2fH5+frPlQS8vL8LDw61Ot61atYqVK1e2OSaP0SyAKrL7EicKy6mq1RHoo2FAzyAqtPUAFFVoqdDWN9udJ4QQorlbbrmF5OTkFl/z9vZ28WiEvaZOncrRo0ctji1YsIAhQ4bw9NNPEx8fj7e3Nzt37jSnCmVlZZGTk8PEiRNtvo9L/6KmpKQwYsQIkpKSrJ5j2jnR0Wm35cuXs2zZMvPz3NzcZuunHkXbKIDy8gM7ekCZmPKfRvQORaNWEervTXigDyWVtZwuqmR4XKiDBiuEEF1XcHAwwcHB7h6GaKfg4GCGDx9ucSwwMJCIiAjz8YULF7Js2TLCw8MJCQnh0UcfZeLEiVx++eU238euACoyMhKNRtOs2JQt016VlZVs3ryZ559/3uo5u3fvJisriy1btlgcj4mJobDQckmrvr6ekpISq/f19fW1mN4rKytrdXxuZ5qBik+G+z8HOwvIAaSfVa4xKj7MfCwhIoCSylrOFFdJACWEEEIAf//731Gr1cyePRutVsv06dPtasMDduZA+fj4MG7cOIvCX3q9np07d7Y57fX++++j1WqZN2+e1XNSUlIYN26cRb8igIkTJ3Lp0iUOHjxoPvbll1+i1+utTrN2OqYAyj+8XcETNMxAjbEIoJRcM0kkF0II0V3t2rWLtWvXmp/7+fmxbt06SkpKqKysZOvWrXblP0E76kAtW7aMN954g40bN3L8+HEWL15MZWWleVfe/PnzWb58ebP3paSkMGvWLCIiWt7WWlZWxvvvv88DDzzQ7LWhQ4cyY8YMFi1aRFpaGt9++y1Llixhzpw5XWMHHjQEUH7tmyWqrtWRVaAkylvMQEUaAygpZSCEEEI4jN05UHfddRcXLlzg2WefJT8/n9GjR5OammpOLM/JyUGttozLsrKy2LNnD9u3b7d63c2bN2MwGPjlL3/Z4uubNm1iyZIlTJ061Tzt9tJLL9k7fM8VlQhj50PceHjvl1BRCPdstTmgyjhfik5vICrY11z/CaCvcSeezEAJIYQQjtOuJPIlS5awZMmSFl/btWtXs2ODBw9u1lSxqQcffJAHH3zQ6uvh4eFdp2hmSy67TnkAbP8daMuUIMrGAKpxAc3GDTj7mWagiqWUgRBCCOEo0gvPEwVGKh/tKKbZuIBmY32NOVAXyrXmsgZCCOF0eh1k74aj/1Y+6nVOvd3kyZN54oknnHoPe7ljTMXFxURFRbVZVf25555j9OjRdl27vV/P+vXrmTlzpt3v83QSQHmKyiKoKVNaugQaa15V2F5M8/C5SwCM6h1mcdxUygDgjCzjCSFcIfNjpbPCxpvhg4XKx7XDleOd1K5du1CpVM2KL7Zm69at/P73vzc/T0hIsEhkdoY//vGP3HrrrSQkJLR63m9+8xuLDWGOolKp+M9//mNx7P777+fQoUPs3r3b4fdzJwmgPMXmubA6Hn7cBkE9lWM2zkAVV2g5W1INwMj45kt+porkp6UiuRDC2TI/hn/Nh7Im/ejK8pTjnTiIsld4eLhT6kk17i/bWFVVFSkpKRadPJoyGAzU19cTFBRkdVOXo/n4+HD33Xd3rbxlJIDyHFpjnSq/UAi0L4A6ck7ZwTegZyAhfs2r5EopAyFEh9VWWn/U1Sjn6HWQ+jTQUs6r8Vjq05bLedau2Q719fUsWbKE0NBQIiMj+d///V+L/FutVstvfvMb4uLiCAwMJDk52SJv98yZM8ycOZMePXoQGBjIsGHD+PTTTzl9+jTXXnstAD169EClUnHfffe1OZ7GS16TJ0/mzJkzLF26FJVKZZGrumfPHq666ir8/f2Jj4/nscceo7Ky4XuQkJDA73//e+bPn09ISIjVfOFPP/0UX19fi2KQppmzzz77jHHjxuHr68uePXuaLeHV19fz2GOPERYWRkREBE8//TT33nsvs2bNsriHXq/nqaeeIjw8nJiYGItmvqZZr9tuuw2VSmUxCzZz5kw+/vhjqqur2/y+dRYSQHmKxmUMTEt4NgZQPzRKIG+JlDIQQnTYn3pZf/zrHuWcM3ubzzxZMCivn9nbcGjtiJav2Q4bN27Ey8uLtLQ0XnzxRdasWcM//vEP8+tLlixh3759bN68mSNHjnDHHXcwY8YMTpw4AcAjjzyCVqvlm2++4ejRo7zwwgsEBQURHx/PBx98ACi7yvPy8njxxRftGtvWrVvp3bs3zz//PHl5eeTl5QFw8uRJZsyYwezZszly5Ahbtmxhz549zTZq/fWvf2XUqFH88MMP/O///m+L99i9ezfjxo1r8bVnnnmG1atXc/z4cUaOHNns9RdeeIFNmzaxYcMGvv32W8rKypotxYHyPQ4MDOTAgQP8+c9/5vnnn2fHjh0AfPfddwBs2LCBvLw883OA8ePHU19fb7V/bWckzdE8hUUAFQle/mDQ2/TWw1YSyE1MpQzOyE48IYQzVRS0fY4959kpPj6ev//976hUKgYPHszRo0f5+9//zqJFi8jJyWHDhg3k5OSY6wf+5je/ITU1lQ0bNvCnP/2JnJwcZs+ezYgRIwDo37+/+drh4eEAREVFERYWZvfYwsPD0Wg0BAcHWxRsXLVqFXPnzjXPVF122WW89NJLXHPNNbz22mvmtmZTpkzh17/+dav3OHPmjNXaiM8//zzXXXed1fe+/PLLLF++nNtuuw2AV155hU8//bTZeSNHjmTFihXmsb7yyivs3LmT6667jp49ldWTsLCwZkUpAwICCA0N5cyZM61+DZ2JBFCeQFcPtRXK535hMH4hJC2y6a0Gg8FqArmJqZRBtizhCSHa67etzCypNMrHoGjr5zTW+Lwnjlo/z06XX365xdLYxIkT+dvf/oZOp+Po0aPodDoGDRpk8R6tVmvOBXrsscdYvHgx27dvZ9q0acyePbvF2RpHOnz4MEeOHGHTpk3mYwaDAb1eT3Z2NkOHDgWUGZy2VFdXW+0j29r7S0tLKSgosOhTq9FoGDduHHq95T/km34/YmNjm7Vas8bf35+qqq7zD3kJoDyBtlGfPt8QUNu+sppTUsWlqjp8NGqGxoa0eE7jUgaV2noCfeU/uxDCTj6BbZ/T9woI6aUkjLeYB6VSXu97hX3XdYCKigo0Gg0HDx5Eo9FYvBYUFATAAw88wPTp0/nkk0/Yvn07q1at4m9/+xuPPvqoU8f10EMP8dhjjzV7rU+fPubPAwPb/j5FRkZy8eLFFl+z5f228Pa2zLNVqVTNgixrSkpKzLNUXYHkQHmCmkvKR58g0NgX3JjqPyX2CsHHq+X/nI1LGUgiuRDCadQamPGC8UnTnp7G5zNWK+c5QdP8mv3793PZZZeh0WgYM2YMOp2OwsJCBg4caPFovNwUHx/Pww8/zNatW/n1r3/NG2+8ASg7yQB0uvbXs/Lx8Wn2/rFjx5KZmdlsTAMHDjTf01ZjxowhMzPT7nGFhoYSHR1tkbOk0+k4dOiQ3dfy9vZu8Xt08uRJampqGDNmjN3X9FQSQHkCLz+ljcuIXyjP62rg3bvgjSlQ1/qOBWsFNJuSPCghhEsk3gJ3vg0hsZbHQ3opxxNvcdqtc3JyWLZsGVlZWbz33nu8/PLLPP744wAMGjSIuXPnMn/+fLZu3Up2djZpaWmsWrWKTz75BIAnnniCzz//nOzsbA4dOsRXX31lXkLr27cvKpWK//73v1y4cIGKigq7x5eQkMA333xDbm4uRUVFADz99NPs3buXJUuWkJ6ezokTJ/joo4+sdvtozfTp0zl27JjVWajWPProo6xatYqPPvqIrKwsHn/8cS5evGixJGqLhIQEdu7cSX5+vsU4du/eTf/+/RkwYIDdY/NUEkB5gpBecMvLMNO4q8PLF05+CbkH29yJ19DCpfWWL/2My3jZshNPCOFsibfAExlw739hdory8YmjTg2eQGlmX11dTVJSEo888giPP/64xZb/DRs2MH/+fH79618zePBgZs2axXfffWdeKtPpdDzyyCPmBvaDBg3i1VdfBSAuLo6VK1fyzDPPEB0d3a4A5/nnn+f06dMMGDDAvJQ1cuRIvv76a3766SeuuuoqxowZw7PPPms1Gbw1I0aMYOzYsfzrX/+y+71PP/00v/zlL5k/fz4TJ04kKCiI6dOnW82psuZvf/sbO3bsID4+3mK26b333mPRIttyezsLlaGtJnVdxLlz54iPj+fs2bP07t3b3cNp25phUHYOFn0JcS1vS63T6Rm24nNq6/V8+etr6N8zyOrlXvziBH//4ifuHN+bP/9ilLNGLYTo5GpqasjOzqZfv352//EU7vfJJ5/w5JNPkpGRgdqOfNqm9Ho9Q4cO5c4777Sopt4ex44dY8qUKfz000+Ehrb8j/3Wfu489e+3ZBN7gtpKpbCcT1BDAnlgpBJAVVifgcrKL6e2Xk+In5d5p501CZFSjVwIIbq6m266iRMnTpCbm0t8fLzN7ztz5gzbt2/nmmuuQavV8sorr5Cdnc3dd9/d4THl5eXx9ttvWw2eOisJoDzB9xtg+//AiDthtpKwSFDbxTTTGxXQbGud2lSNXEoZCCFEx+Xk5JCYmGj19czMTItddK7Unoa/arWat956i9/85jcYDAaGDx/OF198Yc4B64hp06Z1+BqeSAIoT9C4iKaJuZ2L9foabRXQbCxBShkIIYTD9OrVi/T09FZf70zi4+P59ttv3T2MTkX+inqCVgOoIqtvM89AWSmg2VhogDc9Ary5WFXH6eJKhvXqWlOpQgjhSl5eXgwcONDdwxBuJLvwPIG1AMo7wGo7l/KaOn6+oGyjHdnGDjwTU088KWUghGhLN9lfJDyErcU4PYnMQHkCcwDVqJL45b+CK6xvkz2aW4rBAHFh/kQF27ZTpl9EID/kXJJSBkIIq7y9vVGpVFy4cIGePXvaXQdICHsYDAZqa2u5cOECarXa7uKh7iQBlCdoaQaqje2nh88q77El/8nE1NLljCSSCyGs0Gg09O7dm3PnznH69Gl3D0d0EwEBAfTp06dDpRdcTQIoT2Dqhedne16SrQU0G3N3KQOd3kBadgmF5TVEBfuR1C8cjVr+dSuEpwkKCuKyyy6jrq7O3UMR3YBGo8HLy6vTzXZKAOUJBk6D8P4Q0qhAmLYcPnhAKWOwcEez3lH2JJCbmHbiuaMfXmpGHiu3ZZJXWmM+Fhvqx4qZicwYHtvKO4UQ7qDRaJo13RVCNJAAyhNct7L5MS9/+OlzwABVxQ11oYD80hryy2pQq2B4nB0zUMYAqtDFpQxSM/JY/M6hZr3Z80trWPzOIV6bN1aCKCGEEJ1K51ls7G40XhAQoXxeYVkL6vC5SwAMig62KwgylTIA1+3E0+kNrNyW2Sx4AszHVm7LRKeXHT9CCCE6Dwmg3E2vg5oyaGkLp7kWlGU1cnsKaDZlKmXgqmW8tOwSi2W7pgxAXmkNadklLhmPEEII4QgSQLlb6VlYHa88mgqyEkAZZ6BGtSeAcnEeVGG59eCpPecJIYQQnkACKHczlTDwDW7+mmkGqtESnl5v4IixhIE9CeQm5gDKRbWgbK1RZet5QgghhCeQAMrdzAFUSPPXAps3FD5VVEG5th4/bzWDooPsvp25lIGLcqCS+oUTG+qHtc2pKpTdeEn9wl0yHiGEEMIRJIByt5pWakAFRoJ3IBh05kPpxtmnEXGheGns/8/n6hkojVrFipktdyw3BVUrZiZKPSghhBCdigRQ7tZSFXKTScvgf87D9X8wH+pIAjlYljKoqq1v1zXsNWN4LCtuaR5ExYT6SQkDIYQQnZLUgXK31gKoFkradySBHBpKGVysquN0URWJvVpYOnSCUH+lfIKPRk2tTk9koA97np4iM09CCCE6JZmBcrfWAqimp9bpOJ6nLPm1J4HcxB098Uy9+24aqcw2FVXWoq3XtfYWIYQQwmO1K4Bat24dCQkJ+Pn5kZycTFpamtVzJ0+ejEqlava46aabLM47fvw4t9xyC6GhoQQGBjJhwgRycnJavc7DDz/cnuF7lsjLIPFW6DWm+WtVJbDpTvjHdWAwkJlXRp3OQESgD717+Lf7lv2MtaCyXRlAGWfOrhnUk4hApdv2qQvS1FgIIUTnZPcS3pYtW1i2bBnr168nOTmZtWvXMn36dLKysoiKimp2/tatW6mtrTU/Ly4uZtSoUdxxxx3mYydPnmTSpEksXLiQlStXEhISwrFjx/Dzs9zavmjRIp5//nnz84CAAHuH73lG/EJ5tMTbH058rnxeU9qogXBYh5ou9o1Qvm9nXNRUuLZez7Hzxpmz+DAG9AyiuLKEkxcq7GpFI4QQQngKuwOoNWvWsGjRIhYsWADA+vXr+eSTT3jzzTd55plnmp0fHm65PX3z5s0EBARYBFD/8z//w4033sif//xn87EBAwY0u1ZAQAAxMTH2Drnz8vYHn2CoLYfKIg6fLQfan0Bu4uoZqJ8Kyqmt1xPi50VCRAADogJJO13CSZmBEkII0UnZtYRXW1vLwYMHmTZtWsMF1GqmTZvGvn37bLpGSkoKc+bMITBQ+SOu1+v55JNPGDRoENOnTycqKork5GT+85//NHvvpk2biIyMZPjw4SxfvpyqKuszKFqtlrKyMvOjvLzcni/VdeqqwdBKH7jASOVjZSGHzxkLaHYwgHJ1KYP0JjNnA3oq9atOXqhwyf2FEEIIR7MrgCoqKkKn0xEdHW1xPDo6mvz8/Dbfn5aWRkZGBg888ID5WGFhIRUVFaxevZoZM2awfft2brvtNm6//Xa+/vpr83l3330377zzDl999RXLly/nn//8J/PmzbN6r1WrVhEaGmp+JCa2XIvI7f5xHTwfDie/avn1IGVZtLIkj2xjwDOqd8eWvVxdyuCIMf9ppHHc5gCqUAIoIYQQnZNLyxikpKQwYsQIkpKSzMf0xia6t956K0uXLgVg9OjR7N27l/Xr13PNNdcA8OCDD5rfM2LECGJjY5k6dSonT55scblv+fLlLFu2zPw8NzfXM4MobSkY9OBjpaq4sZ3L+dwcYBAJEQGEBfh06JauLmVwuEnrGVMAlV1UiU5vkFIGQgghOh27ZqAiIyPRaDQUFBRYHC8oKGgzN6myspLNmzezcOHCZtf08vJqFtwMHTrUYhdeU8nJyQD8/PPPLb7u6+tLSEiI+REc3EKvOU/QVhkDYwBVUpgLdHz5zsRVpQwqtfWcKFSWT01jj+vhj4+XGm29nvOXqp16fyGEEMIZ7AqgfHx8GDduHDt37jQf0+v17Ny5k4kTJ7b63vfffx+tVtts2c3Hx4cJEyaQlZVlcfynn36ib9++Vq+Xnp4OQGxsJ65irde33soFlCU8n2AKS5V8r47Uf2rMVYnkGbml6A0QE+JHdIiyq1KjVtHfeP+fJQ9KCCFEJ2T3Et6yZcu49957GT9+PElJSaxdu5bKykrzrrz58+cTFxfHqlWrLN6XkpLCrFmziIiIaHbNJ598krvuuourr76aa6+9ltTUVLZt28auXbsApczBu+++y4033khERARHjhxh6dKlXH311YwcObIdX7aHqC0HjAnk1gKoa57BMHk5K//4BVDL6D5hDrm1q0oZHDEmvo9skrc1oGcQP+aXc7KwgmsHNy9/IYQQQngyuwOou+66iwsXLvDss8+Sn5/P6NGjSU1NNSeW5+TkoG7SgiQrK4s9e/awffv2Fq952223sX79elatWsVjjz3G4MGD+eCDD5g0aRKgzFJ98cUX5mAtPj6e2bNn87vf/c7e4XsW0/Kdxhe8/Vo+R60m92IVRRW1eKlVJMY6Jl/JVTNQ6VZazwzoqdxfShkIIYTojNqVRL5kyRKWLFnS4mumWaPGBg8ejKG1rfrA/fffz/3339/ia/Hx8RY78roMG9u4mJKwh8aG4OetccitXZUDZdqB13TpcUCUlDIQQgjReUkzYXfy8lfauFjbgQdQUcjgnfezxaeEbfH/cNit+xkDqIIypZRBgI/jfxSKK7ScLVGSxEe0sIQHcEoCKCGEEJ2QBFDuFDkQ7ny79XM0Pgws3ctANZyLbX//u6ZCA7wJC/DmUlUdZ4qrGOqgpcHGjuQqM2f9IwMJ9fe2eM20hFhUUculqtoOl2YQQgghXKldzYSF69R7B1NrUOLccZGOLXrp7IrkjXv3NRXo60WvUCXvS5bxhBBCdDYSQLmTrr71Ni7AiQuVFKHMDvXxcWygk2DciXe62Dk78aztwDMx50EVSiK5EEKIzkUCKHfa/TeljctnzZswmxw+e4kigxKAqKuLHHr7hEjnzUAZDIZWZ6AA6YknhBCi05IAyp1qjG1cvKzn/6SfvUSxwZifVFHo0Nubl/CcsBMv91I1xZWtl15oKGUgAZQQQojORQIod7KhjEF6oxkoKi849PbmGSgnBFCm0gtDYoOtll5omIGSJTwhhBCdiwRQ7lRzSfloJYCqqq3np4JyiglF7xMMep1Db2/KgTKVMnAkU/2nka20njHlQOWUVKGtd+zXJoQQQjiTBFDuZJ6BCmvx5YzcMvQG2Oh/L+rfnoNrnnTo7cMCfAgLUMoLnHFwInm6Mf9pdCsBVFSwL0G+Xuj0BnKclMguhBBCOIMEUO5kCqB8W84RMiVhj+zTw2lDcEZFcp3eQIaxBtTIeOvLkyqVSvKghBBCdEoSQLmTtkz5aGUJz1ofOUfqZ1zGy3ZgU+GTFyqorNUR4KPhsqjgVs+VPCghhBCdkVQid6e+k6BHAgRFtfhyes4lAJJ6VMM7vwB9Hcz/yKFDMCWSO3IGyjRzNrxXKBq1qtVzG2pByQyUEEKIzkMCKHeatc7qSxfKteReqkalgiFxPeDDHaBSK4nkasc0FIaGUgbZDqwFddg8c9Z6k2SQUgZCCCE6J1nC81CmXWwDewYR1CMGUCk1o6qKHXofZ5QyaKhAHtbmuY2X8AxtVGUXQgghPIUEUO5iMLTaxsWiirfGCwLClRccXQvKwaUMtPU6jucpuV2jbcjd6hMRgEatokJbT2G5tsP3F0IIIVxBAih3KTyutHF5eVyLL6cbZ3HMCeSBxjwpB1cjd3Qpg+N55dTpDPQI8KZ3D/82z/f10tAnXAniJA9KCCFEZyEBlLuY2ri0MAvVuI+cuY5SYKTy0cEzUODYUgaNZ85UqtYTyE0kD0oIIURnIwGUu5iLaDavAXW6uIrS6jp8vNQMjjGWATDt1HNCAOXIUgaHbahA3pSUMhBCCNHZSADlLq3UgGooAxCCj5fxP1FgT6Xgpt6xLVfAOTNQo23YgWdiLmUgM1BCCCE6CSlj4C6tNBJOb5xAbjJ9FdzwglOG0i/SMaUMymrqOGW8RrtmoCQHSgghRCchM1Du0kojYdMymMUuNrXz/lP1NS7hdTSJPONcKQYDxIX5Exnka/P7TDlQ50trqNQ6foZNCCGEcDQJoNzFygxUbb2eY+eV5b1RdszidIRpBiq/rIbqWl27r3PYvHPQ9uU7UHYCRgb5AI4t6CmEEEI4iwRQ7tIjAfpdA5GDLQ7/mF9Gbb2esABv88wQACXZ8M5s2HSHw4cSFuBDqL+xlEFJ+wMY8w68dgR+/XtKHpQQQojOQwIod5nwANz7MYy9x+Jw4yDEogyASg0/fwGnvm61AGd7mSuSd2AG6Eg7duCZSB6UEEKIzkQCKA+i0xvYkVkAQESQDzp9o0DJVMZAp23YwedAporkp9uZB1VYXsP50hpUKhjR274lPGhcC0qW8IQQQng+CaDcpcksUmpGHpNe+JJvThQBsPVQLpNe+JLUjDzlBG9/8DHWhKoscvhwTE2F2zsDdeSskv80sGcQQb72b+6UUgZCCCE6Ewmg3GVdEqzuC7kHSc3IY/E7h8grrbE4Jb+0hsXvHGoIokzVyB3czgUgIdI0A9W+AMq0c3CUDf3vWjLQuIR3qqjScuZNCCGEsMNrr73GyJEjCQkJISQkhIkTJ/LZZ5+ZX6+pqeGRRx4hIiKCoKAgZs+eTUFBgd33kQDKXapKoOYSOo0fK7dl0lLIYDq2clumElSYq5E7IYAyz0C1bwnPvAOvHct3AL3C/PH1UlNbryf3YnW7riGEEEL07t2b1atXc/DgQb7//numTJnCrbfeyrFjxwBYunQp27Zt4/333+frr7/m/Pnz3H777XbfRwppuoPBYC5jkF5oaDbzZHEqkFdaQ1p2CRMDeyoHndHOpUkpA38fjc3vNRgM5gTy9s5AadQq+kUG8mN+OScvVNCn8Q5EIYQQwkYzZ860eP7HP/6R1157jf3799O7d29SUlJ49913mTJlCgAbNmxg6NCh7N+/n8svv9zm+8gMlDvUVYO+DoC8Wh+b3lJYXmNs5xIKOscXm+xIKYOckiouVdXho1EzJKZ5bz9bSR6UEEIIa8rLyykrKzM/tFptm+/R6XRs3ryZyspKJk6cyMGDB6mrq2PatGnmc4YMGUKfPn3Yt2+fXeORAModTEU0VRoiwsJtektUsB/ctAaW58DlDztlWA2lDOxbxjO1nhnauHdfOwyQWlBCCCGsSExMJDQ01PxYtWqV1XOPHj1KUFAQvr6+PPzww3z44YckJiaSn5+Pj48PYWFhFudHR0eTn59v13ja9ddu3bp1JCQk4OfnR3JyMmlpaVbPnTx5MiqVqtnjpptusjjv+PHj3HLLLYSGhhIYGMiECRPIyckxv+6opC+P0KgKeVL/CGJD/VBZOVUFxIb6kdQv3KntXKBxKQP7ZqCOdDD/ycRUyuBnqQUlhBCiiczMTEpLS82P5cuXWz138ODBpKenc+DAARYvXsy9995LZmamQ8dj91/kLVu2sGzZMlasWMGhQ4cYNWoU06dPp7Cw5cTmrVu3kpeXZ35kZGSg0Wi4446GitonT55k0qRJDBkyhF27dnHkyBH+93//Fz8/P/M5jkr68giNAiiNWsWKmYktnmYKqlbMTESjthZiOY4pkfyMnQFURyqQN9YwAyW1oIQQQlgKDg4276wLCQnB19d6z1UfHx8GDhzIuHHjWLVqFaNGjeLFF18kJiaG2tpaLl26ZHF+QUEBMTExdo3H7gBqzZo1LFq0iAULFpCYmMj69esJCAjgzTffbPH88PBwYmJizI8dO3YQEBBgEUD9z//8DzfeeCN//vOfGTNmDAMGDOCWW24hKkrZdVZaWkpKSgpr1qxhypQpjBs3jg0bNrB37172799v75fgfl6+ShuX3hMAmDE8lt/PGt7stJhQP16bN5YZw2OVA0Un4J+3w3t3O2VYplIG9vSjq9fpyTjfvh54TfU3zkCVVNZSUlnboWsJIYQQJnq9Hq1Wy7hx4/D29mbnzp3m17KyssjJyWHixIl2XdOuXXi1tbUcPHjQYtpMrVYzbdo0m5OvUlJSmDNnDoGByh9LvV7PJ598wlNPPcX06dP54Ycf6NevH8uXL2fWrFkAbSZ9tZQ1r9VqLRLMysvL7flSnavXaKWNSyOmXXC9Qv14+oYhRAUry3YWM08GA5zcCb7tT9RuTXtKGfxUUEFNnZ4gXy/6RwZ16P4BPl7EhfmTe6maUxcqCA+0LT9MCCGEMFm+fDk33HADffr0oby8nHfffZddu3bx+eefExoaysKFC1m2bBnh4eGEhITw6KOPMnHiRLt24IGdM1BFRUXodDqio6MtjtuafJWWlkZGRgYPPPCA+VhhYSEVFRWsXr2aGTNmsH37dm677TZuv/12vv76a4B2JX2tWrXKItksMbHlZTJPcbZECVoGxQRz6+g4Jg6IaL5sF2QsY6AtgzrrpQ/ayxRAmUoZ2MJUvmBEXChqBywz9je3dJE8KCGEEPYrLCxk/vz5DB48mKlTp/Ldd9/x+eefc9111wHw97//nZtvvpnZs2dz9dVXExMTw9atW+2+j0vrQKWkpDBixAiSkpLMx/R6PQC33norS5cuBWD06NHs3buX9evXc80117TrXsuXL2fZsmXm57m5uR4dROUYA6g+4a3UP/ILA7W3UgKh8gKExTt0DD0ClVIGpdV1nCmptKkkQUcrkDc1oGcQu08USR6UEEKIdklJSWn1dT8/P9atW8e6des6dB+7ZqAiIyPRaDTNdr/ZknxVWVnJ5s2bWbhwYbNrenl5NQtuhg4dat6F156kL19fX4tks+DgYFu+RNfY+Ty8kADf/MV8yBRAxfdoJYBSqZRaUOCUYprQaCeejct4h886ZgeeibkWlOzEE0II4cHsCqB8fHwYN26cRfKVXq9n586dbSZfvf/++2i1WubNm9fsmhMmTCArK8vi+E8//UTfvn0BHJr05RGqiqH6okVD4bPG9iXxrc1AQcMynrMCKFMtKBt24lXX6sgqUHLLHDcDJUt4QgghPJ/dS3jLli3j3nvvZfz48SQlJbF27VoqKytZsGABAPPnzycuLq5ZgauUlBRmzZpFREREs2s++eST3HXXXVx99dVce+21pKamsm3bNnbt2gXg0KQvj9CojIHJWVuW8AACTf3wnBNA9bWjlEFmXik6vYHIIF9iQ/3aPN8WpqbCOSVVaOt1+HrZ3lJGCCGEcBW7A6i77rqLCxcu8Oyzz5Kfn8/o0aNJTU01J5bn5OSgblLwMSsriz179rB9+/YWr3nbbbexfv16Vq1axWOPPcbgwYP54IMPmDRpkvmcv//976jVambPno1Wq2X69Om8+uqr9g7fMzQJoCq09eZt+/Hh/q2/NyhKeZ/OOdv8+9lRyiDduHw3Oj4Ulcoxdap6BvsS7OtFubaeM8VVDIr2oKVXIYQQwqhdSeRLlixhyZIlLb5mmjVqbPDgwRgaLVe15P777+f++++3+rqjkr48QpMAyjT71CPAm2A/79bfe8srMMt5gWPDDFTbOVCmHXgjO1hAszGVSkX/qCAOn73EycIKCaCEEEJ4JOmF5w6mAMpYz8mcQN7W8h04vZ1LP2MAlVfadikDcwVyB+U/mUgelBBCCE8nAZQ71JQpH5vMQNkUQDlZWIA3IX7KxKQpsGvJpapaThtnqUbGOWYHnom0dBFCCOHpJIByh16jIXY0BCiVts/aUsLApCAT/nkb/OtepwxNpVKZq6K3lgdlaiDcNyKAHoE+Dh1DQwAlM1BCCCE8k0sLaQqjue9bPDWVMGhzBx6AQQcnv2yoB+UECZGBHD5X2upOPGfkP5kMjDIu4RVWYDAYHJagLoQQQjiKzEB5AJuqkJuYyhhUFYPetnYr9jIlkrdWCyrdwQU0G+sTHohGraKyVkdBmbbtNwghhBAuJgGUmxkMhkY5UG2UMAAIiABUYNBDVYlTxmQqZWCtGrnBYHB4C5fGfLzU9DUGk7KMJ4QQwhNJAOVq5w4qbVw23ATAhXIt2no9ahX0CrMhgNJ4mXOnnF1M09oMVH5ZDRfKtWjUKob1artfXnuYW7pIACWEEMIDSQDlatUXlYdW2YlnWr6LDfXHW2Pjfw5zP7xCZ4zQopRBTV3zZUJT/7vLooII8HFOGp05kVx64gkhhPBAEkC5Ws0l5aOphMFFO/KfTMwBVJEDB9agcSmDlgpqmpbvRjth+c6koRaUlDIQQgjheSSAcrUmVchziu3YgWcSFAV+YVDvnATrxqUMWlrGc+YOPBNXLeHp9Ab2nSzmo/Rc9p0sRqdvvWK+EEIIAVLGwPW0lkU0c+xJIDe5/R9Or0jeN0IpZXC6SS0ovd7AEdMOvHjH78AzGRCpBFB5pTVUaOsJ8nX8j2pqRh4rt2WSV1pjPhYb6seKmYnMGB7r8PsJIYToOmQGytWa9sG72I4q5E4OnkCpBQWYq42bZBdXUq6tx9dL7dQ+daEB3kQG+Sr3dMIyXmpGHovfOWQRPAHkl9aw+J1DpGbkOfyeQgghug4JoFzNSiNhT2jj0lhChKmUgWXwYup/Nzwu1Pak93ZyVk88nd7Aym2ZtLRYZzq2clumLOcJIYSwSgIoVwvppbRxCeuDtl5HfpkyA2JXDlTeYXh7FnzwgFOGCI1noCwDKFMLl5FOKKDZlLPyoNKyS5rNPDVmQFk6TMt2Tp0tIYQQnZ/kQLna1U8qDyD3QgUGAwT4aIiwp5+crg5OfQWh8U4aJCQ0KWXg560BIN04A+XMHXgmzuqJV1huPXhqz3lCCCG6H5mBcqOcRk2E7er3ZipjUFEIBucsM/VooZRBbb2ezDwlCd6ZO/BMzEt4hY7NgYoK9nPoeUIIIbofCaDcyNRE2O78J1MApdOCttzBo1KoVKpmy3hZ+eXU1usJ8fMy50g5k2kGKruo0qH5SEn9wokN9cNayKpC2Y2X1C/cYfcUQgjRtUgA5WrrkuHFUXDhJ/t64DXmEwA+SnDhrHYu0LCMZ0okb9z/zq4Zs3aKC/PH10tNrU7PuYst9+VrD41axYqZiS0mkZu+qhUzE9Gonf81CiGE6JwkgHK1i2fg4mnw8jEHUHYlkJsERiofK5zTzgUa7cQzLuGZduCNcsHyHYBaraK/cRbqZwe3dJkxPJZbRjWv9RQR5Mtr88ZKHSghhBCtkgDKleq1UK8s2+EXas6Bal8AFaV8dOYMVKTlDJQrd+CZOLOUwcEzlwD41eQB9DX+N1gpRTSFEELYQAIoV6opa/jcN6RRFfJ2BFBBUeDfA3S1Dhpcc32NS3hniiup1NZzolDJt3LFDjyThqbCjk0k/+rHQnIvVdMjwJvHpl7GmD5hAOQY89KEEEKI1kgZA1cyFdH0DaG0Rk95TT2g7MKz253/dHpFclM/vPOlNXx/5iJ6g5JcHRXiut1pzqoF9c6BMwDcMT4eP2+NRbAohBBCtEVmoFypURVy0+xTZJAv/j4a+6/lgnYujUsZbDt8HnDt8h04ZwnvbEkVX/+kLH3+MqkPAAmRShCbXSQBlBBCiLZJAOVKNZeUj36h5h54fezdgedCjUsZfJ6RDyg78Fypv7Gp8MWqOkoqHbNc+W5aDgYDXHVZpHmWrWEGynG7/YQQQnRdEkC5ksYHYkdBz8EdSyAHOPc9vH0r/OdXDhxgc6ZSBuVaZbnRVTvwTPx9NMSFKUGmI2ahtPU6/vXdWQDmJvc1H+9n/Drzy2qortV1+D5CCCG6NgmgXKnfVfDQN/CLNzuWQA5QXwOndkHOfseNrwVNZ8gSe4U49X4tMedBOaCUQWpGPsWVtcSE+DFtaJT5eFij5UrTfxshhBDCGgmg3ORsRwMocxmDIgeNqLnUjDz+uT/H4tiNL+4mNSPPafdsiSPzoDYZv545SfF4aRp+/FuqvC6EEEJYIwGUm5xt1AevXYKM7Vy0pVDn+Ka3qRl5LH7nEKXVdRbH80trWPzOIZcGUQ1NhTsW2GTll5N2ugSNWsWcCX2avd63SeV1IYQQwhoJoFxpxwp4cRT6tH+Qe0mpN9SnvT3l/MJA7a18XuXYWSid3sDKbZkttjoxHVu5LdOh/ela0xBAdWwGapOxdMF1Q6OJCW1eiqFp5XUhhBDCGgmgXKnsPFw8TXlFOXU6A94aFTHtramkUjU0FXZwO5e07BLySq3PahmAvNIa0rJLHHpfawZEKTNDZ0uqqKlrX4J3pbaerYdyAZh3ed8Wz0mQWlBCCCFsJAGUKxnrQBXVKUFTXJh/xxrWmpbxHJwHVVhu25Kgred1VM8gX4L9vNAb2l9m4KP081Ro6+kXGcgVAyJaPMdUC0pKGQghhGhLuwKodevWkZCQgJ+fH8nJyaSlpVk9d/LkyahUqmaPm266yXzOfffd1+z1GTNmWFwnISGh2TmrV69uz/DdxxhA5df6Ah1IIDcJigb/8Ib+eg4SFWzbrJit53WUSqXq0DKewWDgnf3K8t3dSX1QWwlaTTlQ50ur2z3TJYQQonuwu5XLli1bWLZsGevXryc5OZm1a9cyffp0srKyiIqKanb+1q1bqa1tKIBYXFzMqFGjuOOOOyzOmzFjBhs2bDA/9/X1bXat559/nkWLFpmfBwcH2zt89zIGULk1PoADAqhfbnFKRfKkfuHEhvqRX1rTYh6UCogJ9SOpX7jD723NgJ5BpJ+91K5SBulnL5GZV4aPl5pfjOtt9byIQB+CfL2o0NZztqSKy6I72c+XEEIIl7H7r++aNWtYtGgRCxYsIDExkfXr1xMQEMCbb77Z4vnh4eHExMSYHzt27CAgIKBZAOXr62txXo8ePZpdKzg42OKcwMBAq+PUarWUlZWZH+Xl5fZ+qY5nDKByKpW4td1FNE2c1M5Fo1axYmYioARLjZmer5iZ2LHlRzuZ8qDaMwP1jrF0wc0jY+kR6GP1PJVKRV9JJBdCCGEDu/4C19bWcvDgQaZNm9ZwAbWaadOmsW/fPpuukZKSwpw5c5oFP7t27SIqKorBgwezePFiiouLm7139erVREREMGbMGP7yl79QX19v9T6rVq0iNDTU/EhMTLTxq3QibRkAJ8scFEA50Yzhsbw2b2yz3WoxoX68Nm8sM4bHunQ87S1lcKmqlv8eUfr4WUseb8xUC0oSyYUQQrTGriW8oqIidDod0dHRFsejo6P58ccf23x/WloaGRkZpKSkWByfMWMGt99+O/369ePkyZP89re/5YYbbmDfvn1oNEqj3ccee4yxY8cSHh7O3r17Wb58OXl5eaxZs6bFey1fvpxly5aZn+fm5ro3iNLrIWIA1JTyU5nyNbW7BpTJmX2waxWE94OZLzpgkJZmDI/lusQY0rJLKCyvISpYWbZz5cyTycCohhwog8GASmXbGP598Bzaej2JsSGMsaGPX0MpAwmghBBCWGd3DlRHpKSkMGLECJKSkiyOz5kzx/z5iBEjGDlyJAMGDGDXrl1MnToVwCIYGjlyJD4+Pjz00EOsWrWqxXwpX19fi+NlZWWO/nLso1bDQ99QXavj5LOpgANmoOqqIPtrp1Yj16hVTLSya82V+oQH4KVWUVWrI7+shtjQtpsw6/UGNh1Qlu/mXd7XpqBLmgoLIYSwhV1LeJGRkWg0GgoKCiyOFxQUEBMT0+p7Kysr2bx5MwsXLmzzPv379ycyMpKff/7Z6jnJycnU19dz+vRpm8buKc5eVP4wB/t5ERrg3bGLBZnauVzo4Kg8n7dGbc5POllo2+zQ3pPFZBdVEuTrxa2je9n0HlMtqGypRi6EEKIVdgVQPj4+jBs3jp07d5qP6fV6du7cycSJE1t97/vvv49Wq2XevHlt3ufcuXMUFxcTG2s9zyY9PR21Wt3izj9PZmrh4pD8J1Mhzaoi0Hf9bff2ljIwlS64fWwcgb62TbaalvDOX6pGW9/1v6dCCCHax+4lvGXLlnHvvfcyfvx4kpKSWLt2LZWVlSxYsACA+fPnExcXx6pVqyzel5KSwqxZs4iIsFwOqqioYOXKlcyePZuYmBhOnjzJU089xcCBA5k+fToA+/bt48CBA1x77bUEBwezb98+li5dyrx581rcreeRzuyF//yKfj6DgHsdE0AFRCofDXqovgiBkR2/pgcbEBUEmQU2BVD5pTXsOK7MlNqSPG7SM9iXAB8NVbU6zl2sNgdtQgghRGN2B1B33XUXFy5c4NlnnyU/P5/Ro0eTmppqTizPyclB3WR7fVZWFnv27GH79u3NrqfRaDhy5AgbN27k0qVL9OrVi+uvv57f//735hwmX19fNm/ezHPPPYdWq6Vfv34sXbrUIi/K41VegIvZeAWFAA6oAQWg8VIKaVaXKO1cunoAZccM1ObvctDpDSQlhDPIjnpOSimDQI7nlXGmuFICKCGEEC1qVxL5kiVLWLJkSYuv7dq1q9mxwYMHYzC03HjW39+fzz//vNX7jR07lv3799s9To9irAF1UaeUBXBIAAVKHlR1SbfIgxrQ01gLqo0cqHqdns1pZwGYe3kfu++TEBHA8bwyThdJIrkQQoiWSS88VzH1watXdo/F92h7F5lNgqIgIALqXdOXzp36G2eD8stqqNBarwH2xfFC8stqiAj0Ycbw1jc3tMS0E09KGQghhLDGpWUMurUapYyCqQ+ew4pozv8YbKyJ1NmF+nvTM9iXC+VaTl2oYGTvsBbP23RASR6/Y3w8vl4au++TINXIhRBCtEFmoFzFOANVovNHpYI4R81AdZPgycS8jGclD+p0USW7TxShUsHcZPuX70CqkQshhGibBFCuYgygygwBxIT4tWtmRDRKJLeSB/VumlI485pBPdudZ2aqBXXuYjV1On27riGEEKJrkwDKVQLCqQyI54IhzHEJ5ADZ38DGmfDpk467pgczBVA/Fzafgaqp0/H+90ry+Lxk20sXNBUV7Iuftxqd3kDuxep2X0cIIUTXJQGUq8xYxYbxH/If/aSO98BrTFuhBFHnvnfcNT3YgCjrpQw+PZrHxao64sL8uXZI+wusqtUq+oYbK5LLMp4QQogWSADlQmdLlNkMhyWQQ6N2Ls7rh+dJTDlQp4srqW+yvGaqPP7LpPgONzw2tY05Iy1dhBBCtEACKBfKMbZxiQ93UAI5NBTPrCwEK7W2upJeof74eaup0xk422h5LfN8GYdyLuGlVnHnhPgO36dfpClQk514QgghmpMAylVem8SKvF8RTYljZ6BM/fDqa6DWth5xnZlaraJ/pCmRvOHrfcdYumD68Biigv06fB9TLSjZiSeEEKIlEkC5gl4HBUcZYjhJHV6ODaB8AsFb+WNPRaHjruvBmuZBldfU8Z8fcoH2ly5oylQL6ozMQAkhhGiBBFCuoC0zf1rrFUTPYF/HXj/IOAvVDdq5QPNaUP/5IZeqWh0DegYysX9Ea2+1WV/jEl5OSVWzXCshhBBCAihXMNaAqjL4EhMegsrRxS+DYiAgEuq6x5b7hqbClRgMBt7Zr9R+mpvc12Hf29gQP3y81NTrDZy/1PXb5AghhLCPtHJxBVMRTQIc1wOvsftTu1VF8sa1oL4/c5GsgnL8vNXMHtfbYfdQShkEcKKwgtPFlfSJcOCyqxBCiE5PZqBcoVEVcofmP5l0o+AJGnbIlVbX8bv/ZABw88hYQv29HXofSSQXQghhjQRQrmCegQp0bBXyburrnwrRGGPGrPxyAL7KukBqRp5D7yNNhYUQQlgjAZQrqDTkqWM4b4hwTgD18xfw1s2Q+lvHX9vDpGbksfidQ+ialLwqqahl8TuHHBpEmRLJT0sxTSGEEE1IDpQrDLmRG/HiYl0dnzkjgKopg9O7lXIJXZhOb2DltkxaKhdqAFTAym2ZXJcY0+FK5NB4BkoCKCGEEJZkBsoFymvquFhVB+CcGShzO5euXQcqLbuEvFLrO+IMQF5pDWnZJQ65X4IxB+psSTU6fdev8i6EEMJ2EkC5gKkHXnigD0G+Tpj0M1Ujr+jadaAKy20rJ2DreW3pFeaPt0ZFrU5PXmn3KBEhhBDCNhJAuYDfrpV87PM/3B3wnXNuYAqgtKVQr3XOPTyArS1aHNHKBUCjVplnDKUiuRBCiMYkgHIBdcnPjFRnE+df55wb+IWB2jiz1YWrkSf1Cyc21A9r2U0qIDbUj6R+4Q67p2kZL1sSyYUQQjQiAZQLGIxlDAJCHdNmpBm1umEWqgsHUBq1ihUzEwGaBVGm5ytmJjokgdwkQWpBCSGEaIEEUC6g0SoBVEhYpPNuEhStBFG1XXupacbwWF6bN5aYUMtluphQP16bN5YZw2Mder+ESKkFJYQQojkpY+ACPvVK09vwiJ7Ou8mDu7pNRfIZw2O5LjGGtOwSCstriApWlu0cOfNkItXIhRBCtERmoJxMrzcQoFcCqKieTgyguknwZKJRq5g4IIJbR8cxcUCEU4InaKgFdaa4Cr2UMhBCCI+3atUqJkyYQHBwMFFRUcyaNYusrCyLc2pqanjkkUeIiIggKCiI2bNnU1BQYNd9JIByssKyaoJQtsD3jIxy82iEveLC/PFSq9DW68kvc0x5BOG5dHoD+04W81F6LvtOFkv9LyE6oa+//ppHHnmE/fv3s2PHDurq6rj++uuprGxYSVi6dCnbtm3j/fff5+uvv+b8+fPcfvvtdt1HlvCcLPdCMbWGSMJU1YQEhDnvRlmfwb51EDcOrlvpvPt0M14aNfHhAWQXVXK6uJJeYf7uHpJwktSMPFZuy7Qo1hob6seKmYkOz60TQjhPamqqxfO33nqLqKgoDh48yNVXX01paSkpKSm8++67TJkyBYANGzYwdOhQ9u/fz+WXX27TfWQGyslOl6m4uvZFHo77ALwdU5+oRdWXlHYueenOu0c31TdCakF1daYei00r3eeX1ji8x6IQon3Ky8spKyszP7Ra2+oelpYqG7nCw5USNwcPHqSuro5p06aZzxkyZAh9+vRh3759No9HAignO3tR+aPbxxktXBoLMpUxKHLufbohUykD6YnXNbXVYxGUHouynCeEeyUmJhIaGmp+rFq1qs336PV6nnjiCa688kqGDx8OQH5+Pj4+PoSFhVmcGx0dTX5+vs3jkSU8J8spUQIop/TAayzQmF9V0bX74bmDeQaqSGaguiJ7eixOHOCkWm5CiDZlZmYSFxdnfu7r69vmex555BEyMjLYs2ePw8cjAZSTheZ9y8c+bxB09grgVefdyFRIs6oI9DpQa5x3r25GZqC6Nlf3WBRCtE9wcDAhISE2n79kyRL++9//8s0339C7d2/z8ZiYGGpra7l06ZLFLFRBQQExMTE2X79dS3jr1q0jISEBPz8/kpOTSUtLs3ru5MmTUalUzR433XST+Zz77ruv2eszZsywuE5JSQlz584lJCSEsLAwFi5cSEVFRXuG71KqsjxGqrOJ1J5z7o0CjUU6DXqovujce3UzCZENAZTBIMs4XY2reywKIZzLYDCwZMkSPvzwQ7788kv69etn8fq4cePw9vZm586d5mNZWVnk5OQwceJEm+9j9wzUli1bWLZsGevXryc5OZm1a9cyffp0srKyiIpqvk1/69at1NbWmp8XFxczatQo7rjjDovzZsyYwYYNG8zPm07NzZ07l7y8PPOWxAULFvDggw/y7rvv2vsluExNnQ5DzSXwBt+gHs69mcYb/MOhukRZxgt0YtXzbiYuzB+NWkVNnZ7Cci3RIfKHtCsx9VjML61pMQ9KhVLp3pE9FoUQzvPII4/w7rvv8tFHHxEcHGzOawoNDcXf35/Q0FAWLlzIsmXLCA8PJyQkhEcffZSJEyfavAMP2jEDtWbNGhYtWsSCBQtITExk/fr1BAQE8Oabb7Z4fnh4ODExMebHjh07CAgIaBZA+fr6WpzXo0dDwHH8+HFSU1P5xz/+QXJyMpMmTeLll19m8+bNnD9/vsX7arVai2z98vJye7/UDsu9VE0ISt6MT2CY828YHGNs5yJLTY7k46Umzli+4LQ0Fe5yGvdYbMpZPRaFEM7z2muvUVpayuTJk4mNjTU/tmzZYj7n73//OzfffDOzZ8/m6quvJiYmhq1bt9p1H7sCqNraWg4ePGix9U+tVjNt2jSbt/6lpKQwZ84cAgMDLY7v2rWLqKgoBg8ezOLFiykuLja/tm/fPsLCwhg/frz52LRp01Cr1Rw4cKDF+6xatcoiWz8xseVfkM6UU1JFsEoJoFT+oc6/4eK98OTPED/B+ffqZqSUQdc2Y3gsr84d26ygf3SIc3osCiGcx2AwtPi47777zOf4+fmxbt06SkpKqKysZOvWrXblP4GdAVRRURE6nY7o6GiL47Zu/UtLSyMjI4MHHnjA4viMGTN4++232blzJy+88AJff/01N9xwAzqdDlC2HDZdHvTy8iI8PNzqfZcvX05paan5kZmZac+X6hBnS6rMM1D4uSCA6mbtXFzJlEieLYnkXdbI+DAMBlCrINhXyW74+52jJHgSQrTIpbvwUlJSGDFiBElJSRbH58yZY/58xIgRjBw5kgEDBrBr1y6mTp3arnv5+vpa5FGVlZW1b9AdcLakinEqFwZQwmlMieTSVLjryshViu0Njgmhb3gAqcfyOZJbysSBkk8ohGjOrhmoyMhINBpNs4Z7tmz9q6ysZPPmzSxcuLDN+/Tv35/IyEh+/vlnQNlyWFhoWd+ovr6ekpISu6fcXCmnpAot3mi9QsDfyUnkAJkfw1s3w1d/cv69uhlTU+HT3bwWVFfuFXfMGEAN7xXC6D5hAKSfveS+AQkhPJpdM1A+Pj6MGzeOnTt3MmvWLECp8rlz506WLFnS6nvff/99tFot8+bNa/M+586do7i4mNhYZep84sSJXLp0iYMHDzJu3DgAvvzyS/R6PcnJyfZ8CS51tqSax+uW8Obc8UwZEt32GzqqukRp5+IT2Pa5wi59IxpmoAwGA6puuFza1XvFHTUGUCN6hzI4OhiAH3IuuXFEQghPZvcuvGXLlvHGG2+wceNGjh8/zuLFi6msrGTBggUAzJ8/n+XLlzd7X0pKCrNmzSIiwrKSb0VFBU8++ST79+/n9OnT7Ny5k1tvvZWBAwcyffp0AIYOHcqMGTNYtGgRaWlpfPvttyxZsoQ5c+bQq1ev9nzdTmcwGDhb4qI2LiamauSVF1xzv24kPtwflQoqa3UUVdS2/YYupjv0iss4ryzzD+sVyojeoWjUKvLLashvpUq5EKL7sjuAuuuuu/jrX//Ks88+y+jRo0lPTyc1NdWcWJ6Tk0NenuUv06ysLPbs2dPi8p1Go+HIkSPccsstDBo0iIULFzJu3Dh2795tkcO0adMmhgwZwtSpU7nxxhuZNGkSr7/+ur3Dd5lLVXWUa+sB6N3DVQGUsRp5hQRQjubrpaFXqLGUQTfLg+oOveIKymq4UK5FrYLE2BACfLwYZJyFSj8rhWmFEM21K4l8yZIlVpfsdu3a1ezY4MGDrVZw9vf35/PPP2/znuHh4R5dNLOpsxerUKFnm/9K/N57He54y/l5UOaGwoVgMMiuPAfrFxlI7qVqThdVMiGh+xRV7A694kwJ5AOjgvD3UdogjekTxvG8Mn7IudQlliiFEI7VrlYuom05JVUEUsNwwwk4tQs0bTc97DDTDFR9DdR6fpubzqa71oLqDr3iMnKV5bvhvRp2y46ODwPgB0kkF0K0QAIoJzlbUk0w1coTtTd4+zv/pj6B4G1MIK8obP1cYbfu2lS4O/SKMyWQD49rCKDGGAOoo+dKqdfp3TEsIYQHkwDKSXJKqghRGf/Q+oW6bjktJBaComUGygm66wyUqVecNSqU3XiduVfcsfPNA6gBPYMI9vWiuk5HVoHrW0EJITybBFBO4vIq5CZLvoff/ASxo1x3z27CVEzzdFGl1Zy+rqir94orqtCSV1qDSgWJvULMx9VqFaOMs1BSD0oI0ZQEUE5y9mKTGShXkcRxp+kTHoBKBeXaekoqu1cpg2lDowkwJlc3Fhnk2+l7xZkSyPtFBhLka7mvxpQHlS71oIQQTUgA5QQ6vYHci9XumYESTuPnrSE2RFnKOt3NlvHSTpdQVaujR4A37yxMYmiMssV/waSETh08ARw73zyB3GSMVCQXQlghAZQT5JVWU6834KM2YPALgwAX5oZkfAAbboKv/+K6e3YjjSuSdyfbjyntm6YNjWbSZT25O7kPAF/92Pk3Kxw9Z6xAHtc8gDLNQP18oYKymjpXDksI4eEkgHKCHGMF8rSwG1A9cwZ+8abrbl5ZBGf2QMFR192zG0mINPbE60YzUAaDgR2ZSgB1/TCl9+S1Q5Sq9wfPXORiJ1/OzDAmkA+LC2n2WkSQL/Hh/hgMcORsqauHJoTwYBJAOYGphUu8q1q4NCbVyJ3KNAN1uqj7zEAdO19G7qVq/L01XHVZJKBU1x8SE4zeAF//1Hl/1i5V1XLuolJuZFgLS3gAY+KVArhSkVwI0ZgEUE5wtkT5hdwn3AW1n5oKbFSNXDhcQjdcwttunH26elAkft4NieRThyqzUF8cL3DLuBzBVECzb0QAof7eLZ5jLqgpieRCiEYkgHIC0xLe7JIUePtWOPGF624eJA2Fnak7LuFtP5YPwPWJMRbHpw5V+l9+/dMF6jppoUnT8l1LCeQmoxslknen8hVCiNZJAOUEZy8al/BqspQ2LtUlrru5aQaqphTqta67bzfRx7gsW1pdx6Wqzp37Y4uzJVX8mF+ORq1iijHvyWRU7zAiAn0or6nnu9Mu/Bl3IFMF8pbyn0yG9QrBR6OmuLJhuU8IISSAcgJTDlSA3lgN3JVlDPzCQG2sZSOzUA4X4ONFdIjS17A7zEKZlu8mJPSgR6CPxWsatYrJg5Wg6svjnXPJ+Fiu9R14Jr5eGoYaC2weypE8KCGEQgIoB6vU1lNUocxM+OqM7R9cGUCp1RAcC0ExoJV2Ls7gikRynd7AvpPFfJSey76Txej07lk6srZ8ZzLNmAe1sxOWMyirqTMHwa0t4UFDXzypByWEMPFq+xRhD9MUf6i/NxqtkqDq8kKaTxyViuRO1C8ikLTsEqc1FU7NyGPltkzySmvMx2JD/VgxM9GlRStLKmvNS3PXJUa3eM6kyyLx1qjILqrk1IUK+vcMctn4OuqYMYE8Lsy/2exaU2P6hPHWXgmghBANZAbKwUwJ5PE9/JQ8JABf6/kVTiHBk1P1jXReU+HUjDwWv3PIIngCyC+tYfE7h0jNyHP4Pa3ZebwAvQESY0OsluQI9vPm8v4RAHzZyWahGhoIt/3/p2kn3rHcMrT1OmcOSwjRSUgA5WCm/KeBPdSgr1cOSiuXLsVUysDRM1A6vYGV2zJpabHOdGzltkyXLedtNxfPbHn2ycSUXN7ZyhmYEsjbWr4DZfNAeKAPtTo9x/PKnT00IUQnIAGUg5lmoPqFAP49QOMLPoGuHcThzfDmDbDn7669bzfRN8I5M1Bp2SXNZp4aMwB5pTWkZTt/x1t1rY7dJ5RNCNbyn0ymDlECrO9OX6S0uvO0OzE1ER7eu+0ASqVSNWosLInkQggJoBzONAMVER0PT5+G3xW4fkmtogBy9kJBpmvv202YkshLKmsdGjAUllsPntpzXkd8c+ICNXV6evfwZ2hscKvn9okI4LKoIHR6Q6epSl6preeUcROALTNQ0KigpuRBCSGQAMrhTDWgTPWC3JKPFGgqptm5clI6iyBfL3oGK6UMHFmRPCrYz6HndYS5911iDCobfoanDDWVM+gcy3iZeWUYDBAd4mv+b9mW0bITTwjRiARQDmQwGMxtXNzSB88kyNTOpch9Y+jiEiIcX5E8qV84saF+WAtXVCi78ZL6hTvsni2p1+nZaQyErO2+a2qasSr5V1kXqO8EVckzbKj/1NQoYwB1priKkk7eQFkI0XESQDlQUUUt1XU6VCroXbxXaeOy6wXXD8TcUFhmoJzFtIx3xoG1oDRqFStmJraYRG6yYmYiGrVzZzW/P3ORi1V1hAV4MyGhh03vGRMfRliAN6XVdRzqBD3jTD3wrDUQbkmovzcDeir/3aWxsBBCAigHMiWQx4b44V16WmnjUpDh+oH4K9vKqbwAp74BvWy7djRnzEABTB8WY6503tTvZw13SR2o7ceU2aepQ6Lx0tj2K8JLo+bawaaimp6/jNeeGSiA0fFKQJneCYJEIYRzSQDlQKYE8vjwgIYaUK4uYZD5MaRcZ3xigLdnwtrhynHhMH2dVMrg25+LKSjTEuij4c37JvDinNGMNO4S+7nQ+ZXlDQYD2zON1cfbKF/QlKmcwU4Pb+tSXavjRKFSimC4vQGUsbGwJJILISSAciBTANXHXQFU5sfwr/lQft7yeFmeclyCKIfpF2lcwnNwAPX2vtMA/GJcb6YMieLW0XE8OX0wAP/6/iylVc4tE3A8r5xzF6vx81Zz9WU97Xrv1YN64qVW8XNhhcO/L450PL8MvQEig3yszvZZ07ili95N7XWEEJ5BAigHymk8A+XqNi56HaQ+Da2VYUx9RpbzHKSPcQmvqKKW8hrHBDXnLlaZi1HeMzHBfHzSwEiGxARTVatjU9oZh9zLGtPs01WX9cTfR2PXe0P9vZmQoCS4e/IslKmB8PC4UJt2GDY2JCYYP2815TUNZRCEEN2TBFAOZFHCwNUzUGf2Qtn5Vk4wQFmucp7osBA/byKM/dMcVVBz04Ec9AYlYBoY1dBTTqVSseiq/gC89e1pauudt8vNlP90vY2775qaaipn4MFtXUwJ5LbWf2rMS6M2501JOQMhujcJoByooYSBv+sDqAobE3dtPU+0yZEVyWvqdGxOywHgnol9m70+c1QvokN8KSzX8vHh1gLl9jt3sYrMvDLUKpg6tL0BlPK+A9nFDpuZczRzCxcbeuC1ZEwfJZH8B6lILkS3JgGUg9TW6zlf2qgGlF4HqFwXQAXZ+AfP1vNEmxIiHZdI/t8jeVysqiMuzJ+pxmTsxny81Nx3RT8A/rH7FAaD4/NvTMUzxyeEE26cXbNXv8hA+kcGUqczsPuE59Uh09br+KmgfQnkJlJQUwgBEkA5zPlL1RgM4OetpmeQL9z7MTxbApdd75oB9L0CQnpBa2UYQ+KU84RDmJsKdzAXxmAwsHHvaQDmXt7HaumAu5P7EOij4cf8cqcEJx1dvjPx5N14Wfnl1OsNhAV4Exfm365rmAKoH/PLqa6VnEIhuisJoBzEnEDeI6AhMVWtBrV9ibjtptbADFPRTitB1IzVrhtPN+CoJbz0s5c4mluKj5eau8bHWz0v1N+bOycor7+x+1SH7tnUxcpa0k4rTYrbah7clqnmquSF6Dxsp5op/2lEOxLITWJD/YgO8UWnN5iXA4UQ3U+7Aqh169aRkJCAn58fycnJpKWlWT138uTJqFSqZo+bbrqpxfMffvhhVCoVa9eutTiekJDQ7BqrV69uz/CdolkPPHdIvAXufBtCWii2OGi68rpwmAQH1YJ6e5+ys27myF5EBLW+rf7+K/uhVsHuE0Uczyvr0H0b+/JHJdgZEhNs3mHYXuMTehDs50VJZa3HLXNlnFcCHnsqkDelUqkaLeNJHpQQ3ZXdAdSWLVtYtmwZK1as4NChQ4waNYrp06dTWNjydP3WrVvJy8szPzIyMtBoNNxxxx3Nzv3www/Zv38/vXr1avFazz//vMW1Hn30UXuH7zQWJQzqapQ2Lv+aD/Va1w4k8RZ4IgPu/S/MToFpK5Xjp/dAtfyydyRTAFVYrqWqtr5d17hQruWTI3kA3HtF8+TxpuLDA7hhhBIg/2N3drvu2ZKG4pkdm30C8NaomWyqSu5hzYUzOphAbmKuSO5hAaIQwnXsDqDWrFnDokWLWLBgAYmJiaxfv56AgADefPPNFs8PDw8nJibG/NixYwcBAQHNAqjc3FweffRRNm3ahLe3d4vXCg4OtrhWYGCg1XFqtVrKysrMj/Lycnu/VLtYViG/pLRxOb4NNO1Lxu0QtQb6XQUjfgFXPg5Rw6C2Ar5v+b+RaJ/QAG96BCg/q6eL2reMt+W7HGp1ekbHhzGyd5hN73nQWNLg48O5FJTVtOu+jdXU6fjmJyWnqqP5TyamRHhPKmdQp9PzY57ye8DeFi5NjTFVJJeWLkJ0W3YFULW1tRw8eJBp06Y1XECtZtq0aezbt8+ma6SkpDBnzhyL4Eev13PPPffw5JNPMmzYMKvvXb16NREREYwZM4a//OUv1Ndb/1f/qlWrCA0NNT8SExNtGl97mUoYKDWgjEsrviHQzjwLh1Gp4MrHlM+PfgBO2L3VnZmbCrdjGa9ep+ed/UrpAltmn0xGxYeRlBBOnc7AW8bk847Yc6KI6jodcWH+DOvVsZkZk8mDe6JWKYnW5y46tl9ge50oqKBWpyfYz6vDS+0j4kJRqyCvtMYhQawQovOxK4AqKipCp9MRHW35r9To6Gjy8/PbfH9aWhoZGRk88MADFsdfeOEFvLy8eOyxx6y+97HHHmPz5s189dVXPPTQQ/zpT3/iqaeesnr+8uXLKS0tNT8yMzPbHF9HNCzhuaEGVFuGz4ZbXoYHvnB/QNfFdKSp8I7MAvLLaogI9OHGEfY1CV50tTILtWn/GSq17Vs+NDEt312XGN3uxOqmwgJ8GN9XqUruKbNQpuW7Yb1COvx1Bvp6MSg6GJBZKCG6K5fuwktJSWHEiBEkJSWZjx08eJAXX3yRt956q9VfasuWLWPy5MmMHDmShx9+mL/97W+8/PLLaLUt5xj5+voSEhJifgQHBzv86zEpra6jtFopGhjfw42NhK3ReMPY+eDjxgT3LqojM1Cm5PFfJvXB18u+3ZFTh0TRPzKQspp6/vX9WbvvbaLTG/jCWG7AUct3JlOGelY5A1MCeUeX70xMBTUlD0qI7smuACoyMhKNRkNBgWViaEFBATExrSefVlZWsnnzZhYuXGhxfPfu3RQWFtKnTx+8vLzw8vLizJkz/PrXvyYhIcHq9ZKTk6mvr+f06dP2fAlOYcp/igj0IdDXS8mBAs8JoBrT66HigrtH0WUkRJpmoOwLoH4qKGffqWI0ahV3J/ex+75qtYr7JymFNVP2ZFOva197l4NnLlJSWav0sesX3q5rWDPNGEDtO1nc4VkyRzjaqAeeI5gaC0tFciG6J7sCKB8fH8aNG8fOnTvNx/R6PTt37mTixImtvvf9999Hq9Uyb948i+P33HMPR44cIT093fzo1asXTz75JJ9//rnV66Wnp6NWq4mKal612dUsEsjB82agTHIPwquXw5Z5bZ8rbNJQTNO+Jby3950G4Lqh0fRqZ0HH2WN7Ex7ow7mL1Xx+rH273bYfU5bvpg6JwttKAc/2GtAziD7hAdTq9Oz52b1Vyet1enPZB0cFUKONieRHc0s9rt6VEML57P6NuWzZMt544w02btzI8ePHWbx4MZWVlSxYsACA+fPns3z58mbvS0lJYdasWURERFgcj4iIYPjw4RYPb29vYmJiGDx4MAD79u1j7dq1HD58mFOnTrFp0yaWLl3KvHnz6NGjR3u+bodqVgOqvgaljUuY28bUouBecDEbzu6Hs9ZrdwnbmQKo/LIam6tSl9XUsfVQLgDz7Ugeb8rfR8O8y5X3v96O9i4Gg4HtxvYt1w9zfIsflUplbi7s7nIGp4oqqanTE+ijoV+E9d279hjYM4hgXy+qahvawwghug+7A6i77rqLv/71rzz77LOMHj2a9PR0UlNTzYnlOTk55OXlWbwnKyuLPXv2NFu+s5Wvry+bN2/mmmuuYdiwYfzxj39k6dKlvP766+26nqNZJJADTHxEaeNy8xo3jqoFIbEw8k7l829fdO9YuoiwAG9C/LyAhp+Dtnxw8BxVtTouiwpiYv+Itt/QivkT++Ljpebw2Ut8f8a+paSsgnJySqrw9VJz9aCeHRqHNVOHKL8XvvzxAno3ztIcPafMCif2CkGtdkyivFqtYmS8MpslieRCdD9e7XnTkiVLWLJkSYuv7dq1q9mxwYMH2/Wv46Z5TWPHjmX//v32DNFldHqD+ZdznU6PTm9Ao1YZ27i0XlXaLa54DH54B378BIp+hsiB7h5Rp6ZSqUiIDOTIuVJOF1cyOKb1zQp6vYF/GpPH51+R0OHdYJFBvsweG8d7aWd5/ZtTTEiwPY/J1PvuqssiCfBp16+CNiX1CyfI14uiCi1Hc0sZZcwbcjVTArmjlu9MRseH8e3PxaSfvdiuXDYhROclvfA6IDUjj0kvfMlhYwD1+jfZTHrhS1Iz8tp4pxv1HAyDbgAMsO9ld4+mS7BnJ96en4s4VVRJsK8Xt4+Jc8j9F05SShp8cbyAUxcqbH7fDtPyXQd737XGx0vN1YMiAfcu45krkHeghUtLxkhFciG6LQmg2ik1I4/F7xwir9SyiF5+aQ0nNz9N/ht3Qo5nzpqZC2umvwcVnrHFvDPrZ6wFlW1DIrkpeXz2uN7Kjk0HGBgVxNQhURgMyo48W5y/VM3R3FJUqoZyA84yxbiMt9NN9aD0egPHzhubCPd28AyUMZH8RGEF5TV1Dr22EMKzSQDVDjq9gZXbMmlpUdIATFQfIyb3c3SeWi6gz0SIGw86LWR84O7RdHq2zkCdLakyBxH3TGx/8nhLTIU1/33wHMUVbfdfNM0+je/bg8g2Ghh31LWDe6JSwbHzZeSVVjv1Xi3JLq6kqlaHn7ea/pGOSSA3iQzypXcPfwwGOGKciRZCdA8SQLVDWnZJs5mnxkJQZiJ+vOih316VCq7/PdzzH0h+2N2j6fRMtaDOtFGN/J39ZzAYlJyjAT2DHDqG5H7hjIgLRVvf0B6mNebmwU5cvjOJCPI110xyR1Vy0/Ld0NgQvBxcqgGkoKYQ3ZWH/oX3bIXlrfe+ClEpf0gL6zwwidyk7xUw4Fpp7eIAphmo86XV1NS1XMqgpk7HFmPF8HsnJjh8DCqVyjwL9fa+01bHAVBaVcf+UyWA0r7FFaYONS7juaEquSmAclQF8qZGS0FNIbolCaDaISrYr9XXQ1CWckJ7OGdruMNpy0Hn/krRnVVEoA9Bvl4YDFhtnPvx4fNcqqqjdw9/rh3inJyjG4fHEBfmT3FlLR/+kGv1vC+zCtDpDQyODibBwUta1pjqQX37c5HN9bIcJSPXWEDTwQnkJqYAKv3sJbtrcQkhOi8JoNohqV84saF+tDR340stfiolmXTUwE6wrXnPWlgzDI596O6RdFpKKQPrieQGg4GNe08DMO/yvkqZCyfw0qhZcGUCAP/Yfcpq3SVT+QJnFM+0ZnB0MHFh/mjr9ew96bqq5AaDwVzCYFhciFPuMaxXCN4aFUUVtZy76PocLyGEe0gA1Q4atYoVMxMBmgVRITT8AtX4O+cXtkPpakFbCntfBPnXc7u1lkh+KOcSx86X4eul5q7x8U4dx10T4gn29eLkhUq+ymq+XFZTp+Prn5TNDa7IfzJpXJX8Cxcu4+WUVFFeU4+PRs2gaOc0FPfz1pAYq/y//oOT8qB0egP7ThbzUXou+04WS+sYITyABFDtNGN4LK/NG0tMqOVyXkKIHgMq8A0BtcZNo7PDhAfAOwDyj8KpXe4eTaeVEGG9qbCpdMEto3rRI9DHqeMI9vM2F3R8Y/epZq/vPVlEVa2O2FA/hjtpRsaaKcalyy9/LHDZUpepgfCQ2GCH9/przLyM54SK5KZ6c798Yz+Pb07nl2/s9/x6c0J0A84pP9xNzBgey3WJMaRll1BYXkNUsB9J/cJRcTfUtV1U0SMEhMOYeyDt/5T2LgOudfeIOqWGGSjLJbzC8ho+Par8obv3igSXjOW+KxNI2ZPN/lMlHD1XalH7yLR8d11idIeroNvr8v4RBPhoKCjTcux8mcOrgrfEnP/k5HuN6dODjfvOkH7WsYnkpnpzTcPN/NIaFr9ziNfmjWXG8FiH3lMIYRuZgeogjVrFxAER3Do6jokDIhrauPg6Z7nAKSb+ClRqOPUV5B1x92g6JVNT4aYzUJvTzlKnMzC2T5hLAgaA2FB/Zo7qBVjOQun0Br447vzq49b4eWuYNNBUldw1y3jHzjunAnlTphmojPNl1NbrHXLNturNAazclinLeUK4iQRQAnokwLDblM/3SnuX9jAlkederDb/Aa3T6dl0QOl756rZJ5MHruoHwCdH88i9pOTl/ZBzkaKKWoL9vEjub3vPPEeaZipn8KPz27oYDAbzEp6zlyv7RgTQI8Cb2no9x/PKHHLNturNGYC80hrSsksccj8hhH0kgHK0n7bDv+ZD2hvuHol9rjC2d8n8CKrkF7K9egb5EuCjQW+As8ZSBtuPFVBQpiUyyJcbXLzMMqxXKFcOjECnN7DB2N5lu7H6+NQhUU7NB2rN5CFKaY8j50opLGu9nlpH5V6q5lJVHV5qVZtNnjtKpVJZlDNwhLbqzdl7nhDCsSSAcrTCTCUIyT3o7pHYp9domLEaHjmg5EUJu6hUqmY78TYak8fvTorHx8v1/6s9cJVSWPO9tBx2Hi9g66FzQMMskDtEBfsxypiT1dIuQUcyFdAcFB2Mr5fzN3SMNjYWdlRBzbbqzdl7nhDCsSSAcrQaYz8sP9fkuzjU5YshvJ+7R9FpmXfiFVVxPK+MtOwSNGoVdyc7tu+drSYP6klsiB+VtToWbvyeoopaAH7/SaZbd3CZqpI7u5xBQwK5a3YbmhoLO2oGKiLIp9VGASogNlTZuCKEcD0JoBzNFED5doIaUK3RVrh7BJ1O4xmot/cpuU8zhsU0K3XhKp8fyyevhWWywjIti9855LYgylTOYM+JolZbznSUqYCms1q4NDW6dxgAp4uruFhZ26FrFZbXcP9b35lLs1mLo1bMTHRaYVYhROskgHI0rTGBtDPOQAFUFsGWefDKeKiT3Ap79I3wB2BX1gU+OKgsl90z0T2zT6YdXC1x9w6uYb1CiA72pbpOx9+/+MkphSENBoN5CW+YiwKo0ABv+vdUguiOzEJVaOtZsOE7zl2sJiEigD//YmSzIDzI10tKGAjhZhJAOVpnXsIDZdy5P0B5HhzZ4u7RdBqpGXn85fOfADhTUkWtTo+XWtXhmYj28uQdXJ8fy6dcq/Re/L+vTzmlMGRBmZaiilo0apW5SrgrmBsLtzOAqq3Xs/idgxw7X0ZEoA8b70/izvHx7Hl6Cu8tupz5xoA82FfDdW4oRSGEaCABlKN19gBK463UhQKlpIHeMTVtujJTscOSJsFSvd7Arza5Z6nMU3dwmb5XVU0aCpsKQzrqe2WafRrYMwg/b9d1BBjTgZ14BoOBpz84wu4TRQT4aNiwYIJ5WdhUb+63Nw4lLMCbvDIt35y44MCRCyHsJQGUo9V08iU8gLHzwTcUik/AT5+5ezQerbVihybuWCrzxB1criwMeTTXuQ2ErRnTR9mJl55z0WozZ2v+/HkWH/6Qi0at4tW5YxlpzKlqzM9bw21j4gDYnJbT4fEKIdpPAihHW7wXnsmBPhPdPZL28w2GCfcrn3/7knvH4uE8daksqV84saF+VpOP3bGDy5Xfq2MuTiA3GRwTjK+XmrKaerJb6Itozca9p3lt10kAVt8+gsmDo6ye+8skpdfhzuOFUgNKCDeSAMrR1Gpl9snLuU1jnS75YdD4wNn9cOB1OPpvyN4N+g7smtLrlGs44loewlOXyjRqFStmJgLNd3CZnrt6B5crv1cNFchdG0B5a9TmoM3WxsKfHc3juW3HAPjN9YO4Y3x8q+cPig5mbJ8w6vUG/m3crCCEcD0JoETLgmMaZtE+exI+WAgbb4a1wyHzY/uvl/mx8t6NN3f8Wh7EE5fKTGYMj+W1eWOb7eCKCfVzyw4uV32vCstrKCjTolLh0gRykzHGelA/2NBYOC27hMe3pGMwwNzkPjxy7UCb7jHHOAu15buzdi8VCtEdfPPNN8ycOZNevXqhUqn4z3/+Y/G6wWDg2WefJTY2Fn9/f6ZNm8aJEyfsuocEUI5UfUlp4/LxY5gLuHRWmR9D9tfNj5flKV+jPYFP5sfKe8rOd/xaHsYTl8oamzE81ryD68U5o3lv0eXseXqKW7a/t/W9Agj19+7w9+rYeSUPsX9kIIG+Xh26VnuYKpK3lUh+oqCcBzZ+R229nusSo3n+1uGoWquc2cjNI2MJ9vXiTHEV+08Vd3TIQnQ5lZWVjBo1inXr1rX4+p///Gdeeukl1q9fz4EDBwgMDGT69OnU1Ng+Ay4BlCNVFSttXDI+oNUSwp5Or4PUp628aAwMP3sKKi4offOqSkBX33BKXXXD8YoLygxWa6nDqc902uU8T1wqa8q0g+vW0XFMHBDhtrG09r0yKauuY+fxjjUazjjnnuU7E1NF8h/zyqmubfnnOr+0hnvfTKOspp5xfXvw8i/H2PXfJcDHi1tG9wLgve/OdnjMQnQ1N9xwA3/4wx+47bbbmr1mMBhYu3Ytv/vd77j11lsZOXIkb7/9NufPn282U9UaCaAcqbMX0TQ5s7f5bJEFg1In6q8D4c/9lEdho6KN+15pOP7XgVCe3/q1ynKVe3ZSnrZU5smsfa9iQ/24ckAEBuDR937g+9PtTyR3dQXypnqF+hEV7Eu93mAeS2NlNXXctyGN86U19O8ZyD/mj29XqQVTMvnnGfnNSmgI0RWVl5dTVlZmfmi12nZdJzs7m/z8fKZNm2Y+FhoaSnJyMvv27bP5Oq6f3+7KOnsNKJOKjs0AdJp7OtCM4bFclxhDWnYJheU1RAUry3bSZqM5a98rg8HAQ/88yM4fC1m48Xv+/fBELosOtvv6ph54w3q55/9DlUrF6PgwtmcWkJ5ziQkJDUuS2nodD779PT/ml9Mz2JeNC5LoEdi+DSfD40IZHhdCRm4ZWw+dMzePFqKrSkxMtHi+YsUKnnvuObuvk5+v/KM+OtqysXp0dLT5NVtIAOVIXSWACopu+xyA+R9DwiTlc1WjycxJv4ZJy5TPT++Bt29x3D09mGmpTLSt5e+VilfuHsvd/9jPDzmXuPfNNLb+6kq7egmWVNaSe6kacH0NqMZG9zEGUI3yoPR6A7/+12H2nyohyNeLtxZMID48oEP3mTOhD7/LzeC9tBwWTupncw6VEJ1RZmYmcXFx5ue+vr5uHI0s4TlWVwmg+l4BIb2wnqmigpA4JXhSa5RH41/canXD8YRJtl2r7xWO/RpEp+TvoyHl3gn07xnIeWOeUGl1nc3vN9V/SogIIMTP21nDbNMYYyL5gVPFfJSey76Txfzhk0z+eyQPb42K/7tnnENmyG4d3Qt/bw0nL1Ty/Zm2d/0J0ZkFBwcTEhJifrQ3gIqJUdogFRRYrnwUFBSYX7OFBFCO1FUCKLUGZrxgfGIlNXrGauW8Dl0L+64luoXwQB82LkgiKtiXrIJyFr39PTV1tm0yOOriBsLWFJQpO3mKKmt5fHM6v3xjP29+exqAv/xiFFcOjHTIfYL9vLl5pJJj955UJhfCJv369SMmJoadO3eaj5WVlXHgwAEmTrS9CHa7Aqh169aRkJCAn58fycnJpKWlWT138uTJqFSqZo+bbrqpxfMffvhhVCoVa9eutTheUlLC3LlzCQkJISwsjIULF1JRUdGe4TtPVwmgABJvgTvfhpAmCdAhvZTjiTYsy7V1LY2P/dcS3UJ8eABvLUgi2NeLtOwSlm5Jt6nFyzFj/tNwN+U/gdLvb+mWdKuv+3k79t+tv0xWksk/PZpn12ydEF1ZRUUF6enppKenA0rieHp6Ojk5OahUKp544gn+8Ic/8PHHH3P06FHmz59Pr169mDVrls33sDsHasuWLSxbtoz169eTnJzM2rVrmT59OllZWURFNW8/sHXrVmprG3aIFBcXM2rUKO64445m53744Yfs37+fXr16NXtt7ty55OXlsWPHDurq6liwYAEPPvgg7777rr1fgvNM/i1c8ai7R+E4ibfAkJuUHXIVBUqeUt8r2jdb1PhaF36ET58EXS1EJbb9XtEtJfYK4f/mj+O+N7/js4x8Vm47xspbhrWa5+PuHXht9UZUofT7uy4xxmEbDMbEhzE4OpisgnI+Ss9l/sQEh1xXiM7s+++/59prrzU/X7ZMycu99957eeutt3jqqaeorKzkwQcf5NKlS0yaNInU1FT8/GzPubT7n0Jr1qxh0aJFLFiwgMTERNavX09AQABvvvlmi+eHh4cTExNjfuzYsYOAgIBmAVRubi6PPvoomzZtwtvbMnfh+PHjpKam8o9//IPk5GQmTZrEyy+/zObNmzl/vrXt9i5mauPSFWagTNQa6HcVjPiF8rEjS22mayUtgsuuV46lv+OYcYou6YoBkay5axQqFby97wyvGvvFtaS0uo4zxVUADOvlngRyd/RGVKlUzElS2r+8l3YWQ2cv4iuEA0yePBmDwdDs8dZbbwHK/zfPP/88+fn51NTU8MUXXzBo0CC77mFXAFVbW8vBgwctaieo1WqmTZtmc+2ElJQU5syZQ2BgoPmYXq/nnnvu4cknn2TYsGHN3rNv3z7CwsIYP368+di0adNQq9UcOHCgxftotVqLehHl5eW2fpnCFcbMVT6mv2dZhFOIJm4e2Ytnb1ZmKv/yeRbvf99y4UhTAnnvHv7tLg3QUe7qjXjbmDh8vNQczyvjyLnmtaeEEI5nVwBVVFSETqdrd+2EtLQ0MjIyeOCBByyOv/DCC3h5efHYY4+1+L78/Pxmy4NeXl6Eh4dbve+qVasIDQ01P5rWj3CKL/8IHz8K+RnOv1dnN+gGCIiAinw4ubPt80W3tuDKfjx8zQAAntl6lK9+LGx2ToapgbAb85/c1RsxLMCHG4cru4c2fyfJ5EK4gkt34aWkpDBixAiSkpLMxw4ePMiLL77IW2+95dAaJsuXL6e0tNT8yMzMbPtNHfXjJ3Dobai84Px7dXZePjDyLuXzH/7p3rGITuHpGYO5fWwcOr2BX206xA85ltv2TQU0h7ux/pM7eyOaGgx/nH6eSq3M6grhbHYFUJGRkWg0mnbVTqisrGTz5s0sXLjQ4vju3bspLCykT58+eHl54eXlxZkzZ/j1r39NQkICoNRsKCy0/BdnfX09JSUlVu/r6+trUS8iONj+isZ260q78FxhzDzoeyUMlV14om0qlYoXZo/k6kE9qa7Tcf9b33HqQsNOXFMCubt64IF7eyMm9wunf2QglbU6th32oNxQIboouwIoHx8fxo0bZ1E7Qa/Xs3PnzjZrJ7z//vtotVrmzZtncfyee+7hyJEj5u2G6enp9OrViyeffJLPP/8cgIkTJ3Lp0iUOHjxoft+XX36JXq8nOTnZni/BuSSAsk/0MFjwKYy8090jEZ2Et0bNa3PHMrJ3KBer6pj/Zhp5pdV8+WMhpy5UAjAkxn0zUOC+3ogqlYq7JhiTyaXBsBBOZ3cZg2XLlnHvvfcyfvx4kpKSWLt2LZWVlSxYsACA+fPnExcXx6pVqyzel5KSwqxZs4iIsGzfEBER0eyYt7c3MTExDB48GIChQ4cyY8YMFi1axPr166mrq2PJkiXMmTOnxZIHbqGrh1pjorpfmFuHIkRXFujrxZv3TWD2a3s5U1zFVS98RX2jGlG3vfotK2YmurWJs7t6I84e15u/bs/i8NlLHM8rY2ise4NJIboyu3Og7rrrLv7617/y7LPPMnr0aNLT00lNTTUnlufk5JCXl2fxnqysLPbs2dNs+c4emzZtYsiQIUydOpUbb7yRSZMm8frrr7f7eg6nLWv43E9+adml4gLsfUWS74XNIoN8eWBSPwCL4Akgv7SGxe8cIjUjr6W3uoyp39+to+OYOCDCJY2lI4N8uS5R+V28WSqTC+FUKkM3KRpy7tw54uPjOXv2LL1793b8DUqy4aXR4B0A/+PeX9ydzgeL4Oi/YMIDcNPf3D0a0Qno9AYmvfCl1ZpLKpQlsz1PT3FJ4OJJvvnpAvPfTCPEz4u0/5mGn7e0SRKdm9P/freT9MJzFNMMlOQ/2W/03crHo+9DXbV7xyI6BXcUrOwsJg2MpHcPf8pq6vn0qPxjTghnkQDKUaJHwDM5sOgrd4+k8+l3DYTGK0n4P37i7tGITsBdBSs7A7VaxV3jlWTyzWmSTC6Es0gA5SimNi5NG+aKtqnVMNpYmVxqQgkbuKtgZWdxx/h41CpIO13Cz4Ue1nRdiC5CAijhGUb/Uvl46mu4JMmvonXuLFjZGcSE+jFliNK9QZLJhXAOCaAc5afP4ePH4Oi/3T2SzqlHAvS7GjAo/fGEaIU7C1Z2Fr80Vib/4NA5tPU6N49GiK5HAihHyT0IhzZCjm1NlUULxtwDGp+GgqRCtMJdBSs7i2sG9SQmxI+LVXVsP1bQ9huEEHaxu5CmsEKqkHfc0Ftg4DQI6J7LLsJ+7ipY2Rl4adTcOb43L335M5u/y2HmKA8pOixEFyEBlKOYAihfKaLZbt5+ykMIO5gKVorm7pwQz8tf/cy3PxdzpriSvhGB7h6SEF2GLOE5So3UgXKoCz81fE+FEO3Su0cAV13WE4At0h9PCIeSAMpRZAnPcf7zCKyboBTWFEJ0yC+NDYbfP3iOOp3ezaMRouuQAMpRJIBynKihyscf3nHvOIToAqYOjSYyyIcL5Vq+/LHQ3cMRosuQAMpRzAFUmFuH0SWMvAvUXnD+EBRkuns0QnRqPl5qZo9T+odJTSghHEcCKEdZvAeeOArRw9w9ks4vqCcMmqF8nr7JvWMRoguYM0GpCfX1Txc4f0n6TQrhCBJAOYpfKIT1kV1kjjLmHuXj4c2gq3PvWIRn0+sge7dSxDZ7t/JcWOgXGcjl/cPRG+Bf30syuRCOIGUMhGcaOA2CoqGiQKnyPvRmd49IeKLMjyH1aSg733AspBfMeAESb3HfuDzQL5P6sP9UCf/cd5q+EYHEhEjNLCE6QmagHKHigtLGZefv3T2SrkPjBaPmKJ9nfODesQjPlPkx/Gu+ZfAEUJanHM/82D3j8lAqQKWC4so6lm5J55dv7GfSC1+SmpHn7qEJ0SlJAOUI5XlKG5cf/unukXQt4xbALzbArNfcPRLhafQ6ZeYJQwsvGo+lPiPLeUapGXk8vjkdQ5NvV35pDYvfOdSuIEqnN7DvZDEfpeey72QxOn1L/y2E6LpkCc8RtFJE0ynC+ykPIZo6s7f5zJMFA5TlKuf1u8plw/JEOr2BldsyrYaaKmDltkyuS4yxeTkvNSOPldsyySutMR+LDfVjxczEbt+DUHQfEkA5gtSAcj7TP51Vkq8hUHLjHHleF5aWXWIR6DRlAPJKa5jy110MjQ2hT0QAfcIbHnE9/PHWNCxWpGbksfidQ80CMtNsljRyFt2FBFCOIAGUc+17Fb57A2athz7J7h6N8ARB0Y49rwsrLLcePDV2pqSKMyVVzY6rVdArzJ8+4QHEh/vz6ZF8h85mCdFZSQDlCBJAOVdBBpScUnLMJIASAH2vUHbbWV3GUymv973CpcPyRFHBtpVWeWr6IPx9vMgpqeJsSRVniqvIKalCW6/n3MVqzl2shpOtX8M0m5WWXSINnkWXJwGUI0gA5Vxj5ikFNY99CDe8AD7SUb7bU2uUUgX/uqeFF40zHzNWK+d1c0n9wokN9SO/tKbFmSMVEBPqx0PXDGw2a2QwGLhQruVMSRU5xVXsyCwg9Vh+m/d8/r/HmDmqF8n9whkRF4aPV+v7lXR6A2nZJRSW1xAVLOUVROcgAZQjSADlXH0mQnh/ZRYq8yMYfbe7RyQ8wdCZ0P9aOPWV5fHASPANhp6D3TMuD6NRq1gxM5HF7xxCheW+RVOIsmJmYosBi0qlIirEj6gQPyYkhNMrzN+mAOp4XjnH87IA8PVSM6ZPGEn9IkjuF86YPmEE+DT86ZGEdNFZqQyGphtbu6Zz584RHx/P2bNn6d27t2Mvrq2AqmLwDlDakAjH++av8OXvoe+VsOBTd49GeJLyQijKUhLGg6Jh/2uQ9YlSjHWe1BAzcUSgotMbmPTCl63OZkUE+fDQNQM4ePoiaadLKKmstTjHS61iRO9QkhKUWabXdp1sdi1TKCcJ6QKc/Pe7AySAEp1DaS6sHQ4GPTx6CCIGuHtEwlMVn4R1yaCvg7vfh0HXu3tEHsMRS2WmXXjQ8mxW46DHYDBw8kIFB7JL+C67hANt7AhszLS0uOfpKbKc18156t9vKaQpOofQOBgwVflcGgyLrM8gP6Pl1yIGwOWLlc8//630UmxEo1YxcUAEt46OY+KAiHYFJjOGx/LavLHEhFomp8eE+jWbMVKpVAyMCmZucl/WzhnD3memsPupa/nbHaO4dlDrs/WNE9KF8ESSA+UIX/8ZtOUwfoGSqyOcY/z9ENgTBt3g7pEId6qrgY8fhcoLcM+HMGBK83OufhIOvwfFJyDtDZj4K9ePswubMTyW6xJj7J7NUqlUxIcHEB8egJdGxVc/XWjzXraWYRDC1SSAcoQf3oFLZ5SkVgmgnGfIjcpDdG+H31OCp5DekGClyrhfCEz5HWx7HL5eDSPvgkDZVu9Iptms9rK1vIKt5wnharKE5wiyC08I19DrYd8ryucTHwGNt/Vzx9wD0SOU/z9/eNs14xM2M5VXaGsR8aPDudTUSU9D4XkkgOqo+rqGAOrCj9K81BXyjsCnT8LxT+DovyF7t2d83/U6ZSyeNKauJutTKP5Z+cfK2Pmtn6vWwE1/hdtehysed834hM1M5RWAZkFU4+eb085y6yvfcqKg3GVjE8IWsguvIzI/hs+egvJGncxDeikF/hJvccw9RHNrRypLpo25+/ue+TGkPm1ZGdvdY+qK/nEdnEuDq34NU59192iEA7RWXiHQ14ulWw5TVKHFz1vN87cM547xvVFJT8xupUvtwlu3bh0JCQn4+fmRnJxMWlqa1XMnT56MSqVq9rjpppvM5zz33HMMGTKEwMBAevTowbRp0zhw4IDFdRISEppdY/Xq1e0ZvmNkfgz/mm8ZPAGU5SnHMz92z7i6usyPmwdP4N7vu+lnoWlbEflZcKyc/UrwpPGBpIfsf7+2HAqPO35cokNmDI9lz9NTeG/R5bw4ZzTvLbqcPU9PYcbwWK66rCefPX4VV10WSU2dnqc+OMLjm9Mpr5GdlcL97A6gtmzZwrJly1ixYgWHDh1i1KhRTJ8+ncLCwhbP37p1K3l5eeZHRkYGGo2GO+64w3zOoEGDeOWVVzh69Ch79uwhISGB66+/ngsXLHdoPP/88xbXevTRR+0dvmPodcpsg9WWmkDqM7KE42jm73tL3PR9l58F16koVApljpoDwXY2Cc49BC+Pg81zob627fOFS7VWXqFnsC8bFyTx1IzBaNQqPj58nptf3sORc5fcN2AhaEcAtWbNGhYtWsSCBQtITExk/fr1BAQE8Oabb7Z4fnh4ODExMebHjh07CAgIsAig7r77bqZNm0b//v0ZNmwYa9asoaysjCNHjlhcKzg42OJagYFu6ol2Zm8rTUwBDFCWq5wnHMcTv++eOKauKvEWeOIoTFtp/3sjLwODAUpOQtr/OX5swqnUahW/mjyQfz00kbgwf84UVzH7tb2k7Mmmm2ShCA9kVwBVW1vLwYMHmTZtWsMF1GqmTZvGvn37bLpGSkoKc+bMsRr81NbW8vrrrxMaGsqoUaMsXlu9ejURERGMGTOGv/zlL9TX11u9j1arpayszPwoL3dgAmJFgWPPE7bxxO+7J46pK/PyhYBw+9/nG9yQM/X1n6GyyLHjEi4xrm8PPn3sKmYMi6FOZ+D3/83kgY3fW7SL0ekN7DtZzEfpuew7WYxOLwGWcA676kAVFRWh0+mIjracPo+OjubHH39s8/1paWlkZGSQkpLS7LX//ve/zJkzh6qqKmJjY9mxYweRkZHm1x977DHGjh1LeHg4e/fuZfny5eTl5bFmzZoW77Vq1SpWrmzHv1RtEWTj8oGt5wnb2Pr99PKHqpL2/aG1VVUJHNwANTYG5vteURrfSi0i+5UXwNn9MORmZWdde42eC2mvQ/4R+PIPMHOtw4YoXCc0wJvX5o3lnf1n+P0nx9n5YyE3vribF+eM5mJVrTQmFi5j1y688+fPExcXx969e5k4caL5+FNPPcXXX3/dLPG7qYceeoh9+/Y1W5oDqKysJC8vj6KiIt544w2+/PJLDhw4QFRUVIvXevPNN3nooYeoqKjA19e32etarRatVmt+npubS2JiomOy+PU6pS9bWR4t576olB1YTxzt2C98YcnW73virfD9BiVXJvlhiBriuDEUHlea1R7ZAvU1EJYA+tpWxmTUoz88ehDUxklfvb7hc9G6L1bCnjUwfDb8ouVUAZud2QsbbgCVGh7aDTHDHTNG4RaZ58tY8t4hTl2oRIXV3wqANCbuzLrELrzIyEg0Gg0FBZbLEQUFBcTExLT63srKSjZv3szChQtbfD0wMJCBAwdy+eWXk5KSgpeXV4szVSbJycnU19dz+vTpFl/39fUlJCTE/AgODm79i7OHWqNsTwesVjCZsVqCJ0ez6fu+CvKPQn21MkP0ajL88zb4absStDRlS+0mvR5++hzevhVevRwObVSCp9hRcO1yuP5PrYxJBTf9DWa92hAw1dXAugmw/X/hUk7LX6vUlFJoy+F74++BYbd3/Hp9r4DEWUpT6tRnlLwo0Wkl9gph25JJzB4bZ/WfL6bjK7dlynKecCi7lvB8fHwYN24cO3fuZNasWQDo9Xp27tzJkiVLWn3v+++/j1arZd68eTbdS6/XW8wgNZWeno5arbY6Q+V0ibfAnW9bqf2zWmr/OIst3/ehtygzDftfVQovnvxSeUQMhEnLYMxc5T221m767Cn47g3lc5VaWUq6/FfQ53Iw1aNRa2z/WTi+TSkGufclZWlv6ExIXtxwPakp1eDQ20qh2oiBMNhBbXyuex5+SlWWhOtrwNvfMdcVbhHo68UvxsXzwaFcq+c0bkzckfYzQjRmdy+8ZcuWce+99zJ+/HiSkpJYu3YtlZWVLFiwAID58+cTFxfHqlWrLN6XkpLCrFmziIiw/OGtrKzkj3/8I7fccguxsbEUFRWxbt06cnNzzTv19u3bx4EDB7j22msJDg5m3759LF26lHnz5tGjR4/2fu0dl3gLDLlJ+WNdUaD8Qu57hcw8OVtb33eVChKuVB4XTyvNZA+9rQQtBRnKOabaTU3/3Wqq3XTLSw2VroffDkf+BePmQ9KDENbH/jE1Nny2ktS8/1XI/hoyP1IesaMh/nLjLjEr47rz7e4TROnqYN+ryudXPOq4Jc8efeGxH5SgVHQJtjYclsbEwpHsDqDuuusuLly4wLPPPkt+fj6jR48mNTXVnFiek5ODuskvuqysLPbs2cP27dubXU+j0fDjjz+yceNGioqKiIiIYMKECezevZthw4YBynLc5s2bee6559BqtfTr14+lS5eybNmy9nzNjqXWQD8rDU2F89j6fe+RANP/CJOfgfT34LLrbKzdtFxJOlZroM9E+PVx8GmjbIatY1KrYfAM5VFwDA6sVwK0vHTl0SIDoFKWnYbc1D2C9IytUHYOAqNg5BzHXluCpy7F1obDr+06ycXKWmYMjyUmVJoUi46RVi6i+8neDRtvbvu8e7dBv6udPx6AymL4YgX88M+2z733v10/aDcYYP0kZcZwyv/C1b9xzn0u5cBXf1JqS9lbnFN4DJ3ewKQXviS/tKa1rRwWxvftwQ0jYrlheAy9wlpextXpDaRll1BYXkNUsB9J/cItinwK1/DUv992z0AJ0enZXLup5er6ThEYAf0n2xZAdYeaUlUlSs0n70CY0PLGE4f48GE48y2oveDWV5x3H+FUpsbEi9851Gw3ninc+cNtw6mu1fFZRj4Hz1zke+Pj9//NZHR8GDeNiGXG8BjiwwOA1nv0yW4+ATIDJbojm2egXDzTY+u4JixSikL6hTh/TO5kMCh9D3skOO8eOQfgzesBFTz0tbKzUnRatgY9+aU1fJaRx2dH8/nuTInFZsyRvUPpHxnEf9KbJ6VLSQT38NS/3xJAie7HU+t4tTmuRnyCYc47yqyV6Jh/3w8ZH0DfK+G+Txp2VopOyd5lt8KyGj4/ls+nR/M5kF1MW5UOVEBMqB97np4iy3ku4ql/v6WSn+h+PLWOV5vjUsG4BRA5CAw6y9mSqpKuU9Pox0+g+pLr7jdtpVK9/sy3ym5I0am11pi4JVEhftwzMYH3HrycA7+dxsJJCa2e37gkgujeJAdKdE+eWsfLlnHp9VB8AvwblfB4906oq4bLF8PwX4B3ox1Gep3jSm048lotuXgGttwD3gHw2CEIckGdt7B4uPIx+PoF2P47pcRE9UUpS9IN9Qz2ZWTvMJvOlZIIQgIo0X15ah2vtsalVkPPwQ3nXzqrlEOoq4KPHoEdK2D8/Ury9dk0xxXldEWBz/2vKrNrvce7JngyufJxOPA6lJ6FdxpVPO+uBUy7MVtLIth6nui6JAdKiK6g+qJSLDTtDSUIAFBplGCkGeOShj1FOa0VHm3PtaypKoG/D1MCwXv+AwOu7dj17JH5MfzrnhZecODXJzoFW0oixEoOlEt56t9vmYESoivw76HMolz+CPz4X6WC9zlrzb2NfxY++hXkHVba05hMeqKhYGjWZ5B7SOkbd6CF6ujmazmowOd3KUrwFDPCtcnx5sKqLemGBUy7udZKIpjcOCJWgichAZQQXYrGC4bNgoCItksiaMth918tjyU/1BBAndjR0Mi3VQYoy1WWHNtb9qGu2tjCBrjicdfuhDuz13JZshkHfH2iU5kxPJbX5o1tVhIhwEdDVa2Of+47w/WJ0ST3l7563ZkEUEJ0RbYW2xwwRWnUa+Ll2/B5wpXKjEvxz0oz5rYUnWh/gHH4Pai8AKHxSgDoSrZ+r3L2SQDVjcwYHst1iTEWJRHG9e3Bo+8d4vNjBSx6+3s+WHwFl0UHu3uowk0kgBKiKwqysS3JpGXWg4Lhs5VH9m7bAqjPnoLcg3D5w8oynD1Kzyk5WxMfAY23fe/tKFu/V1/9EU5sV3Y6Dr3F9eMULmcqidDYi3PGcPcb+zmUc4n7NnzH1l9dQXSIJJR3R1IHSoiuqO8Vxoa51pbCVBASp5zniGtpvEFfB+nvKD3s3rpZqeekbymJvQVTn4XHD8PY+bad70htfn0oZRXU3nDuO/jPI8ryp+iW/Lw1/OPeCfSPDCT3UjX3bfiO8po6dw9LuIEEUEJ0RY4sFmrLtWa/CQu/gGG3KzNJp3fD5rvh5bGQ/U3za+p1yszW0X8rH/U6pR6TKf/KlWwpYHrb/8GyTJi8XJklCwhvOGX3GriQ1fy6LX2N7eWoa3nimDrhtcIDfXhrQRKRQT4czyvjV5sOUafTt/8+olOSMgZCdGUt1m6Ka1+xUFuvVXpOKadw8C2ouQSPHoKIAcprep0yM+XselLt0Z7v1bnv4R9Tlc8HTFWW9wZMVXZCelr9LUfW8ZJrAXD47CXmvL6f6jods8f25q93jEQlrYAczlP/fksAJURX565K5LWVcHoPDJrecCzlejhrrbyCyv31luz9XhVkwq4/KUGhwTgDERQLFXktnOzG+luOrOMl17K41pc/FvDAxu/RG+CxKQNZdv1ghGN56t9vCaCEEK5RXgB/G9TKCW5q4uwIF08bZ902Qm0b+VH+4XDzWqWivInGxzLQPLMXKi7Af5+Aams914zfr9v/AdXF1u836AZ4cUTrpRpMYxpyk1IKA+D8D8psYmN6fRtjouWvb8CUhuXZgkwoOWnDtVRKNfob/mJ5rcb6Xqksp+p1sGYIVBRaH1dwLCw9pvxslWRDQUbL5+n1yoaIinzr42ryc/rugRx+++FRAFbdPoJfJvWxPg5hN0/9+y278IQQrlH0UxsndOJ6Sz0SYPofod/VSl/C1lSXwPtNkuUDIuCpUw3Pv/qTkkfWKuP3a8f/Qu731k+7d1sbda4ajem35xsCqLQ3IH1TG2No5VqNPX64IYA6/B7sfcmGCxmUmcCm12rsgS+VAOrM3taDJ4DyvIafrZ+/gE9/Y8MYrIyryc/p3cl9yCut5uUvf+Z3/8kgJsSPa4e4sBWRcAsJoIQQrmFrvSVbz/NEtu7OCx8AgZENz/1CLV+PGqoEPSUn275WQDjEJ1t/vdzG72f4AMuq9OH9ml+3ssi2MTX9+jSN6ouF9VGu295rNWYKyuz92QqKtv49s3Vc5ZYzVMuuG0TupWq2HsrlV5sOseWhy21uTCw6JwmghBCuYWu9JVvP80S2jn3mi63Pst34F2XXV1vV5AGueKz1a2W3NZPVaEze/g3Pr35SeTS9li1jau3rS1qkPBxxLRN7f7YSb7GeD2XruL76k1K6Y+RdoNagUqlYfftILpRr2X2iiPvf+o6ti6+kT0SAbWMTnY6UMRBCuIYja1N5KlfX37LlWp44pk59LaOLp2DPWotZOx8vNa/OHcvQ2BCKKmq5b0MaFytr276n6JQkgBJCuIYja1N5KlfX37LlWp44pk59LRXMeg2mroBrnmro21hbBdueIPhiJm8tmECvUD9OFVXywNvfU1mt5di3n/D9f1/n2LefoKuvb3scVujq67v8tToL2YUnhHAtR9am8lTuqL/VGcfUla71/Zvw36XK532v5PyQ+7jh82Aur93Pc95vE6tq2G1YQATnJ65gzPR77RrSD59vpNe+lUTTsOuyq12rJe39+71u3Tr+8pe/kJ+fz6hRo3j55ZdJSkrq8HhMJIASQrieI2tTeSp31d/qbGPqKtfKOwLfvgiZ/wG9MvtS6dWDgLqLGAB1o8ksvfGv7uErXrI5wPjh842M2vsYdOFrWdOev99btmxh/vz5rF+/nuTkZNauXcv7779PVlYWUVGO2SEpAZQQQgjhKKW58N0/MHz/JqqaS1ZP0xugWNUD7T2fovFqCMoMGj/0gT3NzzVl59DV1+P7zxuIMFyyCFIaX6tQFUH9/E/QaFrOzDFofNAHRqOrr8dv3Wh6GoqtXqtI1YPqR46g8VL2makr8lHpm/f709Xr8P3njUQYLrY6rp6/+8l8rfZoz9/v5ORkJkyYwCuvvKKMRa8nPj6eRx99lGeeeabdY2lMduEJIYQQjhIaB9NWcNxnBIlfLrB6mloFPbkI/5xocfwr3SgW1D1tfp7pu4AAlVZ5YiWvXa2CGIop3TiFUFVVi+cc1vfn1to/cLk6k80+xa1eK4qLzPn7/7FfnwjAdp8nGaTOtfq1tDWuYwc+Z9iVN1l/v43Ky8spKyszP/f19cXX17fZebW1tRw8eJDly5c3jEWtZtq0aezbt6/D4zBf02FXEkIIIQQAVWVFNp1Xa9BQY/A2P3Qqb3y91OaHFm9qDbYtQepRW1yr8aPeeN0Y9SWbrhWjvmQeQ52q5WvaOq7qi60EX3ZITEwkNDTU/Fi1alWL5xUVFaHT6YiOtixvER0dTX6+tQrz9pMZKCGEEMLB/HvE2XTeiev/aTE7Mw3Isjgjl2PffsKwHXe3ea3z179udaZnnPG6x77Vw45X2rzWohuuYO2VNxif3dDiObaOy9bvRVsyMzOJi2u4VkuzT64kM1BCCCGEgw1Jnk4BEeZk6qb0BsgngiHJ01s+oZtdyxbBwcGEhISYH9YCqMjISDQaDQUFlhXqCwoKiImJcchYQAIoIYQQwuE0Xl6cn7gCoFmAYXqeN3GFTcnV3eFajuTj48O4cePYuXNnw3j0enbu3MnEiRNbead9JIASQgghnGDM9Hs5fMVLXFBFWBwvVEXYvb2/O1zLkZYtW8Ybb7zBxo0bOX78OIsXL6ayspIFC6wn9ttLyhgIIYQQTqSrr+fHA59TfTEX/x5xDEme3u5Zme5wraba+/f7lVdeMRfSHD16NC+99BLJya003raXoR1eeeUVQ9++fQ2+vr6GpKQkw4EDB6yee8011xiAZo8bb7zRfM6KFSsMgwcPNgQEBBjCwsIMU6dONezfv9/iOsXFxYa7777bEBwcbAgNDTXcf//9hvLycpvHfPbsWQNgOHv2rP1fsBBCCCHcwlP/ftu9hLdlyxaWLVvGihUr+P/27jwoimsLA/g3gDOjyCIQcbAQguACohEBxaUslYghwSWJ4IaYUjQlVoykDBqMPDQuURNIDGpEIsaiQtwXRBS3UnBXMCqICyiYAhQFRU1kmfv+8DkvI2holMHB71fVVenbt7vPPRmZU909t8+ePYtu3brBx8cHt27dqrX/li1bUFhYqFkuXLgAQ0NDjBw5UtOnQ4cO+Omnn3D+/HmkpaXB3t4egwcPxu3btzV9xo4di4sXLyI1NRVJSUk4fPgwJk+eLDV8IiIiopcm+Rbey87uGR0djblz56KwsBDGxsa19rl//z7MzMywb98+DBo0CNnZ2XB2dsapU6fg7u4OAEhJSYGvry9u3rwJGxubfz0vb+ERERHpn9f1+1vSFains3t6e3v//wASZ/eMi4vDqFGjnls8VVRUYPXq1TAzM0O3bt0AAMeOHYO5ubmmeAIAb29vGBgY4MSJE7Ue5/Hjx7h//75mKS8vr+swiYiIiF5IUgH1srN7njx5EhcuXMCkSZNqbEtKSkLLli2hVCoRFRWF1NRUWFlZAQCKiopqvPzPyMgIFhYWzz3vokWLtGYsdXZ2ruswiYiIiF5Ip9MYxMXFwdXVFZ6enjW2DRgwAJmZmTh69CiGDBkCf3//5z5XVRezZ8/GvXv3NEtWVtbLhE5ERESkIamAepnZPR8+fIjExERMnDix1u3GxsZwdHREr169EBcXByMjI8TFxQEA2rRpU6OYqqqqwt27d597XoVCoTVjqYmJSV2HSURERPRCkgqol5ndc+PGjXj8+DHGjRtXp3Op1Wo8fvzkDdReXl4oKyvDmTNnNNsPHDgAtVr9aud0ICIiIqoDybNchYaGIigoCO7u7vD09ER0dLTW7J7jx49H27Zta7wlOS4uDsOHD4elpfZspQ8fPsSCBQswdOhQqFQqlJSUICYmBn/++admqoPOnTtjyJAhCA4OxqpVq1BZWYlp06Zh1KhRdfoFHhEREdGrJLmACggIwO3btzF37lzN7J4pKSmaB8vz8/NhYKB9YSsnJwdpaWnYu3dvjeMZGhri0qVLWLduHUpKSmBpaQkPDw8cOXIELi4umn4JCQmYNm0aBg0aBAMDA3z00Uf48ccfpYZPRERE9NLemFe55Ofnw87ODidPnoRKpWrscIiIiKgOCgsL4enpiRs3bqBdu3aNHY6Gbl+R3IiePvhe2y8AiYiI6PVWXFz8WhVQb8wVqKqqKmRkZMDa2rrGLUYpysvL4ezsjKysLP6yr4Ex17rDXOsOc61bzLfuNFSu1Wo1iouL0b17dxi9ohcUvwpvTAH1qjx9zcy9e/dgamra2OE0acy17jDXusNc6xbzrTtvWq51OpEmERERUVPAAoqIiIhIIhZQEikUCkREREChUDR2KE0ec607zLXuMNe6xXzrzpuWaz4DRURERCQRr0ARERERScQCioiIiEgiFlBEREREErGAIiIiIpKIBVQtYmJiYG9vD6VSiZ49e+LkyZMv7L9x40Z06tQJSqUSrq6uSE5O1lGk+k9KrmNjY9GvXz+0atUKrVq1gre397/+v6H/k/q5fioxMREymQzDhw9v2ACbEKm5LisrQ0hICFQqFRQKBTp06MC/I3UkNdfR0dHo2LEjmjdvDltbW8yYMQN///23jqLVX4cPH4afnx9sbGwgk8mwbdu2f93n0KFDcHNzg0KhgKOjI+Lj4xs8Tp0SpCUxMVHI5XLxyy+/iIsXL4rg4GBhbm4uiouLa+2fnp4uDA0NxZIlS0RWVpaYM2eOaNasmTh//ryOI9c/UnM9ZswYERMTIzIyMkR2draYMGGCMDMzEzdv3tRx5PpHaq6fysvLE23bthX9+vUTw4YN002wek5qrh8/fizc3d2Fr6+vSEtLE3l5eeLQoUMiMzNTx5HrH6m5TkhIEAqFQiQkJIi8vDyxZ88eoVKpxIwZM3Qcuf5JTk4W4eHhYsuWLQKA2Lp16wv75+bmihYtWojQ0FCRlZUlli9fLgwNDUVKSopuAtYBFlDP8PT0FCEhIZr16upqYWNjIxYtWlRrf39/f/H+++9rtfXs2VNMmTKlQeNsCqTm+llVVVXCxMRErFu3rqFCbDLqk+uqqirRu3dvsWbNGhEUFMQCqo6k5nrlypXCwcFBVFRU6CrEJkNqrkNCQsTAgQO12kJDQ0WfPn0aNM6mpi4F1JdffilcXFy02gICAoSPj08DRqZbvIX3DxUVFThz5gy8vb01bQYGBvD29saxY8dq3efYsWNa/QHAx8fnuf3pifrk+lmPHj1CZWUlLCwsGirMJqG+uZ43bx5at26NiRMn6iLMJqE+ud6xYwe8vLwQEhICa2trdOnSBQsXLkR1dbWuwtZL9cl17969cebMGc1tvtzcXCQnJ8PX11cnMb9J3oTvxtfntcavgZKSElRXV8Pa2lqr3draGpcuXap1n6Kiolr7FxUVNVicTUF9cv2ssLAw2NjY1PhHStrqk+u0tDTExcUhMzNTBxE2HfXJdW5uLg4cOICxY8ciOTkZV69exdSpU1FZWYmIiAhdhK2X6pPrMWPGoKSkBH379oUQAlVVVfj000/x1Vdf6SLkN8rzvhvv37+Pv/76C82bN2+kyF4dXoEivbR48WIkJiZi69atUCqVjR1Ok1JeXo7AwEDExsbCysqqscNp8tRqNVq3bo3Vq1ejR48eCAgIQHh4OFatWtXYoTU5hw4dwsKFC7FixQqcPXsWW7Zswa5duzB//vzGDo30EK9A/YOVlRUMDQ1RXFys1V5cXIw2bdrUuk+bNm0k9acn6pPrp5YtW4bFixdj37596Nq1a0OG2SRIzfW1a9dw/fp1+Pn5adrUajUAwMjICDk5OWjfvn3DBq2n6vO5VqlUaNasGQwNDTVtnTt3RlFRESoqKiCXyxs0Zn1Vn1x//fXXCAwMxKRJkwAArq6uePjwISZPnozw8HAYGPCawqvyvO9GU1PTJnH1CeAVKC1yuRw9evTA/v37NW1qtRr79++Hl5dXrft4eXlp9QeA1NTU5/anJ+qTawBYsmQJ5s+fj5SUFLi7u+siVL0nNdedOnXC+fPnkZmZqVmGDh2KAQMGIDMzE7a2troMX6/U53Pdp08fXL16VVOkAsDly5ehUqlYPL1AfXL96NGjGkXS08JV8LWwr9Qb8d3Y2E+xv24SExOFQqEQ8fHxIisrS0yePFmYm5uLoqIiIYQQgYGBYtasWZr+6enpwsjISCxbtkxkZ2eLiIgITmNQR1JzvXjxYiGXy8WmTZtEYWGhZikvL2+sIegNqbl+Fn+FV3dSc52fny9MTEzEtGnTRE5OjkhKShKtW7cW33zzTWMNQW9IzXVERIQwMTERv/32m8jNzRV79+4V7du3F/7+/o01BL1RXl4uMjIyREZGhgAgvv/+e5GRkSFu3LghhBBi1qxZIjAwUNP/6TQGM2fOFNnZ2SImJobTGLwJli9fLtq1ayfkcrnw9PQUx48f12zr37+/CAoK0uq/YcMG0aFDByGXy4WLi4vYtWuXjiPWX1JybWdnJwDUWCIiInQfuB6S+rn+JxZQ0kjN9dGjR0XPnj2FQqEQDg4OYsGCBaKqqkrHUesnKbmurKwU//nPf0T79u2FUqkUtra2YurUqaK0tFT3geuZgwcP1vr392l+g4KCRP/+/Wvs88477wi5XC4cHBzE2rVrdR53Q5IJweuWRERERFLwGSgiIiIiiVhAEREREUnEAoqIiIhIIhZQRERERBKxgCIiIiKSiAUUERERkUQsoIiIiIgkYgFFREREJBELKCLSC/b29oiOjm7sMIiIALCAIqJ/mDBhAoYPH97YYdTq1KlTmDx5coOfx97eHjKZDDKZDC1atICrqyvWrFkj+TgymQzbtm179QES0WuBBRQRNarKyso69XvrrbfQokWLBo7miXnz5qGwsBAXLlzAuHHjEBwcjN27d+vk3ESkH1hAEVGdXbhwAe+99x5atmwJa2trBAYGoqSkRLM9JSUFffv2hbm5OSwtLfHBBx/g2rVrmu3Xr1+HTCbD77//jv79+0OpVCIhIUFz5WvZsmVQqVSwtLRESEiIVnH17C08mUyGNWvWYMSIEWjRogWcnJywY8cOrXh37NgBJycnKJVKDBgwAOvWrYNMJkNZWdkLx2liYoI2bdrAwcEBYWFhsLCwQGpqqmb7qVOn8O6778LKygpmZmbo378/zp49qxUrAIwYMQIymUyzDgDbt2+Hm5sblEolHBwcEBkZiaqqqrqkn4heIyygiKhOysrKMHDgQHTv3h2nT59GSkoKiouL4e/vr+nz8OFDhIaG4vTp09i/fz8MDAwwYsQIqNVqrWPNmjUL06dPR3Z2Nnx8fAAABw8exLVr13Dw4EGsW7cO8fHxiI+Pf2FMkZGR8Pf3xx9//AFfX1+MHTsWd+/eBQDk5eXh448/xvDhw3Hu3DlMmTIF4eHhksasVquxefNmlJaWQi6Xa9rLy8sRFBSEtLQ0HD9+HE5OTvD19UV5eTmAJwUWAKxduxaFhYWa9SNHjmD8+PGYPn06srKy8PPPPyM+Ph4LFiyQFBcRvQYEEdH/BAUFiWHDhtW6bf78+WLw4MFabQUFBQKAyMnJqXWf27dvCwDi/PnzQggh8vLyBAARHR1d47x2dnaiqqpK0zZy5EgREBCgWbezsxNRUVGadQBizpw5mvUHDx4IAGL37t1CCCHCwsJEly5dtM4THh4uAIjS0tLaE/C/88jlcmFsbCyMjIwEAGFhYSGuXLny3H2qq6uFiYmJ2Llzp1Z8W7du1eo3aNAgsXDhQq229evXC5VK9dxjE9HriVegiKhOzp07h4MHD6Jly5aapVOnTgCguU135coVjB49Gg4ODjA1NdXcusrPz9c6lru7e43ju7i4wNDQULOuUqlw69atF8bUtWtXzX8bGxvD1NRUs09OTg48PDy0+nt6etZprDNnzkRmZiYOHDiAnj17IioqCo6OjprtxcXFCA4OhpOTE8zMzGBqaooHDx7UGOezzp07h3nz5mnlMDg4GIWFhXj06FGdYiOi14NRYwdARPrhwYMH8PPzw7fffltjm0qlAgD4+fnBzs4OsbGxsLGxgVqtRpcuXVBRUaHV39jYuMYxmjVrprUuk8lq3Pp7FfvUhZWVFRwdHeHo6IiNGzfC1dUV7u7ucHZ2BgAEBQXhzp07+OGHH2BnZweFQgEvL68a43zWgwcPEBkZiQ8//LDGNqVS+dJxE5HusIAiojpxc3PD5s2bYW9vDyOjmn867ty5g5ycHMTGxqJfv34AgLS0NF2HqdGxY0ckJydrtT19FkkKW1tbBAQEYPbs2di+fTsAID09HStWrICvry8AoKCgQOtheuBJcVddXa3V5ubmhpycHK2rWUSkn3gLj4i03Lt3D5mZmVpLQUEBQkJCcPfuXYwePRqnTp3CtWvXsGfPHnzyySeorq5Gq1atYGlpidWrV+Pq1as4cOAAQkNDG20cU6ZMwaVLlxAWFobLly9jw4YNmofSZTKZpGNNnz4dO3fuxOnTpwEATk5OWL9+PbKzs3HixAmMHTsWzZs319rH3t4e+/fvR1FREUpLSwEAc+fOxa+//orIyEhcvHgR2dnZSExMxJw5c15+wESkUyygiEjLoUOH0L17d60lMjISNjY2SE9PR3V1NQYPHgxXV1d8/vnnMDc3h4GBAQwMDJCYmIgzZ86gS5cumDFjBpYuXdpo43j77bexadMmbNmyBV27dsXKlSs1v8JTKBSSjuXs7IzBgwdj7ty5AIC4uDiUlpbCzc0NgYGB+Oyzz9C6dWutfb777jukpqbC1tYW3bt3BwD4+PggKSkJe/fuhYeHB3r16oWoqCjY2dm9ghETkS7JhBCisYMgItKFBQsWYNWqVSgoKGjsUIhIz/EZKCJqslasWAEPDw9YWloiPT0dS5cuxbRp0xo7LCJqAlhAEVGTdeXKFXzzzTe4e/cu2rVrhy+++AKzZ89u7LCIqAngLTwiIiIiifgQOREREZFELKCIiIiIJGIBRURERCQRCygiIiIiiVhAEREREUnEAoqIiIhIIhZQRERERBKxgCIiIiKS6L+dFKcPEgK16gAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It looks like smaller learning rates, as long as they're not too small, give us the best performance."
      ],
      "metadata": {
        "id": "I8Cu-2urcaqR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Our best ROC AUC score is:"
      ],
      "metadata": {
        "id": "I2UOwZIAcqwB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "max(val_aucs)"
      ],
      "metadata": {
        "id": "d0R01nV-D2rr",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ba05362e-a273-4465-c36a-365fbd1ce8db"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.7712970684456989"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Which, is a bit better than what we found above. Hooray!"
      ],
      "metadata": {
        "id": "4RMFtkm7ctAi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Determine the learning rate and number of trees used for the best ROC AUC score above."
      ],
      "metadata": {
        "id": "BDnbj1Zoc6LJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "### YOUR CODE HERE ###\n",
        "max_auc = max(val_aucs)\n",
        "max_auc_index = val_aucs.index(max_auc)\n",
        "best_learning_rate = learning_rates[max_auc_index]\n",
        "best_num_trees = best_iters[max_auc_index]\n",
        "print(f\"Best Learning Rate: {best_learning_rate}\")\n",
        "print(f\"Best Number of Trees: {best_num_trees}\")"
      ],
      "metadata": {
        "id": "yuQ5dOYgdBOX",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a5a7a7e5-e7d7-4bec-f19c-d8e2c1270cee"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Best Learning Rate: 0.0925\n",
            "Best Number of Trees: 15\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Parameter Search"
      ],
      "metadata": {
        "id": "3Ot4jOkDdloe"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The learning rate and early stopping rounds aren't the only hyperparameters that we can play with here. Here are some others:"
      ],
      "metadata": {
        "id": "COlNFxOWdrl3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "* *max_depth* - The maximum depth to which trees within the ensemble can be grown.\n",
        "* *gamma* - Limits the complexity of trees in the ensemble by only allowing a node to be split if the reduction in the loss function is greater than a certain amount.\n",
        "* *min_child_weight* - Controls the complexity of trees by only splitting nodes if they have at least a certain amount of \"sample weight\". If all samples have equal weighting, this equates to a minimum number of training samples in a node.\n",
        "* *colsample_bytree* - This is a randomly selected fraction of features that are used to grow each tree in the ensemble.\n",
        "* *subsample* - Controls what fraction of samples from the training data is randomly selected prior to growing a new tree for the ensemble.\n",
        "\n"
      ],
      "metadata": {
        "id": "XK4wSQQLd8oI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll now create a parameter grid with different possible values for these parameters. For the ordinal parameters, we'll specify a range, while for the numeric ones will just use a uniform sample."
      ],
      "metadata": {
        "id": "5eiuuFD0iL8q"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "param_grid = {'max_depth': [2,3,4,5,6,7],\n",
        "              'gamma':uniform(loc=0.0, scale = 3),\n",
        "              'min_child_weight':list(range(1,151)),\n",
        "              'colsample_bytree':uniform(loc = 0.1, scale = 0.9),\n",
        "              'subsample':uniform(loc=0.5, scale = 0.5),\n",
        "              'learning_rate':uniform(loc=0.01, scale = 0.5),\n",
        "              }"
      ],
      "metadata": {
        "id": "GtSkPD33EmWE"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, we'll create a list of 1,000 samples from the param_grid above."
      ],
      "metadata": {
        "id": "1OWpFVYPifAY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "rng = np.random.RandomState(6600)\n",
        "n_iter = 1000\n",
        "param_list = list(ParameterSampler(param_grid, n_iter, random_state = rng))"
      ],
      "metadata": {
        "id": "i2YAuBp6FXj5"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "For example, our first random sample is:"
      ],
      "metadata": {
        "id": "NkuLJz6Yilug"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "param_list[0]"
      ],
      "metadata": {
        "id": "8rX8o5HlFik5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "423d5f92-877a-4910-ec29-b8661f18db6e"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'colsample_bytree': 0.2792456627458437,\n",
              " 'gamma': 2.9653930463578906,\n",
              " 'learning_rate': 0.46566911647102255,\n",
              " 'max_depth': 3,\n",
              " 'min_child_weight': 8,\n",
              " 'subsample': 0.577004433672033}"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, we also want to specify our objective function and evaluation metric as part of our parameters, so we can expand our parameters with:"
      ],
      "metadata": {
        "id": "K3ShogoTkRzv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "param_list[0]['objective'] = 'binary:logistic'\n",
        "param_list[0]['eval_metric'] = 'auc'"
      ],
      "metadata": {
        "id": "IVZyu-a2kanN"
      },
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, we should have the parameters we need:"
      ],
      "metadata": {
        "id": "4xwWzGLRkfvZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "param_list[0]"
      ],
      "metadata": {
        "id": "xPJpygPWkhek",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "db7f815e-1be1-43e3-8bfd-f138275b5986"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'colsample_bytree': 0.2792456627458437,\n",
              " 'gamma': 2.9653930463578906,\n",
              " 'learning_rate': 0.46566911647102255,\n",
              " 'max_depth': 3,\n",
              " 'min_child_weight': 8,\n",
              " 'subsample': 0.577004433672033,\n",
              " 'objective': 'binary:logistic',\n",
              " 'eval_metric': 'auc'}"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can train a model using this set of parameters:"
      ],
      "metadata": {
        "id": "cHe20iiSixZ5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "xgb_model_2 = xgb.train(param_list[0], dtrain=dtrain, num_boost_round=1000, early_stopping_rounds=30, evals=[(dtrain, 'train'), (dval, 'val')])"
      ],
      "metadata": {
        "id": "-Zt4KsckFo8P",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7b77be51-1891-4bf1-9d10-501a459d7f37"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[0]\ttrain-auc:0.58162\tval-auc:0.56806\n",
            "[1]\ttrain-auc:0.60715\tval-auc:0.55886\n",
            "[2]\ttrain-auc:0.62185\tval-auc:0.57417\n",
            "[3]\ttrain-auc:0.65081\tval-auc:0.59406\n",
            "[4]\ttrain-auc:0.65262\tval-auc:0.57417\n",
            "[5]\ttrain-auc:0.65637\tval-auc:0.58018\n",
            "[6]\ttrain-auc:0.66047\tval-auc:0.58289\n",
            "[7]\ttrain-auc:0.66948\tval-auc:0.57421\n",
            "[8]\ttrain-auc:0.67768\tval-auc:0.57659\n",
            "[9]\ttrain-auc:0.72481\tval-auc:0.64205\n",
            "[10]\ttrain-auc:0.72796\tval-auc:0.63649\n",
            "[11]\ttrain-auc:0.73347\tval-auc:0.63613\n",
            "[12]\ttrain-auc:0.73726\tval-auc:0.63422\n",
            "[13]\ttrain-auc:0.74766\tval-auc:0.64696\n",
            "[14]\ttrain-auc:0.75157\tval-auc:0.64732\n",
            "[15]\ttrain-auc:0.75842\tval-auc:0.64044\n",
            "[16]\ttrain-auc:0.76019\tval-auc:0.63671\n",
            "[17]\ttrain-auc:0.76412\tval-auc:0.64030\n",
            "[18]\ttrain-auc:0.76621\tval-auc:0.64183\n",
            "[19]\ttrain-auc:0.76913\tval-auc:0.63500\n",
            "[20]\ttrain-auc:0.77072\tval-auc:0.63007\n",
            "[21]\ttrain-auc:0.77549\tval-auc:0.62888\n",
            "[22]\ttrain-auc:0.77738\tval-auc:0.63095\n",
            "[23]\ttrain-auc:0.77928\tval-auc:0.62749\n",
            "[24]\ttrain-auc:0.78471\tval-auc:0.62086\n",
            "[25]\ttrain-auc:0.78745\tval-auc:0.61883\n",
            "[26]\ttrain-auc:0.79069\tval-auc:0.61657\n",
            "[27]\ttrain-auc:0.79411\tval-auc:0.61483\n",
            "[28]\ttrain-auc:0.79526\tval-auc:0.61842\n",
            "[29]\ttrain-auc:0.79688\tval-auc:0.61811\n",
            "[30]\ttrain-auc:0.79858\tval-auc:0.61666\n",
            "[31]\ttrain-auc:0.80043\tval-auc:0.62061\n",
            "[32]\ttrain-auc:0.81426\tval-auc:0.64081\n",
            "[33]\ttrain-auc:0.81781\tval-auc:0.64007\n",
            "[34]\ttrain-auc:0.82170\tval-auc:0.63903\n",
            "[35]\ttrain-auc:0.82183\tval-auc:0.63605\n",
            "[36]\ttrain-auc:0.82428\tval-auc:0.63268\n",
            "[37]\ttrain-auc:0.82584\tval-auc:0.63427\n",
            "[38]\ttrain-auc:0.82616\tval-auc:0.63669\n",
            "[39]\ttrain-auc:0.82739\tval-auc:0.63852\n",
            "[40]\ttrain-auc:0.82853\tval-auc:0.64075\n",
            "[41]\ttrain-auc:0.83160\tval-auc:0.63970\n",
            "[42]\ttrain-auc:0.83357\tval-auc:0.64228\n",
            "[43]\ttrain-auc:0.83631\tval-auc:0.64102\n",
            "[44]\ttrain-auc:0.84463\tval-auc:0.65832\n",
            "[45]\ttrain-auc:0.84560\tval-auc:0.65545\n",
            "[46]\ttrain-auc:0.84716\tval-auc:0.65135\n",
            "[47]\ttrain-auc:0.84831\tval-auc:0.65501\n",
            "[48]\ttrain-auc:0.84975\tval-auc:0.65707\n",
            "[49]\ttrain-auc:0.85034\tval-auc:0.65793\n",
            "[50]\ttrain-auc:0.85271\tval-auc:0.65696\n",
            "[51]\ttrain-auc:0.85337\tval-auc:0.65640\n",
            "[52]\ttrain-auc:0.85672\tval-auc:0.65307\n",
            "[53]\ttrain-auc:0.85829\tval-auc:0.65133\n",
            "[54]\ttrain-auc:0.86029\tval-auc:0.65242\n",
            "[55]\ttrain-auc:0.86362\tval-auc:0.64849\n",
            "[56]\ttrain-auc:0.86442\tval-auc:0.64812\n",
            "[57]\ttrain-auc:0.87054\tval-auc:0.66734\n",
            "[58]\ttrain-auc:0.87113\tval-auc:0.66702\n",
            "[59]\ttrain-auc:0.87275\tval-auc:0.66698\n",
            "[60]\ttrain-auc:0.87339\tval-auc:0.66619\n",
            "[61]\ttrain-auc:0.87556\tval-auc:0.66603\n",
            "[62]\ttrain-auc:0.87678\tval-auc:0.66617\n",
            "[63]\ttrain-auc:0.87808\tval-auc:0.67010\n",
            "[64]\ttrain-auc:0.87885\tval-auc:0.66632\n",
            "[65]\ttrain-auc:0.87999\tval-auc:0.66393\n",
            "[66]\ttrain-auc:0.88127\tval-auc:0.66518\n",
            "[67]\ttrain-auc:0.88217\tval-auc:0.66535\n",
            "[68]\ttrain-auc:0.88378\tval-auc:0.66980\n",
            "[69]\ttrain-auc:0.88521\tval-auc:0.66671\n",
            "[70]\ttrain-auc:0.88610\tval-auc:0.66708\n",
            "[71]\ttrain-auc:0.88792\tval-auc:0.66913\n",
            "[72]\ttrain-auc:0.88872\tval-auc:0.66830\n",
            "[73]\ttrain-auc:0.88910\tval-auc:0.66587\n",
            "[74]\ttrain-auc:0.89002\tval-auc:0.66646\n",
            "[75]\ttrain-auc:0.89123\tval-auc:0.66666\n",
            "[76]\ttrain-auc:0.89294\tval-auc:0.66783\n",
            "[77]\ttrain-auc:0.89350\tval-auc:0.66951\n",
            "[78]\ttrain-auc:0.89525\tval-auc:0.66688\n",
            "[79]\ttrain-auc:0.89654\tval-auc:0.66738\n",
            "[80]\ttrain-auc:0.89758\tval-auc:0.67056\n",
            "[81]\ttrain-auc:0.89888\tval-auc:0.66801\n",
            "[82]\ttrain-auc:0.90043\tval-auc:0.66870\n",
            "[83]\ttrain-auc:0.90128\tval-auc:0.66783\n",
            "[84]\ttrain-auc:0.90190\tval-auc:0.66960\n",
            "[85]\ttrain-auc:0.90348\tval-auc:0.66721\n",
            "[86]\ttrain-auc:0.90465\tval-auc:0.66621\n",
            "[87]\ttrain-auc:0.90543\tval-auc:0.66462\n",
            "[88]\ttrain-auc:0.90668\tval-auc:0.66614\n",
            "[89]\ttrain-auc:0.90776\tval-auc:0.66638\n",
            "[90]\ttrain-auc:0.90803\tval-auc:0.66543\n",
            "[91]\ttrain-auc:0.91050\tval-auc:0.66280\n",
            "[92]\ttrain-auc:0.91258\tval-auc:0.66072\n",
            "[93]\ttrain-auc:0.91258\tval-auc:0.66072\n",
            "[94]\ttrain-auc:0.91266\tval-auc:0.66035\n",
            "[95]\ttrain-auc:0.91459\tval-auc:0.65971\n",
            "[96]\ttrain-auc:0.91459\tval-auc:0.65971\n",
            "[97]\ttrain-auc:0.91563\tval-auc:0.66119\n",
            "[98]\ttrain-auc:0.91660\tval-auc:0.65812\n",
            "[99]\ttrain-auc:0.91692\tval-auc:0.65695\n",
            "[100]\ttrain-auc:0.91874\tval-auc:0.65699\n",
            "[101]\ttrain-auc:0.91874\tval-auc:0.65699\n",
            "[102]\ttrain-auc:0.91874\tval-auc:0.65699\n",
            "[103]\ttrain-auc:0.92023\tval-auc:0.65635\n",
            "[104]\ttrain-auc:0.92087\tval-auc:0.65807\n",
            "[105]\ttrain-auc:0.92222\tval-auc:0.65913\n",
            "[106]\ttrain-auc:0.92233\tval-auc:0.65871\n",
            "[107]\ttrain-auc:0.92277\tval-auc:0.65968\n",
            "[108]\ttrain-auc:0.92487\tval-auc:0.65774\n",
            "[109]\ttrain-auc:0.92520\tval-auc:0.65832\n",
            "[110]\ttrain-auc:0.92648\tval-auc:0.65493\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, what you should do is go through all the parameter combinations in our grid and figure out what the best model (best stopping time for highest validation ROC AUC) for each of them. Then, determine the optimal parameter combination. Note this might take a couple minutes to run."
      ],
      "metadata": {
        "id": "KGRHylB1kvRO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "val_aucs = []\n",
        "for params in param_list:\n",
        "  ### YOUR CODE HERE ###\n",
        "    params['objective'] = 'binary:logistic'\n",
        "    params['eval_metric'] = 'auc'\n",
        "\n",
        "    # Train XGBoost model with early stopping\n",
        "    xgb_model = xgb.train(params, dtrain=dtrain, num_boost_round=1000,\n",
        "                          early_stopping_rounds=30, evals=[(dtrain, 'train'), (dval, 'val')],\n",
        "                          verbose_eval=False)\n",
        "\n",
        "    val_set_pred_prob = xgb_model.predict(dval, iteration_range=(0, xgb_model.best_iteration + 1))\n",
        "\n",
        "    # Calculate and store validation ROC AUC\n",
        "    val_auc = roc_auc_score(y_val, val_set_pred_prob)\n",
        "    val_aucs.append(val_auc)\n"
      ],
      "metadata": {
        "id": "Klo5I2z3H7aX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's create a dataframe from the various parameters, and add a column with the corresponding ROC AUC scores for the best XGBoost models."
      ],
      "metadata": {
        "id": "9b05GDcdmwUS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "xgb_param_search_df = pd.DataFrame(param_list)\n",
        "xgb_param_search_df['Validation ROC AUC'] = val_aucs\n",
        "xgb_param_search_df.head()"
      ],
      "metadata": {
        "id": "yNI0cQorJz8k",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 244
        },
        "outputId": "f078db74-803f-4027-ed86-fddf671ff59d"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   colsample_bytree     gamma  learning_rate  max_depth  min_child_weight  \\\n",
              "0          0.279246  2.965393       0.465669          3                 8   \n",
              "1          0.464960  0.176980       0.395888          3                20   \n",
              "2          0.751689  2.221332       0.062536          5                93   \n",
              "3          0.777785  2.790733       0.120128          6               142   \n",
              "4          0.794387  0.217399       0.267152          4               104   \n",
              "\n",
              "   subsample        objective eval_metric  Validation ROC AUC  \n",
              "0   0.577004  binary:logistic         auc            0.670565  \n",
              "1   0.795599  binary:logistic         auc            0.737164  \n",
              "2   0.503705  binary:logistic         auc            0.725925  \n",
              "3   0.517629  binary:logistic         auc            0.710717  \n",
              "4   0.919446  binary:logistic         auc            0.728061  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-a5346431-2af1-4d47-afd0-1e3133c0886c\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>colsample_bytree</th>\n",
              "      <th>gamma</th>\n",
              "      <th>learning_rate</th>\n",
              "      <th>max_depth</th>\n",
              "      <th>min_child_weight</th>\n",
              "      <th>subsample</th>\n",
              "      <th>objective</th>\n",
              "      <th>eval_metric</th>\n",
              "      <th>Validation ROC AUC</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.279246</td>\n",
              "      <td>2.965393</td>\n",
              "      <td>0.465669</td>\n",
              "      <td>3</td>\n",
              "      <td>8</td>\n",
              "      <td>0.577004</td>\n",
              "      <td>binary:logistic</td>\n",
              "      <td>auc</td>\n",
              "      <td>0.670565</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.464960</td>\n",
              "      <td>0.176980</td>\n",
              "      <td>0.395888</td>\n",
              "      <td>3</td>\n",
              "      <td>20</td>\n",
              "      <td>0.795599</td>\n",
              "      <td>binary:logistic</td>\n",
              "      <td>auc</td>\n",
              "      <td>0.737164</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.751689</td>\n",
              "      <td>2.221332</td>\n",
              "      <td>0.062536</td>\n",
              "      <td>5</td>\n",
              "      <td>93</td>\n",
              "      <td>0.503705</td>\n",
              "      <td>binary:logistic</td>\n",
              "      <td>auc</td>\n",
              "      <td>0.725925</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.777785</td>\n",
              "      <td>2.790733</td>\n",
              "      <td>0.120128</td>\n",
              "      <td>6</td>\n",
              "      <td>142</td>\n",
              "      <td>0.517629</td>\n",
              "      <td>binary:logistic</td>\n",
              "      <td>auc</td>\n",
              "      <td>0.710717</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.794387</td>\n",
              "      <td>0.217399</td>\n",
              "      <td>0.267152</td>\n",
              "      <td>4</td>\n",
              "      <td>104</td>\n",
              "      <td>0.919446</td>\n",
              "      <td>binary:logistic</td>\n",
              "      <td>auc</td>\n",
              "      <td>0.728061</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a5346431-2af1-4d47-afd0-1e3133c0886c')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-a5346431-2af1-4d47-afd0-1e3133c0886c button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-a5346431-2af1-4d47-afd0-1e3133c0886c');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-b3e3df8e-419e-4fd0-a103-bdee373612d5\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b3e3df8e-419e-4fd0-a103-bdee373612d5')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-b3e3df8e-419e-4fd0-a103-bdee373612d5 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "xgb_param_search_df",
              "summary": "{\n  \"name\": \"xgb_param_search_df\",\n  \"rows\": 1000,\n  \"fields\": [\n    {\n      \"column\": \"colsample_bytree\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.2618906133507472,\n        \"min\": 0.10049747555446586,\n        \"max\": 0.9984166369107523,\n        \"num_unique_values\": 1000,\n        \"samples\": [\n          0.8100937143557805,\n          0.35867087330612524,\n          0.21322125666255504\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"gamma\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.874466634623138,\n        \"min\": 0.00047389612127002323,\n        \"max\": 2.9981008402209435,\n        \"num_unique_values\": 1000,\n        \"samples\": [\n          2.5198196921862177,\n          2.1615229421458606,\n          1.4558425185851156\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"learning_rate\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.14207293582763358,\n        \"min\": 0.011405817241474428,\n        \"max\": 0.509658448781877,\n        \"num_unique_values\": 1000,\n        \"samples\": [\n          0.49477038625463166,\n          0.28133706907048445,\n          0.02556007414139571\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"max_depth\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1,\n        \"min\": 2,\n        \"max\": 7,\n        \"num_unique_values\": 6,\n        \"samples\": [\n          3,\n          5,\n          7\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"min_child_weight\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 42,\n        \"min\": 1,\n        \"max\": 150,\n        \"num_unique_values\": 150,\n        \"samples\": [\n          109,\n          57,\n          95\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"subsample\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.1476099453928048,\n        \"min\": 0.5002909436713507,\n        \"max\": 0.9999949880371576,\n        \"num_unique_values\": 1000,\n        \"samples\": [\n          0.5965579025017047,\n          0.8341614750006976,\n          0.7655295923649292\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"objective\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"binary:logistic\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"eval_metric\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"auc\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Validation ROC AUC\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.026424701350133122,\n        \"min\": 0.5864030726794534,\n        \"max\": 0.7730190358586182,\n        \"num_unique_values\": 989,\n        \"samples\": [\n          0.7355106164603246\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Determine the parameter set with the best ROC AUC score. What can you say about it?"
      ],
      "metadata": {
        "id": "9nBz7L0-nHYy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "### YOUR CODE HERE ###\n",
        "best_roc_auc = xgb_param_search_df['Validation ROC AUC'].max()\n",
        "best_param_row = xgb_param_search_df[xgb_param_search_df['Validation ROC AUC'] == best_roc_auc]\n",
        "best_params = best_param_row.drop(columns=['Validation ROC AUC']).iloc[0].to_dict()\n",
        "print(best_params)"
      ],
      "metadata": {
        "id": "UAP3SqY9KATt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a3609737-f492-46e5-d1e6-9d0291ca394e"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'colsample_bytree': 0.6260970518554606, 'gamma': 2.279187079807467, 'learning_rate': 0.19044150619652406, 'max_depth': 7, 'min_child_weight': 1, 'subsample': 0.9007100657126261, 'objective': 'binary:logistic', 'eval_metric': 'auc'}\n"
          ]
        }
      ]
    }
  ]
}