{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "4Taf4-erLdt1" }, "source": [ "## Use Glove in Pytorch to Finish NLP task - sentiment analysis\n", "Presented by: Long Xu\n", "Contact: xulong3@mail2.sysu.edu.cn or you can find me in the wecom group chat." ] }, { "cell_type": "markdown", "metadata": { "id": "R4HY9cPzLasO" }, "source": [ "Word embeddings can provide more meaningful vector representations of words. Using pre-trained word embeddings in NLP tasks can greatly improve training efficiency. In this tutorial, I will share with you how to use pre-trained **GloVe** word embeddings in PyTorch and use them to complete a simple NLP task - sentiment analysis.\n", "https://nlp.stanford.edu/projects/glove/" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": { "id": "H-PBi0IEM4fg" }, "source": [ "## Glove Word Embedding\n" ] }, { "cell_type": "markdown", "metadata": { "id": "Sqs4ERy4NIjl" }, "source": [ "GloVe is an **unsupervised learning algorithm** for obtaining vector representations for words. Training is performed on aggregated global word-word co-occurrence statistics from a corpus, and the resulting representations showcase **interesting linear substructures** of the word vector space." ] }, { "cell_type": "markdown", "metadata": { "id": "oBhEoO1jOKkS" }, "source": [ "\n", "\n", "* Wikipedia+gigaword（6B）\n", "* crawler（42B）\n", "* crawler（840B）\n", "* twitter(27B)\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "E8QB_RrRPY1c" }, "source": [ "According to the size of the word embedding vector, it can be divided into different dimensions such as 50 dims, 100 dims, 200 dims, etc." ] }, { "cell_type": "markdown", "metadata": { "id": "jMWbG8Z2PpqR" }, "source": [ "We do not need to GloVe on our own, we can use official glove." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#download the dataset we need\n", "import urllib.request\n", "import tarfile\n", "import os\n", "\n", "url = \"https://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\"\n", "save_path = \"aclImdb_v1\"\n", "\n", "urllib.request.urlretrieve(url, save_path)\n", "\n", "\n", "with tarfile.open(save_path, 'r:gz') as tar:\n", " tar.extractall()\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# colab environment\n", "# !pip install torch==2.2.1+cu121 torchvision==0.17.1+cu121 torchaudio==2.2.1+cu121 torchtext==0.17.1 -f https://download.pytorch.org/whl/torch_stable.html " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.13.1+cu117\n" ] } ], "source": [ "import torch\n", "print(torch.__version__) #modelarts of huaweicloud\n", "import torchtext" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "xxUUoK2_9AZL" }, "outputs": [], "source": [ "import torch\n", "from torchtext.vocab import GloVe # GloVe is well embedded in the torchtext package, easy to use" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "EDlBJmoW9ENa", "outputId": "5171de94-1162-4c03-e4a6-08ce6b13ce1c" }, "outputs": [], "source": [ "glove = GloVe(name='6B', dim=100)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([0.8798])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from torch.nn.functional import cosine_similarity\n", "cat_vec = glove.get_vecs_by_tokens(['cat'])\n", "dog_vet = glove.get_vecs_by_tokens(['dog'])\n", "cosine_similarity(cat_vec, dog_vet)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "70Y0vNg1-hwT", "outputId": "8cd75432-9147-461e-f2a0-52e3fd42e3e4" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([[ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000],\n", " [ 0.3668, -0.0684, 0.2483, -0.3375, 0.8622, -0.3554, 0.5974, 0.3588,\n", " -0.0489, 0.2077, 0.5716, -0.3194, 0.2442, 0.1925, -0.2414, 0.0193,\n", " 0.2527, 0.2217, -0.3872, 0.1858, 0.0425, -0.1987, 0.5122, 1.0127,\n", " 0.2462, -0.1484, 0.3403, -0.4819, 0.2935, -0.2058, 0.1011, 0.5394,\n", " -0.0551, -0.2674, -0.2362, 0.1941, 0.3256, 0.2670, -0.7003, -0.1265,\n", " -0.0400, -0.1269, 0.3574, -0.0255, 0.3822, 0.0805, 0.5139, -1.1916,\n", " -0.3035, -0.9837, -0.3032, -0.2324, 0.3917, 0.6209, -0.2342, -2.4306,\n", " -0.3120, 0.2025, 1.3618, 0.2672, -0.5367, 0.3688, -0.4156, -0.4474,\n", " 0.5197, -0.1808, -0.2543, 0.2538, 0.5296, 0.4168, 0.4412, 0.4566,\n", " -0.7514, 0.3024, -0.8125, -0.0619, 0.2169, 0.4603, -1.0819, -0.2741,\n", " 0.4917, 0.2430, 0.0773, 0.0366, -0.3204, -0.0267, 0.1401, -0.3633,\n", " -0.2605, 0.4131, 0.0760, -0.0543, -0.1023, 0.6122, -0.2879, 0.3588,\n", " -0.0892, 0.5206, 0.3982, -0.9714],\n", " [ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000],\n", " [-0.1077, 0.1105, 0.5981, -0.5436, 0.6740, 0.1066, 0.0389, 0.3548,\n", " 0.0635, -0.0942, 0.1579, -0.8166, 0.1417, 0.2194, 0.5850, -0.5216,\n", " 0.2278, -0.1664, -0.6823, 0.3587, 0.4257, 0.1902, 0.9196, 0.5756,\n", " 0.4618, 0.4236, -0.0954, -0.4275, -0.1657, -0.0568, -0.2959, 0.2604,\n", " -0.2661, -0.0704, -0.2766, 0.1582, 0.6982, 0.4308, 0.2795, -0.4544,\n", " -0.3380, -0.5818, 0.2236, -0.5778, -0.2686, -0.2042, 0.5639, -0.5852,\n", " -0.1436, -0.6422, 0.0055, -0.3525, 0.1616, 1.1796, -0.4767, -2.7553,\n", " -0.1321, -0.0477, 1.0655, 1.1034, -0.2208, 0.1867, 0.1318, 0.1512,\n", " 0.7131, -0.3521, 0.9135, 0.6178, 0.7099, 0.2395, -0.1457, -0.3786,\n", " -0.0460, -0.4737, 0.2385, 0.2054, -0.1900, 0.3251, -1.1112, -0.3634,\n", " 0.9868, -0.0848, -0.5401, 0.1173, -1.0194, -0.2442, 0.1277, 0.0139,\n", " 0.0804, -0.3541, 0.3495, -0.7226, 0.3755, 0.4441, -0.9906, 0.6121,\n", " -0.3511, -0.8316, 0.4529, 0.0826],\n", " [ 0.2309, 0.2828, 0.6318, -0.5941, -0.5860, 0.6326, 0.2440, -0.1411,\n", " 0.0608, -0.7898, -0.2910, 0.1429, 0.7227, 0.2043, 0.1407, 0.9876,\n", " 0.5253, 0.0975, 0.8822, 0.5122, 0.4020, 0.2117, -0.0131, -0.7162,\n", " 0.5539, 1.1452, -0.8804, -0.5022, -0.2281, 0.0239, 0.1072, 0.0837,\n", " 0.5501, 0.5848, 0.7582, 0.4571, -0.2800, 0.2522, 0.6896, -0.6097,\n", " 0.1958, 0.0442, -0.3114, -0.6883, -0.2272, 0.4618, -0.7716, 0.1021,\n", " 0.5564, 0.0674, -0.5721, 0.2374, 0.4717, 0.8277, -0.2926, -1.3422,\n", " -0.0993, 0.2814, 0.4160, 0.1058, 0.6220, 0.8950, -0.2345, 0.5135,\n", " 0.9938, 1.1846, -0.1636, 0.2065, 0.7385, 0.2406, -0.9647, 0.1348,\n", " -0.0072, 0.3302, -0.1236, 0.2719, -0.4095, 0.0219, -0.6069, 0.4076,\n", " 0.1957, -0.4180, 0.1864, -0.0327, -0.7857, -0.1385, 0.0440, -0.0844,\n", " 0.0491, 0.2410, 0.4527, -0.1868, 0.4618, 0.0891, -0.1819, -0.0152,\n", " -0.7368, -0.1453, 0.1510, -0.7149]])\n" ] } ], "source": [ "# Get vectors\n", "tensor = glove.get_vecs_by_tokens(['', '1998', '199999998', ',', 'cat'], True)\n", "print(tensor)" ] }, { "cell_type": "markdown", "metadata": { "id": "mcsIsvQKQMPT" }, "source": [ "This function provided by PyTorch is very convenient. If the token is not in GloVe, the function will return a vector of all zeros. If you run the above code, you can observe something interesting: the empty string and uncommon numbers like 199999998 are not in the vocabulary, while the common numbers like 1998 and punctuation are in the vocabulary." ] }, { "cell_type": "markdown", "metadata": { "id": "r8rGmORoQoM0" }, "source": [ "The GloVe class internally maintains a matrix, an array of vectors for each word. Therefore, GloVe needs a **mapping table** to map words to vector array indexs. **glove.itos** and **glove.stoi** complete the mapping between indexs and word strings. For example, with the following code, we can know the size of the vocabulary and access the first few words of the vocabulary:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "AMxuy-BP-vzR", "outputId": "b99afe95-4800-4027-ee06-0ac01e4c9ae8" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "400000\n", "the , . of\n" ] } ], "source": [ "myvocab = glove.itos\n", "print(len(myvocab))\n", "print(myvocab[0], myvocab[1], myvocab[2], myvocab[3])" ] }, { "cell_type": "markdown", "metadata": { "id": "i-r8OTZVRyKp" }, "source": [ "Let's understand the meaning of word embedding through a practical example. Word embeddings are vectors, and the relationship between the vectors often corresponds to the semantic relationship. Using the relative relationship of word embeddings, we can answer the question \"x1 is to y1, who is x2 to?\". For example, if a man is to a woman, then a king is to a queen. Suppose the vector we are looking for is y2, we want x1-y1=x2-y2, that is, find a vector y2 that is closest to x2-(x1-y1). This process can be described by the following code. " ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "okJaylv5-4-l", "outputId": "82660090-3ce9-45f2-bedb-dfa65eb05e3f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "queen\n", "short\n", "yellow\n" ] } ], "source": [ "def get_counterpart(x1, y1, x2):\n", " \"\"\"Find y2 that makes x1-y1=x2-y2\"\"\"\n", " # Convert input words to their corresponding indices in the GloVe vocabulary\n", " x1_id = glove.stoi[x1] # Get index of x1 in GloVe vocabulary\n", " y1_id = glove.stoi[y1] # Get index of y1 in GloVe vocabulary\n", " x2_id = glove.stoi[x2] # Get index of x2 in GloVe vocabulary\n", " \n", " # Retrieve word vectors for x1, y1, and x2 from GloVe embeddings\n", " x1, y1, x2 = glove.get_vecs_by_tokens([x1,y1, x2], True)\n", " \n", " # Calculate the target vector: x2 - x1 + y1\n", " # This represents the vector relationship we want to preserve\n", " target = x2 - x1 + y1\n", " \n", " # Initialize variables to track maximum similarity and corresponding word index\n", " max_sim = 0 \n", " max_id = -1 # Index of the word with maximum similarity\n", " \n", " # Iterate through all words in the custom vocabulary (myvocab)\n", " for i in range(len(myvocab)):\n", " # Get the word vector for the current vocabulary word\n", " vector = glove.get_vecs_by_tokens([myvocab[i]], True)[0]\n", " \n", " # Compute cosine similarity between target vector and current word vector\n", " # torch.dot computes the dot product as a measure of similarity\n", " cossim = torch.dot(target, vector)\n", " \n", " # Update max similarity and index if current similarity is higher\n", " # and the word is not one of the input words (x1, y1, x2)\n", " if cossim > max_sim and i not in {x1_id, y1_id, x2_id}:\n", " max_sim = cossim\n", " max_id = i\n", " \n", " # Return the word from myvocab with the highest similarity\n", " return myvocab[max_id]\n", "\n", "\n", "print(get_counterpart('man', 'woman', 'king'))\n", "print(get_counterpart('more', 'less', 'long'))\n", "print(get_counterpart('apple', 'red', 'banana'))" ] }, { "cell_type": "markdown", "metadata": { "id": "Y8hkOiBETJtV" }, "source": [ "The sentiment analysis task is a relatively simple two-category NLP task: given a passage, whether the sentiment of the output passage is positive or negative.\n", "\n", "\n", "* **Positive**: I went and saw this movie last night after being coaxed to by a few friends of mine. I’ll admit that I was reluctant to see it because from what I knew of Ashton Kutcher he was only able to do comedy. I was wrong. Kutcher played the character of Jake Fischer **very well**, and Kevin Costner played Ben Randall with such professionalism.\n", "* **Negative**: This is a pale imitation of 'Officer and a Gentleman.' There is **NO chemistry** between Kutcher and the unknown woman who plays his love interest. The dialog is **wooden**, the situations **hackneyed**.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "jWf1MF51V-ES" }, "source": [ "These reviews are selected from a large dataset of movies from Stanford University. It contains movie reviews on **IMDb**. This data set is the most commonly used data set in sentiment analysis, and most beginers will use it to train a sentiment analysis model when learning NLP." ] }, { "cell_type": "markdown", "metadata": { "id": "SPFW9VPCWv1S" }, "source": [ "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABIwAAAG1CAYAAACbG29tAAAgAElEQVR4nOzdfXjU9Znv8bfiUCSg4wNRN1iNSpAaqeAuqRDdii4J9eD2SIlgFQ8bttssXZFKdWk5snhAthZLcUvptmS5QLsgCu6SWhIOBitBDVVSMSgO1fhAVCYqw8MgZk7o+SNB8oPwmEwG4vt1Xb06+f4e7nt66R/9XPf3O6ds27Hzz0iSJEmSJElNTk11A5IkSZIkSTqxGBhJkiRJkiQp4PCBUUOC919dyaJF69jeTg2d0H1IkiRJkiR9ARw+MHrtMf75X+ezYlOM+nZq6ITuQ5IkSZIk6QvALWmSJEmSJEkKOK1N37ZnN7sTjR9DXbsS6tSmb5ckSZIkSVI7OGXbjp1/PnDxtfmj+NdVh3qkF6N/9gA39ti/svutlSyY+xgvvJ/YvxjqymU3/hPjR13FmU3B0e5XfsX3H1rNbsIMnTqXUZcd8OrYWh6+++e8kujKX9/7b1yzvvCY+pAkSZIkSVLrtXpLWl3FLO773/N54X3o2vMq/jpvCDcO7EVXdvOnFT/m+9NXUtfQeG/Xr97O2IEhIMaKR1dSF3jTbl5Z8h+8koDQwH9g1Fe7trY1SZIkSZIkHYcWJ4w+9+p8Rv/rSrhsDA9PHcJBwzxbnuKf71vC+4S55q4ZFOWE91/7ZC1zf/hzXtgZ4qtFj3BPbtO1z6eIQlxz1y8pymkKhv70GP805Wm2hwZx1y++x182z4uO1IckSZIkSZLaTCsmjBK8VPIU7wNdh34/GBYBnD2I20ZdASR45fmN7N63Hh7E2LFXESLBC48t408NQEOUFY8+zXZCXFP0d8GwSJIkSZIkSe3q+AOjPVX8oSIBpPONr/dq8ZYzL72icRrolQhvN1/PHcOo3iH45Gn+c1WU7S/M58k/QWjg97gzx7RIkiRJkiQplY7/V9K2RHgFgG3835+O5/ct3bN3FzEAanj/E/jK2fsupHPjP9zK899/jD8tmsj3SZAIDeKuMQMwLpIkSZIkSUqt4w+MPk00bTNLsH1r9Mj3Nxzw93k3cefNv+V/L4+RAC4bU+BWNEmSJEmSpBPA8QdGnSAEJLiKwp/+E391hLAndOD13X9k1eoYhEKEEgn+9N9P89rAMXzF0EiSJEmSJCmljv8Mo/MzuAiA99me6ErX7of/T6hT84d389L8n/L7nXDZrQ8wdmAIPlnJ3Cc3kmjd95EkSZIkSVIrHV1gtDN+cJBz9hX8VU+AKP93zeZjKrq94t+Y+3wCzr6J24ZczDUFt3NZCLaX/ZxFrx0mMmqpD0mSJEmSJLWpwwdGTdvO2LqRzbEDL2YwcNhVhIDtv/0pc59v4Ryjht28U/YUL33SbK1uJbPn/ZEEYW4supXLOgE9hjD2bzOAGKvmPsZre46lD0mSJEmSJLWlTv886Yf/csir3RO8v+JFtuyto6r8ZT7c/h4byp7lk6xruCQNunw5i/R3X+Sl92Ns+UMpvy3fxPsfv8nrr/yRl1b9hl/9+jFWVtXTO+96LkkDGqKs+ulMVkf30vWvxzMhP6MxCAK6Z2Wy5/er+dMnb/LqZ334m6+m0+ko+5AkSZIkSVLbOfyEUZcB3HnPEHqEgD1v80LZSn7/yjb2JznpXHPXw/yf0QPoEYJEbCMvlK1kVdlKfr++lkSXDK4Z/XcMPLvx7rpVP2fRGwkIDWLs7VcRON+6Uy++9Z0hdKWFrWlH7EOSJEmSJElt5ZRtO3b++Yh3NSTYvbspwOnSla6hQ92zjbr36qBHJj3SQnRt8cZWOJo+JEmSJEmS1CpHFxhJkiRJkiTpC+PofiVNkiRJkiRJXxgGRpIkSZIkSQowMJIkSZIkSVKAgZEkSZIkSZICDIwkSZIkSZIUYGAkSZIkSZKkAAMjSZIkSZIkBRgYSZIkSZIkKcDASJIkSZIkSQEGRpIkSZIkSQowMJIkSZIkSVKAgZEkSZIkSZICDIwkSZIkSZIUYGAkSZIkSZKkAAMjSZIkSZIkBZyW6gbaQpdQh/gakiRJkiRJJwQnjCRJkiRJkhRgYCRJkiRJkqQAAyNJkiRJkiQFGBhJkiRJkiQpwMBIkiRJkiRJAQZGkiRJkiRJCjAwkiRJkiRJUoCBkSRJkiRJkgIMjCRJkiRJkhRgYCRJkiRJkqQAAyNJkiRJkiQFGBhJkiRJkiQpwMBIkiRJkiRJAQZGkiRJkiRJCjAwkiRJkiRJUoCBkSRJkiRJkgIMjCRJkiRJkhRgYCRJkiRJkqQAA6OWbCimoGAEk1dEU92JJEmSJElSuzMwkiRJkiRJUoCBkSRJkiRJkgIMjCRJkiRJkhRgYCRJkiRJkqQAAyNJkiRJkiQFGBilRIzqp2bzwHemUlaX6l4OdCL3JkmSJEmS2sNp7VGk4ZPNPP+HDbz+4S4a9kLGoDu5Oas9Kp+otlC5qIJqsshJdSsHOZF7kyRJkiRJ7SF5gdHeera+WUXlhs3U7mhIWhlJkiRJkiS1reRtSXv3ef67YlNjWNStJwPy+5GRtGKSJEmSJElqK8mbMLq4N33O2knDFYO4ttfZdGIztUkrdhLYUEzBtNIDFiMUjxtB8QGreZOfoLDvAYsNUaqeLGbxM69SE0tAKI30rMGMHDuc3Iy0lmse+Awh0i68krxb7mDY13qS1qmNepMkSZIkSR1KEs8wuoBrvzksea//IolXU3zvg5TVJfavJeJEN5bwyIQ1vDH5ZxT2TTv4mYlTKfu4+WKC+HvrWTZ7PfHuBj+SJEmSJKll7XLotYC+hSxZUtj0RzXFBVMpI4vCOdPJ63G4B+NUzmsMi8IDCpk4Np+sMJCIEVk1j5nzKyn78TyyfzWenGaZUe0z8yj7GMLXjWfK2FwyujSuJ+pqqCxfTLRTsxLH3ZskSZIkSeqIDIwOIzJ/HAXzD1ztyVfYwmvH9KZWhC+15Sxem4ALRzFlQj4Z+4KeUJisoROZtGMc9y2toPzlseRctz8x2hZt3ADYu1+/z8MigFCPTHJvnXQcjUiSJEmSpC+K5B16rTYR2/gitUDm4Nz9YVEzmX36A1AVqQms98zqTwio/MVUfrk6Qixx8LOSJEmSJEktccLoMLLGzGHa0PSU9rCltjEIqlkwjoIFR/9c+Lq7mPLuVKYur6F87o8onxsifEUO+TfezOCvZRJuIXySJEmSJEkCJ4xOfA3HOxqURtbtD/GbOVMovKk/Gd0htrGCxbPv5TtjplLyZrxN25QkSZIkSR2HE0YnuPSMLCBy/NNOPbLJuzObvDsh8WE1pY/N5tF11Tw6cznZc0eR2eYdS5IkSZKkk50TRqkSAogR33P429Iv600aEHm5mlhrS56fzbAJRQwG+LiayMet602SJEmSJHVMBkYpkUnvbIAoS3+9jMjhTqS+ZDDDLgQ2zOXemSVU18ZJNDRdS8Sp3VzO4h/Ppryu+UNRymfPYPHqCLU7m727IUHti2tYD9C9N5nntLI3SZIkSZLUIZ2ybcfOPyfn1ZtZPv95ao90218MpCivV6sqdQm18c66DcUUTCtN6qHXiY3FjJta2uLUUN7kJyjs22zhw1Im31NM5JDZTRaFc6aT12Pf31HKfjSO4s2Huj9M3uSfUdg3rfW9SZIkSZKkDscJoxQJXVHIrBlFDL4inbTQEW4+P59pv5zFXcP7kxnef3MonE72oFHc9dAPmoVFAOnk3d142HVmjzT2PRHqnk72DYXcP+fXhwyLjrk3SZIkSZLU4SRxwqj9nIwTRpIkSZIkSScqJ4wkSZIkSZIUYGAkSZIkSZKkAAMjSZIkSZIkBRgYSZIkSZIkKcDASJIkSZIkSQH+SpokSZIkSZICnDCSJEmSJElSgIGRJEmSJEmSAgyMJEmSJEmSFGBgJEmSJEmSpAADI0mSJEmSJAUYGEmSJEmSJCnAwEiSJEmSJEkBBkaSJEmSJEkKMDCSJEmSJElSgIFRSzYUU1Awgskroqnu5GAncm+SJEmSJKlDMDCSJEmSJElSgIGRJEmSJEmSAgyMJEmSJEmSFGBgJEmSJEmSpAADI0mSJEmSJAUYGEmSJEmSJCngtKRX2PYWa9Zt4PUPt9OwFzi1E2edfwUDBvXjkm5Jry5JkiRJkqRjlNQJox2bnuE/lq+h+v2msAhgbwPb3t9A2dL/4vm6ZFaXJEmSJEnS8UjehFF8I2WVW/hsb2fO7TOQvKsv4oxQA7u3bqR0VRVb67fzyh82cvU3ruBLSWtCkiRJkiRJxyp5gVHaFdzU/0NWf2kgN2Wd3rTYia7n9eWmvlv4j5fqYGstb3EFfZLWhI5aXSmTxxUTGTKFJWMzqVm9kEcXraE6loAuYbJvKOK7t/cnvVMLzzZEqXqymMXPvEpNLAGhNNKzBjNy7HByM9JarvdxNWVPLmXp2mpie1q+JW/yExT2bbNvKEmSJEmSjlJSzzDqeuUN3NTC+pcuOIczqGNHMovr+NRUsnjmbJati+1f2xOj+ukZTIhN5FfjcwhEQPFqiu99kLK6xP61RJzoxhIembCGNyb/jMK+B4RGH5Yy+Z5iIgkkSZIkSdIJKPmHXrfkk+2NYVGos9vRTjSbS1kGhAcUMWX8YDJCEF07m8mzK4itXU7F7TnknbPv5jiV8xrDovCAQiaOzScrDCRiRFbNY+b8Ssp+PI/sX40n5/PMKEb5vMawKHzdeKaMzSWjC7BnCxXzpvLIczFC/cYzZ1Iu4RR8fUmSJEmSlJLA6FNeiXwAQNfM3lzS/g10CJH54yiYf+BqT77CFl47pjdlUThnOnnNVsJDpjBrbPbnk0Tpg0Yx/HcVFG+OsKUW2BcY1ZazeG0CLhzFlAn5ZOzbrhYKkzV0IpN2jOO+pRWUvzyWnOua3razmsoNAIMpKsrd/0yXnuQWFVH93AzKq17mjT255HQ5pi8iSZIkSZLaSFJ/Ja0lW1/8XeOvo3XuycC/uqC9y+uIBlM0Jju47Yx0MjIPvjO28UVqgczBzYKfZjL79AegKlJzdKUTCeqPsVtJkiRJktT22nXCaMemlTz9+i44tRvZQ26gV+f2rN6xZI2Zw7Sh6W33wrp9HzoTaulg6xZsqW0MgmoWjKNgwVHW6X4x2RdC1XvlzJ175f4tabEayh+bRwVA3yvp7XSRJEmSJEkp026B0bZXf8dTL9Xx2and6DP4G1zbo70qK2kajufU6p4MuzOfZdNKiT03mwnPzQ5eDmVROHaw5xdJkiRJkpRC7RIYba0q4ek/fsJnp3Yje8g3ufaCoxxh0QktPSMLiBzbtFPDFpYtKCV+Tg7DroqzZm01sT1AlzDZg0Zxx22DyeyezK4lSZIkSdKRJDkwaqD2hf9i+aZdcOqZfPUb32Sgk0UdRvplvUkjQuTlamJDj3IqaGMZS9+DzDtHc8dN6dzxD8nuUpIkSZIkHaskHnr9KZtXL20MizqfzdWGRR3PJYMZdiGwYS73ziyhujZOoqHpWiJO7eZyFv94NuV1+x+Jx6IkgC2VFUTq4sR3Hs+2NkmSJEmSlEzJmzB683lWvf1p4+f6T3j5twt4uaX7ul3OiBE5nJu0RpQ0nXpyyw8KWX9PMZF1C3lg3cIWbsqi8O/2/5V29WByu6ynYtMiJo9bdNDdoXAm2TeMpPBb/Ul356IkSZIkSSmRvAmjhoYj36OT3/n5TPvlLO4a3p/McOjz5VA4nexBo7jroR+Q13yyLC2bYUMzG+/pnk76+emkd9//XCJWQ9XSGUyYWkrUf4QkSZIkSUqJ5E0YZQ2hKCtpb1db65HPtCX5h7ycPfYJlow9xMXuPcm9dRK5tx65THTlg0x+qobQ5aN5aMowMg6YIoptXsTMHy0jsmkxZW/lc0evo/8KkiRJkiSpbSTxDCPpQFGqfh8hAeT87cFhEUD4kt70BKCe+vr27U6SJEmSJDUyMFI76gxNIVHl48VUNT8kG0jUVlMyay7lAN1vIOfSVPQoSZIkSZKStyVNOkiY3G/ls3RaKbG3S5kxobTl20KZDPvBSLK7tG93kiRJkiSpkYGR2lVa30J+NSeHsmVLKdlQw7a6OAkAQoTP70329Tcw/MZcMrqnuFFJkiRJkr7ADIzU/npkk/cP2eSlug9JkiRJktSiU7bt2PnnVDfRWl1C5l6SJEmSJEltxUOvJUmSJEmSFGBgJEmSJEmSpAADI0mSJEmSJAUYGEmSJEmSJCnAwEiSJEmSJEkBBkaSJEmSJEkKMDCSJEmSJElSgIGRJEmSJEmSAgyMJEmSJEmSFGBg1JINxRQUjGDyimiqOzmYvUmSJEmSpCQzMJIkSZIkSVKAgZEkSZIkSZICDIwkSZIkSZIUYGAkSZIkSZKkAAMjSZIkSZIkBRgYSamyZwsVC2YwobCY6o5QR5IkSZLUYZyW1LfX11H9hz9Q/fYnbKtvaFwLnc65F17OwKv7ktEtqdXV0TREqXqymIUrX6V2ZwJCaaRn5XDL7aMZfGlam5WJ11ZQ8tiTlG2oJZ6AUPcMsoeMpvBb/Unv1GZlYGc1pU+vp5b8NnxpCutIkiRJkjqMpE4Yvb76d6yJ1O0PiwASn/LRW1UsX/pfrPsomdXVocSrKf7eOGYsXd8YFgEk4kQ3lvPLSXfz8HPRNilTu2Iq35kwm2UvN4ZFAImdtVQtncGEu4qpjrdJGUmSJEmSTmhJnTA6rXM3Mvr0Jeerl3De6Z2ABnZv3czq31fybnw7f3z1HQZcf1EyW1CHEKPikQcp+xhCF+cz8fuj6Xd+CBIxIitmM/Wxair/fTZlfaaT16MVZd5exoz51SRCmeR9fyKjr0onRILYWyXM/JdFROpKKX4mj1k392yzbyZJkiRJ0okoqYFRr+uH0yuw0omu513OTX3fZe4LH9BQX5/M8uooNi2nuCoB3QczcUoh/fbtPguFybp5ClN2/D2Tl0dYXBYh7/as469z8S1Mun0r1QOKyDt/32KIcK9bKLrlGSY8HqW2sprozT1JP84S0RU/4nvzIweslvJAQekBa1kUztkXgMWpnP33PLw2QejyQmZNyT9ga1ycqrnjmLE63nj9gXw4rjqSJEmSJDVKyaHXu+s/A6BT586pKK+TTOSlcuJAxt8O2x8W7fPxetZsaNwnFl9XTW0ra2Xc3Dwsarbeq38r39waaeSM/SF550Bi00IeWRncfhd/YR4zV8fhnHwm3Zd/3GGWJEmSJEn7JPfQ6wMlPmXr2y+xuuoT6HwB137N7Wg6khg1kTiQRv8+wa1g8Y0LeeDBEmqazhriw61EGyCjLQ+mbhKtfavxQ7dutOZ47fSh01kytOmPulImjysmQj73Lykk+3APpmUzsiifymmlRB6bTVm/6Y3BVryS4p9XkCBMXtFIstNaWUeSJEmSJNojMPqokt+UbGJHs6Uv9ejLzTf0I+P0pFdXCkTmj6Ng/oGrx7v1qZ76PQAZpIeblhriVD85lRlLa0iE0hk8YRRp82ZTsnML0U+Att5e1bCFilURIETuDTmtCoxaI61vIVNufZUJj0co/rcS+j1wLZFHZlORgKw7p1PYN1WdSZIkSZI6mpRsSfusbgPL/3slr2xLRXWdXKJE3272584aSmb8PQ8srYEeOXz3oTl895os0lvYRtY24lTPn8ri9yB0+WhGDgglq9BRyfjmDym8PASbFzJj0oPMrWo81+iufDeiSZIkSZLaTvInjM7N4dtjcho/761nd907rFv3Eq9/9AHP/+4Zuo64gV4eZdShZI2Zw7ShbRVghDnrfOBD2BYp4YF/X0j1HggPKGLKhMFN28/qie9so3IHqH36QWasjJ045wN1Sifvn0az5u5iIm/XQCiXe+478BBsSZIkSZJap30njE7tTNfzevH1Yd/g6jOB+i28vHFXu7agk01n0roDRFg2eyHVe8LkjHmIWRMH7z+rqCFK9EOAnqSf3VZ149Qs/RH3LoiQOCeXe6YVfn4+UMo1JGDfuU2JaiLvp7QbSZIkSVIHlJItaXAmGec1HmDU8Fl9alrQSSKdjMuatoF1yeaOGT/jnqGZwXOENldTCdArs40OvI4TeWwqkx+PkDgnn0kPjSfnnLZ4bxuIV1M8dSERwuQNzydMjJKfFlMdT3VjkiRJkqSOJEWB0XZqt34KdOLc89tsJEQdVO8rcggBXJhDzqUHjvnEqSwrJQ5k5GS3fstYQ5yqeXczeXkN9Mjn/pmF9Ot+lM/GIyyedBsFBbdx3+MRjpjhfP5v3y7ie46qAFULZ1L2MYRv/gGFt46k6Po0+LiUhxeuP3S9Y64jSZIkSfqia9/AKPEpO7Zu5tmS3/HyduCMy7n64nbtQCeh0NXDGX4hsHkhD8wtpybWdCERpWr+VB5Zm4Dugxl9Q8/WFWqIUjHrbmasjBG6eBhTHjq2bWjR5xaw7M0EkKBm6S8orz3CA+dcQu/uABUUz6ug9ghhTnTFg8xcHYdzhjFxVBaQRr/RReSGIL56JjNWRtukjiRJkiRJSTz0ejPL5z/PIf8/c7dLuPEbf8m5yWtAHUWnntzyg0LW31NMZPVc7ls9N3g9lMmwH4ymXyvPGIo9M5tH1jWmUYm3S5g8pqTlG4dMYcnY7IOWE4nmMz4JEnuPVDGLvFuzKJ0XIfbcbCY8NztwrXDOdPJ6NP35YQmPzI+QIMyw748ma9/Wu7QcCr+XS+WsCiILZlPWdzp5B/1i3DHUkSRJkiSJ9p4w6tSZrmf3JDt3GN8ZcS29Tm/X6jqZnZ/PtF8+xHdvyCbcpWmtS5jMQaO5/2cPccflrT+Rur6hdc9n3PCP3HJpCAiROXw8eRce+Zn0IdOZdd8t9Ls0TOhQN31+blHjVrQ7egUvp11zJ0X9QpCIUPyTZdS28D2Oqo4kSZIkSU1O2bZj559T3URrdQm18aDUhmIKppW28c/DtxF7kyRJkiRJSZaiQ68lSZIkSZJ0ojIwkiRJkiRJUkASD72WUqCulMnjiokc00P53L+kkIOPsZYkSZIk6YvJCSNJkiRJkiQFOGGkjqVHPtOW5Ke6C0mSJEmSTmr+SpokSZIkSZIC3JImSZIkSZKkAAMjSZIkSZIkBRgYSZIkSZIkKcDASJIkSZIkSQEGRpIkSZIkSQowMJIkSZIkSVKAgZEkSZIkSZICDIwkSZIkSZIUYGDUkg3FFBSMYPKKaKo7OZi9SZIkSZKkJDMwkiRJkiRJUoCBkSRJkiRJkgIMjCRJkiRJkhRgYCRJkiRJkqQAAyNJkiRJkiQFGBhJqbJnCxULZjChsJjqjlBHkiRJktRhnNaexT578xl+89wWPgPgAr4+Zgh92rMBnfzqqilbtojFz0WIJyBv8hMU9m3D98cilP/3UsrWvkpNLAGESOvRm5z/MZxbhmST3qkNa+2spvTp9dSS34YvTWEdSZIkSVKH0X6B0d53ePbFfWGRdAwaEkQ3PsOyJxezZlOcRNIKRSn7yY8o3tx8LUG8rpry+dWs+e0wJj08muwuSWtAkiRJkqQTQrsFRm/9/nneqoczLriA//fBB+xur8I6+b21iPumlRAHQpcO465/vITqe2ZTnoRSofT+3PKNkeQPyCQcAhoSxN56huKfFlNZV0LJuuFkX5eWhMqSJEmSJJ042icwevtZnn27Hrpdzo1Xw6rfftAuZdVB9BrMLf2icNNohvVNB6p5IymF0hk8flJwqVOIcK987vgf5VQuqCEejwPHHxhFV/yI782PHLBaygMFpQesZVE4Zzp5PQDiVM7+ex5emyB0eSGzpuQfsDUuTtXcccxYHW+8/kA+HFcdSZIkSZIaJf/Q670f8OyL7/AZ3cj+eg7nnZL0iupwejJs0sSmsCg14rviAKSlpWK6KI2csT8k7xxIbFrIIyujwd5emMfM1XE4J59J9+WTuv+VJEmSJEkdRdInjGqfX8Prn8IZl1/HtT2Aj5JdUWpDiRg1FYt4eHmU0OWFFA5qXWCUPnQ6S4Y2/VFXyuRxxUTI5/4lhWQf7sG0bEYW5VM5rZTIY7Mp6zedvPOBeCXFP68gQZi8opFkp7WyjiRJkiRJJDswev95Vm3+tHErWo57Xr4oIvPHUTD/wNWTaOvThmIKpjXfupVG1s2TmDaqP+G2/JW0Y5TWt5Apt77KhMcjFP9bCf0euJbII7OpSEDWndMp7OvZSpIkSZKktpG8LWl761izdjO7921FS/7mNylJ4kSWz2TCQyXUxFPbScY3f0jh5SHYvJAZkx5kblXjuUZ35bsRTZIkSZLUdpI2YbS18jmqdzXbiqYvjKwxc5g29CQOMPoWsmRJYePnhgTR9yop+cUvKKtayOQfh5j1QArPCeqUTt4/jWbN3cVE3q6BUC733HfgIdiSJEmSJLVOcuZ+6ipZFdnlVjSd/DqFSL84l8IZ/8Kw7pDYtJiyzSnuqSEBiabPiWoi76e0G0mSJElSB5SUCaOP3tzCjr3Ark0sW7DpEHd9wLPzF/AscEafYXz7a2cnoxWpbXTKIrsflDwXp/7TFPYRr6Z46kIihMkb/jUql5ZS8tNi+s0s/PzAa0mSJEmSWsuThaSj0RChugqgP70zD3FPPMLiSbdRUHAb9z0e4YjHHX3+b98u4nuOpok4VQtnUvYxhG/+AYW3jqTo+jT4uJSHF64/dL1jriNJkiRJ+qJLyoTRuV8bTtHXDnHxo0p+U7KJHVzA18cMoU8yGpDaSiJOtLaKsn//BSU7ITx0GDndW741+twClr3ZuFesZukvKM/9GcMyDvPucy6hd3eI7KygeN7V9BybS0aXQ98eXfEgM1fH4ZxhTByVBUC/0UXkVsykYvVMZlz6M6YNaeF0pWOsI0mSJElS0g69ltpONcUFUylr4UrZtBH714dMYcnY7OMvU1fK5HHFRFq8GCL9mvHcPzqb0CEeTySaz/gkSOw9UsEs8m7NonRehNhzs5nw3OzAtcI508nbdwTYhyU8Mj9CgjDDvj+arH2HXKflUPi9XCpnVRBZMJuyvtPJO78VdSRJkiRJwi1p0mGFwulkDxrFXQ//mp9PyD3sr5Fl3E/PbzsAACAASURBVPCP3HJpCAiROXw8eRce+f3pQ6Yz675b6Hdp+JBB1P5zixq3ot3RK3g57Zo7KeoXgkSE4p8so7bhOOtIkiRJktTklG07dv451U20VpdQGw9KbSimYFrpifnz8PYmSZIkSZKSzAkjSZIkSZIkBRgYSZIkSZIkKcBDr9WxHPbg6kPJ5/4lhbTiuGxJkiRJkjoUJ4wkSZIkSZIU4ISROpYe+Uxbkp/qLiRJkiRJOqn5K2mSJEmSJEkKcEuaJEmSJEmSAgyMJEmSJEmSFGBgJEmSJEmSpAADI0mSJEmSJAUYGEmSJEmSJCnAwEiSJEmSJEkBBkaSJEmSJEkKMDCSJEmSJElSgIGRJEmSJEmSAgyMWrKhmIKCEUxeEU11JwezN0mSJEmSlGQGRpIkSZIkSQowMJIkSZIkSVKAgZEkSZIkSZICDIwkSZIkSZIUYGAkSZIkSZKkAAMjSZIkSZIkBZyWzJe/XraAZ98/zA3dLmfEiBzOTWYTkiRJkiRJOiZOGEmSJEmSJCkgqRNG+2QMupObs9qjkiRJkiRJklrLCSNJkiRJkiQFGBhJkiRJkiQpwMBIkiRJkiRJAe1yhlHt2gXMXdv4uVPnzpxx5kX0urovV1/QrT3Kq51F5o+jYP6Bq1kUzplOXo9UdCRJkiRJko5Fu08YNdTXs61uM+tKl/KbtR/wWXs3IEmSJEmSpMNK6oRRn7w76dN8YW89u+veYd26l3j9o3p2RJ7l2YxR5F2czC7UbvoWsmRJYaq7kCRJkiRJrdS+E0andqbreb34+rCv0+d0gHo++vCTdm1BkiRJkiRJh5eiQ68v4NxwaipLkiRJkiTp8FITGH26ibc+avx4encPvtYxaIhSMWsc3y64je/NqiDacBTPxCMsnnQbBQW3cd/jEeJHU+d4npEkSZIkqYNo38Co/lO2bn6eJ/6rktoEcMblDOrTuV1b0EluYwlzX4iSIEH0hV+w+OXEER+JPreAZW8mgAQ1S39Bee2RyxzPM5IkSZIkdRRJPPT6E9Y8UUL1rkNcPrMXNw3N4bwUbYrTySmxp/k/UAk4igmjRKL5fFCCxN7kPCNJkiRJUkfRvnFNp850PfsSBvzNcL5zy0C+fHq7VlcHELp6FEXXpBMiRPo14xk5IHTEZzJu+EduuTQEhMgcPp68C49c53iekSRJkiSpo0jihNHZXDviTq5NXgF9EXVKJ3fCHHInHMMzaVmMnPGfjDyWOsfzjCRJkiRJHYQbwiRJkiRJkhRgYCRJkiRJkqQAAyNJkiRJkiQFGBhJkiRJkiQpwMBIkiRJkiRJAads27Hzz6luorW6hJL4Y2+SJEmSJElfME4YSZIkSZIkKcDASJIkSZIkSQEGRpIkSZIkSQowMJIkSZIkSVKAgZEkSZIkSZICDIwkSZIkSZIUYGAkSZIkSZKkAAMjSZIkSZIkBRgYSZIkSZIkKcDAqCUbiikoGMHkFdFUd3KwE7k3SZIkSZLUIRgYSZIkSZIkKcDASJIkSZIkSQEGRpIkSZIkSQowMJIkSZIkSVKAgZEkSZIkSZICDIwkSZIkSZIUcFr7lGngo80vUfnaO2zd+SmfJZqWu13OiBE5nNs+TUiSJEmSJOkoJD8w2vUWq1asYfOupFeSJEmSJElSG0huYFT/Fk//9xrerQe6XUD2Vf34as+zOeP0TkktK0mSJEmSpOOX1MCodt1LvFsPX/qLgYzM60XXZBZT69SVMnlcMZHrJ7GkqDe1a5dS/J/lvFEXJxFKI+u6sRSNzSWjpaxvZw3l/7mQZeveILozAaE00rNyGPatUeRdEW65XkOUqieLWfzMq9TEEkCItAuvJO+WOxj2tZ6kmSlKkiRJkpQyyQuM9m7m5Tc/hdBFXPs3hkUnjS1vUDZvLsUrY/vXEnEiz8zm3tpdzHogn/Rmt8c3LeSB/1NCTYLA/dGN5RRvXEP5zf/C/bdnkda8Rrya4olTKfu4+WKC+HvrWTZ7PfHuT1DYNxlfTpIkSZIkHY3kBUbv1vLhXiDjy/Tyt9hOHpuXUbwZwgMKmTg2n6wwxN9cxoz7FxHZtJDF627grgGhxnvj63n0JyXUJEJkDp3IXaP6k9EFSMSIrJrHzPmV1Cz/Cctyfs0dvfaXqH1mHmUfQ/i68UwZm9v4DJCoq6GyfDFRp4skSZIkSUqppAVG2z76mAbgjNMbeP3FZ6h+9wM+ijc0XuzUmbP+4gquH9SX805PVgcdW2T+OArmH7jak6+whdeO6U1ZFM6ZTl6zlfCQKcwam/35VFDapbcwceyrfGduNZUb3oAB2QDUPrOQ8p0QGjSe+8f03z9FFAqTNXQiU3bfzYTHayktq2RkrxyaYia2RWsB6N2v3+dhEUCoRya5t046pu4lSZIkSVLbS9rsT8P/a/zvHZHnefb1LfvDIoCGera9V8WyZSt5/dNkdaDjk89dzcKifcKXXUk6kKjZQhSAGNUv1QJp5OflHHQ/QMagG8gEElVvUNNsvWdWf0JA5S+m8svVEWKJFh6WJEmSJEkpk9xfSQM4tRsZvfuSc+VFnJfWGWhgx5YNrPr9BrbWf8CaF9+hz/UXJb2NDqNvIUuWFLb9e+uOcL1LV8JA9MMoMSCdKI2DQhmkn3OIZ87PJAuo2Zmgvtly+Lq7mPLuVKYur6F87o8onxsifEUO+TfezOCvZRJ2S5okSZIkSSmV9NOFzuh9PTd/rVdTWATQiTN69uOWay6iE9Dw0Yd8lOwm1HbOOYvG3z2rp37P8b4kjazbH+I3c6ZQeFN/MrpDbGMFi2ffy3fGTKXkzXibtStJkiRJko5d0gKj7mmNhxPFP93Z8g1nnN64jemzT9mRrCbUdj6obdxWdtZZTdvP0ul5MUAt0dghnvmwhgjA+WdxVkvXe2STd+ckZhX/J795ZAp3DAjDnmoenbk8sIVNkiRJkiS1r6QFRl+64BzOABo+fJd39x58ffcHHzcGRd3O5NxkNaE2Eqdy9TMkgKyrsj4PjDL7pAFxyletp6WZoMiqxuAn7epsMo5QIXR+NsMmFDEY4ONqIh+3Zf+SJEmSJOlYJG9L2rmX0+tMYM9brCqt4t0dTYde761n6+ZneWp9HdCJjK9cwRlJa0LHbgu1tXES+84o37OFqvlTeWRtArrnM/L69M/vzLp+OBlAfPVMHpi/ntp9W9QSUaofn8rU5TEIZTHyG1nN3h+lfPYMFq+OULuz2WnXDQlqX1zDeoDuvck81LlIkiRJkiQp6ZJ46PWZDLj2cjb/bhM7tm7g6aUbDrqj68UDuTGrcwvPKnWqKZ7wvyg+cDmUxcgfjiS7S7O1jGFMGrueCfOqqVkxgwkrDnwmnbz7fkhej+ByIrqeZXPXs6zF+mHyxg8nq8VrkiRJkiSpPST3V9J65PDt4eezbm0Vf/xwOw17gVM78aXu6WRfPZABF3VLankdjxChUIJE0/BPKJxJ9g03M/obuWR0P/ju9CFT+NWl5Tz62FIqI1HiCaBLmOxBw7nllnyyexz0BHl3T4HflVC+7g221MVJAKHu6fQeMOwQz0iSJEmSpPaU3MAIoNtFDMi7iAFJL6S2cQOTflNI9jE8kXbpYL47ZTDfPdoHemSTd2c2eXceR3uSJEmSJCnpkneGkSRJkiRJkk5KBkaSJEmSJEkKMDCSJEmSJElSgIGRJEmSJEmSApJ/6LVODj3ymbYkP9VdSJIkSZKkE8Ap23bs/HOqm2itLiFzL0mSJEmSpLbiljRJkiRJkiQFGBhJkiRJkiQpwMBIkiRJkiRJAQZGkiRJkiRJCjAwkiRJkiRJUoCBkSRJkiRJkgIMjCRJkiRJkhRgYCRJkiRJkqQAAyNJkiRJkiQFGBi1ZEMxBQUjmLwimupOJEmSJEmS2p2BkSRJkiRJkgIMjCRJkiRJkhRgYCRJkiRJkqQAAyNJkiRJkiQFGBipBQkiS+/l2wUj+PakZUT2pLofSZIkSZLUngyMUiJG9VOzeeA7UymrS3UvLXmDNY/XkAASby5iTSTV/UiSJEmSpPZ0WrJe/HrZAp59/yhu7HY5I0bkcG6yGjkhbaFyUQXVZJGT6lZa1Jtrb82k/PEauHQU12aluh9JkiRJktSekhYY6WQWImv4Q/xmeKr7kCRJkiRJqZC0wKhP3p30Ocz1t1Yvouztes64sNcXbLpIkiRJkiTpxJaaCaPtVax7tx5CF9C//9kpaaHdbSimYFrpAYsRiseNoPiA1bzJT1DYd//f1fNG8MDKLArnTCfv7Dg1K+fxyJOV1O5MEApnk180njv6hYMvaUgQ3fgMZSvKqYxsIbozAYQIX3olg79VyMir0w9qsbHOgav53L+kkOyWvlNdKZPHFRMZMoUlYzOpWb2QRxetoTqWgC5hsm8o4ru39ye909H8DyRJkiRJkk4UKTn0evPLG9m2F8647C/p0zkVHZyMatiyZQslU/+e++ZXULszAUAiVk3JjB9QvDERuDu68l/43rRiSl6uaQqLABLE3lzPsh/fzfceb8OTrGsqWTzzbu6bW94YFgHsiVH99Awm/LySeNtVkiRJkiRJ7aD9J4y2V/HyOw1wag+yvyjTRQB9C1mypLDpj2qKC6ZSRtPUUI+jeUGC8pn3kkhA5k2TuOe2/qSfGqdq/t3MWBmjbHklI6/IJW3f7Z3Oot9NhQy78Vp6n59GqBOQiBFZ/hOmPh4hunQpFd+YRG73/RWyxz7BkrFNf+ybHjqa1jaXsgwIDyhiyvjBZIQgunY2k2dXEFu7nIrbc8g752heJEmSJEmSTgTtHBjV8/qLG9kGdL2sH189waeLIvPHUTD/wNWefIUtvHZMbzqWYOjQEgnIHvsz7h+ybztZGv3+djhZK4uJVL9BDbmfbx1LHzKRSQe+IBQma/h4RlaO49G3X+WNdyH3itb1tE94yBRmjc3+PLBKHzSK4b+roHhzhC21gIGRJEmSJEknjfbdkvZRFevfb5wuuuqvLmjX0h1B+vDpzcKiJj16kgmQaOmJlnUONT3Q0FadDaZozP6wqFE6GZlt9X5JkiRJktSe2nHCqJ7XX97MDk6O6SKArDFzmDb04MOhUyV8RloLq9kULnmCwhauRDeUsuypEta/GSW2J5mddW7c8iZJkiRJkjqE9guMPniJdU4XtZM41fPu5oGVsVQ3IkmSJEmSTkLtFBjV80rVZnZz8kwXndRqyyleGQPC5Iz5AYU3ZhEO7bsYpexH4yjenML+JEmSJEnSCa19zjB6/yX+uBWni5oLAcSIJ2GrWGzji9QCXHwzdwxtHhYBDfXUH8N5R5IkSZIk6YunHQKjT3llvdNFQZn0zgaIsvTXy4jE2jbBCffo2fih9kUq3ow3fm5IULuhhIfvupdH327TcpIkSZIkqYNJ/pa096v4Yx2N00X9nS5qlEbOzfmEq0qJbVrE5O8sClzNm/wEhX1b8fo+15J3TjllH0dYPOl/sbj5tVA6mRfHqXk7fsBD1RQXTKXsoJeV8kBB6ed/nWgHgUuSJEmSpLaX5Amj7axb1zRddGk/vnp6cqudTEJXFDJrRhGDr0gnLXTk+49Jl2wKH5rCyKszPn93qHs62TcUMW3OHO65PqONC0qSJEmSpI7klG07dv451U20VpdQGw9KbSimYFqp0zSSJEmSJOkLqX0OvZYkSZIkSdJJw8BIkiRJkiRJAQZGkiRJkiRJCjAwkiRJkiRJUoCBkSRJkiRJkgL8lTRJkiRJkiQFOGEkSZIkSZKkAAMjSZIkSZIkBRgYSZIkSZIkKcDASJIkSZIkSQEGRpIkSZIkSQowMJIkSZIkSVKAgZEkSZIkSZICDIwkSZIkSZIUYGAkSZIkSZKkAAOjlmwopqBgBJNXRFPdSdvoaN9HkiRJkiQllYGRJEmSJEmSAgyMJEmSJEmSFGBgJEmSJEmSpAADI0mSJEmSJAUYGEmSJEmSJCnAwEiSJEmSJEkBpyW7wI4PNrBu3Sbein1Kw97GtU6dz+T8S69g4NW9ODeU7A4kSZIkSZJ0LJIaGG19cSnLXt910HpD/XZqX3+eJ96t46ZvDeTLzjlJkiRJkiSdMJIXGO2qYvXru4DOnHf1jeT36UHXELC3nm3vVlFWsYlt8c28tPEv+fKVnZPWhiRJkiRJko5N8gKj9+vYBnDuFeT37UHXfeunduasi3O4dstbLN9cz6fxXcDZSWtDSVRXyuRxxUSGTGHJ2ExqVi/k0UVrqI4loEuY7BuK+O7t/Unv1MKzDVGqnixm8TOvUhNLQCiN9KzBjBw7nNyMtJbrfVxN2ZNLWbq2mtielm/Jm/wEhX3b7BtKkiRJkvSFlLzA6OwzOYMP2LH9Q97a1Zfsbs2u7a3jrQ/qgU6ce75h0UmvppLFM2ezbF1s/9qeGNVPz2BCbCK/Gp9DIAKKV1N874OU1SX2ryXiRDeW8MiENbwx+WcU9j0gNPqwlMn3FBNJIEmSJEmSkix5gdG5/Rhw4Vuseu8D1ixdxOuX/SXX/dVFnB7dxJoXqnh3F3zpL3K49uKkdaD2srmUZUB4QBFTxg8mIwTRtbOZPLuC2NrlVNyeQ945+26OUzmvMSwKDyhk4th8ssJAIkZk1Txmzq+k7MfzyP7VeHI+z4xilM9rDIvC141nythcMroAe7ZQMW8qjzwXI9RvPHMm5RJOwdeXJEmSJKmjSeKh153pdeM3+dKLq1j1xid8FHmeZZHnGy+d2o2MqwdyU98LaGm3kpIjMn8cBfMPXO3JV9jCa8f0piwK50wnr9lKeMgUZo3N/nySKH3QKIb/roLizRG21AL7AqPachavTcCFo5gyIZ+Mff8AhMJkDZ3IpB3juG9pBeUvjyXnuqa37aymcgPAYIqKcvc/06UnuUVFVD83g/Kql3ljTy45XY7pi0iSJEmSpBYk+ffJTid8QTrdDzzTeu8udka3s2NvcqurvQymaEx2cNsZ6WRkHnxnbOOL1AKZg5sFP81k9ukPQFWk5uhKJxLUH2O3kiRJkiTp8JI4YQRbX/ovlr26HejMuX0Gkte/O+8+t5o17+1ix3uVLH5qO7f8zxzOS3JspUZZY+YwbWh6272wbt+HzoSOclRsS21jEFSzYBwFC46yTveLyb4Qqt4rZ+7cK/dvSYvVUP7YPCoA+l5Jb6eLJEmSJElqE8kLjD54ntKmsChj0De5Oet0ALJvHM4l7z3PU+Wb2bFjE6UvfJk7B12QtDZ0gmk4nlOrezLsznyWTSsl9txsJjw3O3g5lEXh2MGeXyRJkiRJUhtJWmBU++Y77AY463K+3hQW7dP1woFce/E7PP1WPbs/rGUbF3BWshrRCSU9IwuIHNu0U8MWli0oJX5ODsOuirNmbTWxPUCXMNmDRnHHbYPJ7J7MriVJkiRJ+mJJWmC0e0/TyTJdTueMFq53/VJn8PSZL5z0y3qTRuT/t3e/0VVX957H3yU9gAQuBzWn0ERroBzlcuQW6BCV4GhwSGhXuC0UireC44RpyzALypTqwrKkUJVV612oS0v/kGEutoL0kttLbivpQOiVQA23JLfx0EIyEv9wlB5GCIajNGdC50EC5CQB+XcA4f16RPZv79/ev7OWD/ysvb+b+p1Rmiae4a6gXRWsfwty75/JjM+HmPG1dK9SkiRJkqSrW9qqB904KKvtH3+qZcPOfRw6ng0da+HQ69Vsee0IAH0GZru76GoyuIDiG4C6FTz4ZDnRWIJka/uzZIJYQyVrv/c0lQdODkk0xUkC+6qrqD+QINF8LsfaJEmSJEnSmUrbDqNew/4Df7PnV/z+cAuxus2sreumU//hFN1u/aKrSkYOk79VQs03S6nfsZqlO1Z30ylMyX85+Vfm6ALye9dQtXsNi+as6dI7EMwlMn46JV8aRegMi29LkiRJkqRTS9/9ZD2yuOMLU5g0ejDXZ/Yk4/hMPTLo1T+LYaMn8JUvfNYb0q5GA4t49IfLmTtlFLnBwInmQDBEZOy9zH3iWxRmdeifGaF4Ym5bn34hQgNDhPqdHJdsaqR2/TLmL9lIvBVJkiRJknSe0ndLGkCPvmSPGMfUEWmdRZdKVhGPris65ePIrJ+zbtYpHvbLIf/LC8n/8odPE//14yz6p0YCt8zkicXFZHfaRdTUsIYnv11G/e61VOwtYsbQM/8ESZIkSZLUlft7dJmLU/uv9SSBvL/tGhYBBAffTA4ALbRYR12SJEmSpPNmYKTLXE9oD4mqXyyltmORbCAZi1K+fAWVAP3GkzfkUqxRkiRJkqQrS3qPpEnnLUj+l4pY/+hGml7fyLL5G7vvFsil+FvTifS+uKuTJEmSJOlKZGCky17miBJ+/FweFWXrKa9r5NCBBEkAAgQH3kzk7vFMuSef7H6XeKGSJEmSJF0hDIz00ZAVofBrEQov9TokSZIkSboKfOzQe81/udSLOF+9A+ZekiRJkiRJF4pFryVJkiRJkpTCwEiSJEmSJEkpDIwkSZIkSZKUwsBIkiRJkiRJKQyMJEmSJEmSlMLASJIkSZIkSSkMjCRJkiRJkpTCwEiSJEmSJEkpDIy6U1fKtGlTWfRS/FKv5MK40r5HkiRJkiSllYGRJEmSJEmSUhgYSZIkSZIkKYWBkSRJkiRJklIYGEmSJEmSJCmFgZEkSZIkSZJSGBhJkiRJkiQpxcfTPUHrwd1s2VrH3qYPaD0GZPRkwCduYczYkQzum+7ZJUmSJEmSdLbSGhj96Xe/oOzVw6mNrS0ceruOivV7iXxuCuOy0rkCSZIkSZIkna30BUZvb2fjq4ehxzXcOOZu7v50Fn0C0PrBAf79Xzex450jRH9TzbCpeVyftkVIkiRJkiTpbKUtMGr4QwPvAwMihXx+WP8T7RnXZDG6qJD3f15O9Mheom/ncdcn07UKpdWBjSyaU0r93QtZN/tmYtvWU/pCJXsOJEgGMgnfOYvZs/LJzuhmbHMjlS+spmzHHuLNSQhkEgrnUfyleykcHux+vtY4tf9YytrNr9LYlAQCZN5wK4WTZ1B8Ww6Z3c0jSZIkSZLOWpoCo4PsPwRwDdk39u/m+bWEc64huvsDYm8fhk9210cfGfv2ULFyBaW/bjrZlkxQv/lpHowdYfnSIkIduid2r2bpd8tpTJLSP76rktJdW6mc9B0euS9MZsc5ElFKFyyh4t2OjUkSb9VQ9nQNiX4/p2REOj5OkiRJkqSrT5oCo570CgB8wIH9LZDVs0uPT1wXBD7gvaZ3AQOjj7SGMkobIDimhAWziggHIfFaGcseWUP97tWs3TGeuWMCbX0TNTz//XIakwFyJy5g7r2jyO4NJJuo37SSJ1dV07jh+5Tl/YQZQ09OEdu8kop3IXjnPBbPym8bAyQPNFJduZa4u4skSZIkSbpg0hQY9WXwwL7sPHSEP9VVsLXfnYy5sT+9erTVMHrjD3X8dtc7bV1bW9OzBHVRv2oO01Z1bs3hr9nHH87qTWFKnnuMwg4twQmLWT4rcmJXUOaQySyY9SpfXRGlum4PjIkAENu8mspmCIydxyMPjDq5iygQJDxxAYvf/wbzX4yxsaKa6UPzaI+ZOBSPAXDzyJEnwiKAQFYu+V9eeFarlyRJkiRJp5e2GkbXj8lj2Jub+WPiINEtvyCarol0GShiboew6Ljgp28lRJR44z7iRAjRRPR3MSCTosK8Lv0BsseOJ/fF1TTW7qGRPMLt7TnhUQR+XUP1D5bww5ZZTM8PEwx08wJJkiRJknTe0ndLWo8c7vpiMddv38bOtw7yfnu9moye/bn+plv4TGA3FbsO0yuzb9qWoHYjSli3ruTCv/fAhzzv3YcgEN8fpwkIEadto1A2oetOMWZgLmGgsTlJS4fm4J1zWfzmEpZsaKRyxbepXBEgODyPonsmUXBbLkGPpEmSJEmSdMGkLzACCFxL5D8WE+nmUayqFoBg/1MlB7piXDeAtnvPWmg5eq4vySR83xP8rDBKxa/K2fjyq8R2VbF2VxVrfxRhxuIFFA/pbs+SJEmSJEk6Wz0uyazH9hJ9vQXoT86NXQti6wrxToxGgAED2o+fhci5CSBGvOkUY/Y3Ug8wcAADunueFaHw/oUsL32Bnz2zmBljgnA0yvNPbmibS5IkSZIknbeLHxgdO0L0f1ezNwl8Yih/4wVpV6gE1Vs2kwTCnwmfCIxyh2UCCSo31ZDoZlT9prbgJ3N0hOwPmSEwMELx/NkUALwbpf7dC7l+SZIkSZKuXhctMPpz4iBv/nE7ZevWs/XtFug5iLvuHk6vi7UApdE+YrEEyeMX3h3dR+2qJTyzLQn9iph+d+hEz/DdU8gGElueZOmqGmLHj6gl40RfXMKSDU0QCDP9c+EO749T+fQy1m6pJ9acPNncmiT2ylZqAPrdTK6nGyVJkiRJuiDSWMPoIFt/Xk70SDePMnMY95/GM+ya9M2uiylK6fz/TGnn5kCY6Q9PJ9K7Q1t2MQtn1TB/ZZTGl5Yx/6XOY0IUPvQwhVmpzcl4DWUraijrdv4ghfOmEO72mSRJkiRJOlvpLXrdQUbPnvxV/08xOHILo2+6Fi+1upIECASSJNs3/wSCuUTGT2Lm5/LJ7te1d2jCYn48pJLnf7qe6vo4iSTQO0hk7BQmTy4iktVlBIXfWAy/Kqdyxx72HUiQBAL9Qtw8pvgUYyRJkiRJ0rlKY2B0LeOm3s+49E2gy8Z4Fv6spNvb8E4lc0gBX19cwNfPdEBWhML7IxTefw7LkyRJkiRJZ+XS3JImSZIkSZKky5aBkSRJkiRJklIYGEmSJEmSJCmFgZEkSZIkSZJSXLRb0nQFyiri0XVFl3oVkiRJkiTpAvvYofea/3KpF3G+egfMvSRJkiRJki4Uj6RJkiRJkiQphYGRJEmSJEmSUhgYSZIkSZIkKYWBkSRJkiRJklIYGEmSJEmSJCmFgZEkSZIkSZJSGBhJkiRJkiQphYGRJEmSJEmSUhgYSZIkSZIkKYWBUXfqSpk2bSqLXopf6pVITGW78AAADQ1JREFUkiRJkiRddAZGkiRJkiRJSmFgJEmSJEmSpBQGRpIkSZIkSUphYCRJkiRJkqQUBkaXTJL69Q/ylWlT+crCMuqPXqx541R8eyrTpk2ltK5D84GNLJo2lWnTSolerKVcCdoLpE/79kYskS5JkiRJulIYGF0ye9j6YiNJIPnaGrbWX+r1SJIkSZIktfn42Q5oPdjA9n+r44/7j9B6DLLH3s+k8GkGJA8S3b6NnW8c5P1WoEcGvfqFiIy+gzGf6nvuK//Iu5lxX86l8sVGGHIv4073G0qSJEmSJF1EZxYYHWvhT6/VUl3XQOy91jN/+wcNbCjbTqyl47ta+fPhd9hZWc4bn7mHqSOzzm7FV4wA4SlP8LMpl3odkiRJkiRJqc4sMHpzO/9c9QatAH1zGJOfRWxjLbHTDjrMjor2sCgzh3F3jyOS1RNaDrP337ZQUX+Y/1v3MttvmMId15/3d0iSJEmSJOkCObPA6KabGTagmdbhYxk39FoyaPiQsAjYW8u/HwICgxhXPJ7INe3tPfszeOwXKGxZQ8XrR4i++gZ33P2p8/iEj5boyqks/XXn1iIeWVdCpGtvSqctoWJoCc8+PJjK7z1O+e4Weo6cyfceLCKzYQ1LHy+jsTWTkfc/wcIJodThrQkaX17N8/9YzZ4DCZJAIBiiZ+I0CxyaTQiI71xD6QsVRN9KkAxkEr5zFrNn5ZOdcR4ff7SaZ0qepCqZSfFj/4sZQ0/RZ+aTVBHh6z9eTEGww7PmRipfWE3Zjj3Em5MQyCQUzqP4S/dSODzYzcvaHYhSUbae8g7jskcUMuXvJpF/Q2Zq39YmGl+ppHzTZqKvxWk6yol5Jt83k4Ihmd1OQW4OoeO/95qtRJuS0DubkV+cxdxJETLP53eTJEmSJOkiO8MaRoMY94Xis3rx3jf20Qr0yb31ZFh03KHd/PFA2zm11v0xYnyK7LN6+9XmEFUrv0PZ7iQAydrVlG/rS/xHZTQmARLUriyl6vaF5PdrH5KIUvrg41QcSKa8KdkUJ7Wl61y1q+aw+qUO/ZIJ6jc/zYOxIyxfWkTodMNPp3ceRRMyqfplgsrqemYM7Vq4KVn7ClUAI8YxqkMGlNi9mqXfLW//3pPriu+qpHTXVionfYdH7gvTOc6Jv/wki56tponUcbGdZTzz3gDCj6V+T3TVf+0a6LXP88OFW6md9wO+ObabcCrZSMWyxymt67DAozFq1yxhftNinnsgQuA0P40kSZIkSZeTsy56fWYOs/9gK5DBwOxBKU/ee+03/FPVG7x/rL3h6BEOHYPsq+S+tsisn7NuVvsfBzayaE4pH3pB2uvlrG8IMXnZUxS9uYSvrohS+aMfkLxpJsuX5hF9ZA6lDTXsaYT8EQBNVD3TFhYFRkxm4QNfIpLdFlckD9Sw+rvLqNh/irkayihtgOCYEhbMKiIchMRrZSx7ZA31u1ezdsd45o459+gjfEcBmb8sJ7HlFervC5MaGSWp2VEFBMifMI4TsUyihue/X05jMkDuxAXMvXcU2b2BZBP1m1by5KpqGjd8n7K8n6TuWmpY3R4WBcid+N/4+hfzyQ0CrUnib1VTUdFCF4Fc8u+dRHH+SHKuzSSQARzdR9XKJTzzchPV/3MD9WNn0iXq2rKaUgLkTlzIN+8bRSgA8W1Ps+jpKppeWsnGCU9RbCoqSZIkSfqISFNg1EJrK8A19Dl+EdqxIzRsrWDT3iPQoyfZnx1Or9/XsjfZzKGDwGVYx6h+1RymrercmsNfs48/nNWbwpQ89xiF51rfO5mk54RZTBkCgUQOECWZ7EnxjGKyM+BQLtDQoX/DBkprk3BdMYsX3ku4w3GoQFYOOf2AUwVGQHDCYpbPipzYrZM5ZDILZr3KV1dEqa7bA2O6Hp47Y0MLmHxDOc+/VUl1w0zCHQOeozVUbwMC4ykYfTKUim1eTWUzBMbO45EHRp3cRRQIEp64gMXvf4P5L8bYWFHN9KF57Tt5mqh8sZwmIHz/Uzz6+Q77iDIChG7KZ8bXui4vcv8TXY8G9s4hf/Zsoi8vo7J5D43vQvi6zp0ChB94ikcnnpwnNHYeC/5PLYt+GaO6Lk5x9jnvzZIkSZIk6aJK076eJg51rJNz5A1+84v1bWFR72sZPWEak27NoU+v9Mx+5QlRXNjpSNOImRTf0n3veH2UBBC6pyAlLDozRcztEBYdF/z0rYSAZOM+4mf7yhQ55N0TBtqOpXV0/Dha5oRxRE6su4no72JAJkWFeV3WBZA9djy5QLJ2D43HG5ujVNcBgXyKx1+AoCaj58nf/1g3z4fOZO7ErvPkhkcCUB87v19NkiRJkqSLKU07jPryV70hdhT+/PZ21tY2cOgY9Lp+BIVFI2k7HfVnWv9femY/byNKWLeu5FKvooMgmb07NQ0Mcaoyz/F32mKTyOCcC7eE3n0IAvH9cZrg3OsYAaHbxxNZVU805VhagurfVAHZTL6n44GvOPEYQDahLrt62g3MJQw0Nic5ccjszT1EAW66mdzOv93ptCaIvbKB1f9SRf2bcRKnL/j0oQKZ7VvsDjSRgG4DL0mSJEmSLjdpCox6kfFxgCM07GwAenL9sHF8/rYc+hzvcuww7x0F6MeAa9OziqtdIB03c1034JRB1RkLjqPwzpVEX+5wLK25lq21wNAi8lJq/bTQcvQc5mjlQ4p7dzcmTsWSb1C6+zxTou4E+xgWSZIkSZI+MtIUGF3LwAEQPQL06M+wgkLuuqHTVWmv728ro9O3PwOvkoLXF01GAEjSuL8JRnSKd1oTHGo+h3e+E2s77jVgwAUIPgKMGpNH4OUqKre33ZaWqN1KLRC557ZOu5dC5NwENMSINwHd1YHa39hWOHzgAAYcbxuUTRioP4sdUcmda1i9OwmBXAr/x1xmfianQ+gWpXTaEirO8kvjsb0AZPbr+yE9JUmSJEm6fKQtqrlxUPv/2fcZRKRzWMQH/P4Pb9AK9PnkjZdjveuPtNwhtwJQ/8c9nXbZJIiuepyy0xS87l6C6i2bSQKR27rWNwKgNU7V8jl8Zdrf8d+XVxFvPf0bA6MnUtQPEi+/Qj0JarfVQCCfwts7718KkTssE0hQuamGRDfvqt+0gUYgc3SEE5uTsgZzcz+geSMbf9vdqK727Kpu+73yp1MyOid1h1ZrS7dzn1brPqo21QOZFHy2y71qkiRJkiRdttIWGPUaNpJh1wBHdrNhYy1vvteeILQcILqpnO0HgMAgRo8alK4lXLUyR+S13fS1bSUrtsVJtgJN9VQsf5Blv04QCJxu9D4aX28fA3B0H7WrlvDMtiRcV8TkLoFOu13lrPhtnCRJ4r/9AWt3fsixrowwBZ8LQXMl1XVRamohcPttjOqm3lD47ilkA4ktT7J0VQ2x40fUknGiLy5hyYYmCISZ/rmOoUyYgr/NBpJUPfsgf78henJca5Km16t4/keVKQW8Q6H2mk91W6k6cHyOJhq3reahB5ZRdbrvaW6kMZY48bslm+qpWL6EtW9B4JbpFJ6iQLkkSZIkSZejMzyS1sCGVduJdfMktu0fWLGt/Y9P3sHswvZ70nsM4q7xtxD71W7ee6eOX66vSx3Y4xoG3zGOSOfNR1e0Ux1r2sjSaRtP/BV+4LmU69nPWrCA6ZPWsGhDE1VPz6Hq6eMPAoQmPsw3B67hoVX1pxgc5fkH5/B85+ZAmOkLphM5RQHp5NEjHf+CD9lhBJB9WxHZL66m+h/WcIhMiu7Jo9ssK7uYhbNqmL8ySuNLy5j/Uue1hSh86GEKOx1Xy/78w8yt/zbP7IhT/dMlVP+007ihJRR2+DM0poDwC6XUv1vFM3OqeKZj39655GY30tjdfwQA+yv5+/mVXduvy2fuvKLzKhIuSZIkSdLFlt7qQVl5fGXKXYz+ZH8yjs/UI4M+nxjKXcVTKBx8VaVFF1X4vqf43gP5ZPdri2ACwQjFDz3Fsw+c4kjZteOY/VAJxaNzCQVPxjaBYC4jp8xj+Q8fY/KQU1cvCoy+l9m3hwgQIHT7PKaPOe02pjbZ4ygeAfG3YiQHTqLgNLtwQhMW8+NlsykYHiLz+Kt7B4mML+GRp56jZEQ3a8sIkb/gJzz70L3kdxqXO3oyC7/RKcjJKuLRJ+ZRMDx4Irhq+/6FPFv6BDOGd7Ow4cU8Orvt/aF+J0aRmRWh4IHFPPvsPPJOdbubJEmSJEmXqY8deq/5L5d6EeerdyBNtbslSZIkSZKuQt5PJkmSJEmSpBQGRpIkSZIkSUphYCRJkiRJkqQUBkaSJEmSJElKYWAkSZIkSZKkFAZGkiRJkiRJSmFgJEmSJEmSpBQGRpIkSZIkSUphYCRJkiRJkqQUBkaSJEmSJElKYWAkSZIkSZKkFAZGkiRJkiRJSmFgJEmSJEmSpBQGRpIkSZIkSUphYCRJkiRJkqQUBkaSJEmSJElKYWAkSZIkSZKkFAZGkiRJkiRJSvH/AelqaVVkDjfpAAAAAElFTkSuQmCC)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "id": "Ch_xzt3VADpo" }, "outputs": [], "source": [ "import os\n", "def read_imdb(dir='aclImdb', split='pos', is_train=True):\n", " subdir = 'train' if is_train else 'test'\n", " dir = os.path.join(dir, subdir, split)\n", " lines = []\n", " for file in os.listdir(dir):\n", " with open(os.path.join(dir, file), 'rb') as f:\n", " line = f.read().decode('utf-8')\n", " lines.append(line)\n", " return lines" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "xqA-za5zC5nD", "outputId": "b409c136-88f0-4451-ec2a-510b1ca7a600" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['a', ',', 'b']\n" ] } ], "source": [ "from torchtext.data import get_tokenizer\n", "\n", "tokenizer = get_tokenizer('basic_english')\n", "print(tokenizer('a, b'))" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "-FyMlyz_DHnR", "outputId": "e21fb8be-41ac-4e44-c8c5-9d2128cee196" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Length of the file: 12500\n", "lines[0]: This has to be one of the most beautiful, moving, thought provoking films around. It's good family entertainment and at the same time makes you think very hard about the issues involved. Every time I see the \"ghost of Zac riding the bike through the puddle at the end I can't help but cry my eyes out. John Thaw's performance is so touching and it is a shame he is no longer with us. Gone but not forgotten. A outstanding film. Full marks.\n", "lines[0] tokens: ['this', 'has', 'to', 'be', 'one', 'of', 'the', 'most', 'beautiful', ',', 'moving', ',', 'thought', 'provoking', 'films', 'around', '.', 'it', \"'\", 's', 'good', 'family', 'entertainment', 'and', 'at', 'the', 'same', 'time', 'makes', 'you', 'think', 'very', 'hard', 'about', 'the', 'issues', 'involved', '.', 'every', 'time', 'i', 'see', 'the', 'ghost', 'of', 'zac', 'riding', 'the', 'bike', 'through', 'the', 'puddle', 'at', 'the', 'end', 'i', 'can', \"'\", 't', 'help', 'but', 'cry', 'my', 'eyes', 'out', '.', 'john', 'thaw', \"'\", 's', 'performance', 'is', 'so', 'touching', 'and', 'it', 'is', 'a', 'shame', 'he', 'is', 'no', 'longer', 'with', 'us', '.', 'gone', 'but', 'not', 'forgotten', '.', 'a', 'outstanding', 'film', '.', 'full', 'marks', '.']\n" ] } ], "source": [ "from torchtext.data import get_tokenizer\n", "lines = read_imdb()\n", "print('Length of the file:', len(lines))\n", "print('lines[0]:', lines[0])\n", "tokenizer = get_tokenizer('basic_english')\n", "tokens = tokenizer(lines[0])\n", "print('lines[0] tokens:', tokens)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "x3BX2ChaDSEP", "outputId": "a0a296b1-534c-4441-d007-0dc8a6df5201" }, "outputs": [], "source": [ "from torch.utils.data import DataLoader, Dataset\n", "from torchtext.data import get_tokenizer\n", "from torchtext.vocab import GloVe\n", "\n", "\n", "GLOVE_DIM = 100\n", "GLOVE = GloVe(name='6B', dim=GLOVE_DIM)\n", "\n", "\n", "class IMDBDataset(Dataset):\n", " def __init__(self, is_train=True, dir='data/aclImdb'):\n", " super().__init__()\n", " self.tokenizer = get_tokenizer('basic_english')\n", " pos_lines = read_imdb(dir, 'pos', is_train)\n", " neg_lines = read_imdb(dir, 'neg', is_train)\n", " self.lines = pos_lines + neg_lines\n", " self.pos_length = len(pos_lines)\n", " self.neg_length = len(neg_lines)\n", "\n", " def __len__(self):\n", " return self.pos_length + self.neg_length\n", "\n", " def __getitem__(self, index):\n", " sentence = self.tokenizer(self.lines[index])\n", " x = GLOVE.get_vecs_by_tokens(sentence)\n", " label = 1 if index < self.pos_length else 0\n", " return x, label" ] }, { "cell_type": "markdown", "metadata": { "id": "SXLxfSEIYRYt" }, "source": [ "When PyTorch DataLoader obtains a batch of Dataset data, it actually calls **Dataset.\\_\\_getitem\\_\\_** first to obtain several samples, and then stitches all the samples into a batch. For example, use **\\_\\_getitem\\_\\_** to obtain 4 image tensors of [3, 10, 10], and then stitch them into a batch of [4, 3, 10, 10]. However, sequence data usually have different lengths, and **\\_\\_getitem\\_\\_** may obtain word embedding arrays of unequal lengths such as [10, 100], [15, 100]." ] }, { "cell_type": "markdown", "metadata": { "id": "tCgM84zjZmgy" }, "source": [ "To solve this problem, we have to manually write a function that combines all tensors into a batch. This function is the **collate_fn** function of DataLoader. Our **collate_fn** should be written like this:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "id": "WICY0BI8Z24K" }, "outputs": [], "source": [ "def collate_fn(batch):\n", " x, y = zip(*batch)\n", " x_pad = pad_sequence(x, batch_first=True)\n", " y = torch.Tensor(y)\n", " return x_pad, y" ] }, { "cell_type": "markdown", "metadata": { "id": "Z9OD9D3aZ6SP" }, "source": [ "The input batch of collate_fn is an array of the results of each \\_\\_getitem\\_\\_. For example, in our tutorial, for the first time to obtain a positive sentence of length 10, __getitem__ returns (Tensor[10, 100], 1); for the second time to obtain a negative sentence of length 15, __getitem_ _ returns (Tensor[15, 100], 0). Then, the content of the input batch is: [(Tensor[10, 100], 1), (Tensor[15, 100], 0)] \n", "\n", "We can neatly convert this into two tuples with x, y = zip(*batch):x = (Tensor[10, 100], Tensor[15, 100]), y = (1, 0) \n", "\n", "After that, PyTorch's pad_sequence can fill the array of unequal-length sequences into a whole batch tensor according to the maximum length. That is, after this function, x_pad becomes: x_pad = Tensor[2, 15, 100]\n", "\n", "The batch_first of pad_sequence determines whether the batch is in the first dimension. If it is False, the resulting tensor has shape [15, 2, 100].\n", "\n", "pad_sequence can also determine the padding content, and the default padding is 0. \n", "\n", "\n", " " ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "id": "VLeH99CuEAZM" }, "outputs": [], "source": [ "import torch\n", "from torch.nn.utils.rnn import pad_sequence\n", "\n", "def get_dataloader(dir='aclImdb'):\n", " def collate_fn(batch):\n", " x, y = zip(*batch)\n", " x_pad = pad_sequence(x, batch_first=True)\n", " y = torch.Tensor(y)\n", " return x_pad, y\n", "\n", " train_dataloader = DataLoader(IMDBDataset(True, dir),\n", " batch_size=32,\n", " shuffle=True,\n", " collate_fn=collate_fn)\n", " test_dataloader = DataLoader(IMDBDataset(False, dir),\n", " batch_size=32,\n", " shuffle=True,\n", " collate_fn=collate_fn)\n", " return train_dataloader, test_dataloader" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "id": "JDa2Huh7FdiQ" }, "outputs": [], "source": [ "from torch import nn\n", "class RNN(torch.nn.Module):\n", " def __init__(self, hidden_units=64, dropout_rate=0.5):\n", " super().__init__()\n", " self.drop = nn.Dropout(dropout_rate)\n", " self.rnn = nn.GRU(GLOVE_DIM, hidden_units, 1, batch_first=True)\n", " self.linear = nn.Linear(hidden_units, 1)\n", " self.sigmoid = nn.Sigmoid()\n", "\n", " def forward(self, x: torch.Tensor):\n", " # x shape: [batch, max_word_length, embedding_length]\n", " emb = self.drop(x)\n", " output, _ = self.rnn(emb)\n", " output = output[:, -1]\n", " output = self.linear(output)\n", " output = self.sigmoid(output)\n", "\n", " return output" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "id": "zPuJLM7pFl0J" }, "outputs": [], "source": [ "device = 'cuda:0'\n", "\n", "train_dataloader, test_dataloader = get_dataloader()\n", "model = RNN().to(device)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "vqL5-kBRFp79", "outputId": "214739bf-6075-4576-9db2-ef62a5af3ce2" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 0. loss: 0.6934672594070435\n", "Epoch 1. loss: 0.6929126977920532\n", "Epoch 2. loss: 0.6916374564170837\n", "Epoch 3. loss: 0.6909961700439453\n", "Epoch 4. loss: 0.689192533493042\n", "Epoch 5. loss: 0.6464381217956543\n", "Epoch 6. loss: 0.4718573987483978\n", "Epoch 7. loss: 0.3859955668449402\n", "Epoch 8. loss: 0.3578636646270752\n", "Epoch 9. loss: 0.3450615406036377\n", "Epoch 10. loss: 0.32949769496917725\n", "Epoch 11. loss: 0.3207785189151764\n", "Epoch 12. loss: 0.3144107758998871\n", "Epoch 13. loss: 0.3047332465648651\n", "Epoch 14. loss: 0.3007034957408905\n", "Epoch 15. loss: 0.2887532711029053\n", "Epoch 16. loss: 0.2887987494468689\n", "Epoch 17. loss: 0.2833702862262726\n", "Epoch 18. loss: 0.2852843701839447\n", "Epoch 19. loss: 0.2748873233795166\n", "Epoch 20. loss: 0.2750954329967499\n", "Epoch 21. loss: 0.2722353935241699\n", "Epoch 22. loss: 0.2695784568786621\n", "Epoch 23. loss: 0.26367488503456116\n", "Epoch 24. loss: 0.26122328639030457\n", "Epoch 25. loss: 0.2593787908554077\n", "Epoch 26. loss: 0.2570563554763794\n", "Epoch 27. loss: 0.2550010681152344\n", "Epoch 28. loss: 0.2549012303352356\n", "Epoch 29. loss: 0.2526344656944275\n", "Epoch 30. loss: 0.247743159532547\n", "Epoch 31. loss: 0.2498960942029953\n", "Epoch 32. loss: 0.24333757162094116\n", "Epoch 33. loss: 0.2437007576227188\n", "Epoch 34. loss: 0.2462003231048584\n", "Epoch 35. loss: 0.24142664670944214\n", "Epoch 36. loss: 0.2437993288040161\n", "Epoch 37. loss: 0.24092960357666016\n", "Epoch 38. loss: 0.2363073229789734\n", "Epoch 39. loss: 0.2353615015745163\n", "Epoch 40. loss: 0.2338671237230301\n", "Epoch 41. loss: 0.23543184995651245\n", "Epoch 42. loss: 0.23143352568149567\n", "Epoch 43. loss: 0.2309177815914154\n", "Epoch 44. loss: 0.2363603264093399\n", "Epoch 45. loss: 0.22826024889945984\n", "Epoch 46. loss: 0.22883331775665283\n", "Epoch 47. loss: 0.23002013564109802\n", "Epoch 48. loss: 0.22971011698246002\n", "Epoch 49. loss: 0.22750939428806305\n", "Epoch 50. loss: 0.22622466087341309\n", "Epoch 51. loss: 0.2246144711971283\n", "Epoch 52. loss: 0.22550545632839203\n", "Epoch 53. loss: 0.2243306189775467\n", "Epoch 54. loss: 0.22168390452861786\n", "Epoch 55. loss: 0.21889589726924896\n", "Epoch 56. loss: 0.22113782167434692\n", "Epoch 57. loss: 0.22470778226852417\n", "Epoch 58. loss: 0.2217176854610443\n", "Epoch 59. loss: 0.22118981182575226\n", "Epoch 60. loss: 0.21866659820079803\n", "Epoch 61. loss: 0.2175607979297638\n", "Epoch 62. loss: 0.21843615174293518\n", "Epoch 63. loss: 0.2146735042333603\n", "Epoch 64. loss: 0.2139790952205658\n", "Epoch 65. loss: 0.21336378157138824\n", "Epoch 66. loss: 0.21297763288021088\n", "Epoch 67. loss: 0.2176819145679474\n", "Epoch 68. loss: 0.21801139414310455\n", "Epoch 69. loss: 0.2145390510559082\n", "Epoch 70. loss: 0.21072889864444733\n", "Epoch 71. loss: 0.21235904097557068\n", "Epoch 72. loss: 0.20949867367744446\n", "Epoch 73. loss: 0.21345573663711548\n", "Epoch 74. loss: 0.20737095177173615\n", "Epoch 75. loss: 0.21057772636413574\n", "Epoch 76. loss: 0.20948177576065063\n", "Epoch 77. loss: 0.20646873116493225\n", "Epoch 78. loss: 0.20903337001800537\n", "Epoch 79. loss: 0.20799784362316132\n", "Epoch 80. loss: 0.2072860449552536\n", "Epoch 81. loss: 0.20564407110214233\n", "Epoch 82. loss: 0.20685015618801117\n", "Epoch 83. loss: 0.21029867231845856\n", "Epoch 84. loss: 0.21078412234783173\n", "Epoch 85. loss: 0.20617659389972687\n", "Epoch 86. loss: 0.2050018459558487\n", "Epoch 87. loss: 0.20389209687709808\n", "Epoch 88. loss: 0.20479126274585724\n", "Epoch 89. loss: 0.20785142481327057\n", "Epoch 90. loss: 0.20597250759601593\n", "Epoch 91. loss: 0.20249052345752716\n", "Epoch 92. loss: 0.2056071013212204\n", "Epoch 93. loss: 0.20395542681217194\n", "Epoch 94. loss: 0.20316976308822632\n", "Epoch 95. loss: 0.20478659868240356\n", "Epoch 96. loss: 0.20521263778209686\n", "Epoch 97. loss: 0.2066359668970108\n", "Epoch 98. loss: 0.20095206797122955\n", "Epoch 99. loss: 0.2047610729932785\n" ] } ], "source": [ "# train\n", "\n", "optimizer = torch.optim.Adam(model.parameters(), lr=0.001)\n", "citerion = torch.nn.BCELoss()\n", "for epoch in range(100):\n", "\n", " loss_sum = 0\n", " dataset_len = len(train_dataloader.dataset)\n", "\n", " for x, y in train_dataloader:\n", " batchsize = y.shape[0]\n", " x = x.to(device)\n", " y = y.to(device)\n", " hat_y = model(x)\n", " hat_y = hat_y.squeeze(-1)\n", " loss = citerion(hat_y, y)\n", "\n", " optimizer.zero_grad()\n", " loss.backward()\n", " torch.nn.utils.clip_grad_norm_(model.parameters(), 0.5)\n", " optimizer.step()\n", "\n", " loss_sum += loss * batchsize\n", "\n", " print(f'Epoch {epoch}. loss: {loss_sum / dataset_len}')\n", "\n", "torch.save(model.state_dict(), 'rnn.pth')\n" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "B92fqNM-KTWp", "outputId": "7ab3e2b3-8f27-4a7a-9e4b-440a31d3f8a8" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy: 0.90552\n" ] } ], "source": [ "# test\n", "\n", "# model.load_state_dict(\n", "# torch.load('rnn.pth', 'cuda:0'))\n", "\n", "accuracy = 0\n", "dataset_len = len(test_dataloader.dataset)\n", "model.eval()\n", "for x, y in test_dataloader:\n", " x = x.to(device)\n", " y = y.to(device)\n", " with torch.no_grad():\n", " hat_y = model(x)\n", " hat_y.squeeze_(1)\n", " predictions = torch.where(hat_y > 0.5, 1, 0)\n", " score = torch.sum(torch.where(predictions == y, 1, 0))\n", " accuracy += score.item()\n", "accuracy /= dataset_len\n", "\n", "print(f'Accuracy: {accuracy}')" ] }, { "cell_type": "markdown", "metadata": { "id": "ybdfnFX7bpTZ" }, "source": [ "## Other useful tools: Hugging face https://huggingface.co/\n" ] }, { "cell_type": "markdown", "metadata": { "id": "PMXlKeUkd-tj" }, "source": [ "So many interesting open-source models, like GPT-2, CLIP, BERT, T5 Language model,etc. And it is easy to use.\n", "https://zhuanlan.zhihu.com/p/535100411" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "EX83QS3KdkhM", "outputId": "5e97d302-88be-4a5f-8159-c9933fdf74cb" }, "outputs": [], "source": [ "# ! pip install transformers" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 368, "referenced_widgets": [ "0efaa591c4b54b93b29bd69721d66b31", "1f172a738ef94b83ae662a89d9dbbe62", "5bb99d1007fc4a4a8856e904b378615e", "5cd97d49a9a048db9d98866c42e32110", "c3f92a9d4c894d0cb80bb54323fcfcd6", "857b7b858ca64896ae49a5b314e93439", "7c354ccc9f164a2584617eabe2ad08d4", "c1cdbd492cf84bc3acde9da7810f786a", "a73f6b88fdb44af697057bfe5055bb37", "937228cc59af474dae0f299f6379aa79", "9f19fb55995841b481d938dcb387b0b3", "a8825b6f36414dbeaf4026d81c9b08e9", "824aff761b22429385ec9010019b8c10", "ff8c117aa7b747e1963d5cb641e47114", "1acf38c44f4d43568a467bbeb1e97a4a", "30aa741beadd4ac2911f5ccb27e4405e", "0887f89d0ee44fd88875bc46c4b59d57", "bf6807fcb9ca412c85e65f387fe85722", "7a98448742e94892baf3b24849a38458", "14d4e8b86ef54e93a0858ea7e9f0072d", "58baa8ce9c2e43cd8152a58d4ecbd438", "b755ce6fa3144a13a15388b75aad273e", "9b73d0029fe5464ebf23809ca9e62e16", "b37783c1e4b647b9aca1db9a20a68f91", "b58c9a8d05aa47ee82548e06ba5a4799", "7c67f1b4d9b34492ae772a103e20fe45", "03ebe654db5742c6ae86fcf95f2282e4", "0e48cc2661ba41268f5711800ba1e649", "3b97a4fae0684e0f878153b21db5cef4", "f68cf7907a1f42e79064b146bc5d532c", "d8579cc9452f40efb5d59b35965c09e1", "5143452f882d45fd978d756560c0b0f4", "f694d0aba43a4c0da514e54aedc18b68", "002877424ef04670987fe767030907f1", "d4116be457ee483e892a16e584a1f92b", "ea3f57c97650471c9f05696682e8bfa1", "6bb267b5c91d47b08d2bcc55127f0016", "6760203bdfb84018af7d7ea3b9f3fd4d", "5d1bee6606464bf89a4f29451ca884b2", "3090f48db7404f6a83443110c9718f0b", "c10d4345b03341f48e6f3393494bdd25", "8962569b83834c9da97eaaf6e16b72bf", "9a1ed14d6ece4e7b82cde01fae6e4bab", "bb90c44ad3794eacb2fce813940fcf2b", "6ef3faaab3c946c2826f31ebfacc7bca", "1b526080028a422683e24c17023fce5e", "69fef8b0eaa34bb5b422142a7d3772a8", "b4ed44c11b61434892310d03f4661728", "fcc840dd2dd74d118318c7da0492152f", "359329f484364952a7e99880e9eea7da", "77e0e85ba0884fc189e8dd816888b792", "61b6ee991f514dc6913e7782160f2efd", "f975fd5849114f638808ca7eb9788593", "02624e7e9ad54ba09d544ac002242cb1", "ce547fff1ba240aab1017e92cbb5d4c5", "59f5f82ad8fa4cffa43858b63161ba43", "e27034fd69f346a08ae7fb0e0807e72c", "dd48ee460fd54beabbe6b30e5b3eabbd", "4386d570231248b895a7bd3cc4b03f8e", "684d583551d94d54a6f9c953c94558aa", "cc7e1638a7e14e36aa9d8f8c199d2fa9", "b37b8d1dfde4492384963023a8f450d7", "ca837db77ade45299ccb8f01214a8bf4", "607558dbc3eb471cb11fab329b73faa7", "a878fb10014d48a4b7db93afbae892f2", "0857477126d04faf96394181d9e31b0d" ] }, "id": "WIRjKPoCW2Wa", "outputId": "7b5b3e14-d3e1-48c0-9de7-3e50f15ef23e" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Downloading config.json: 665B [00:00, 307kB/s] \n", "Downloading model.safetensors: 100%|██████████| 548M/548M [00:36<00:00, 15.2MB/s] \n", "Downloading tokenizer_config.json: 100%|██████████| 26.0/26.0 [00:00<00:00, 4.22kB/s]\n", "Downloading vocab.json: 1.04MB [00:02, 372kB/s] \n", "Downloading merges.txt: 456kB [00:01, 272kB/s] \n", "Downloading tokenizer.json: 1.36MB [00:05, 260kB/s]\n", "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n" ] }, { "data": { "text/plain": [ "[{'generated_text': \"Hello, I'm a language model, I'm writing a new language for you. But first, I'd like to tell you about the language itself\"},\n", " {'generated_text': \"Hello, I'm a language model, and I'm trying to be as expressive as possible. In order to be expressive, it is necessary to know\"},\n", " {'generated_text': \"Hello, I'm a language model, so I don't get much of a license anymore, but I'm probably more familiar with other languages on that\"},\n", " {'generated_text': \"Hello, I'm a language model, a functional model... It's not me, it's me!\\n\\nI won't bore you with how\"},\n", " {'generated_text': \"Hello, I'm a language model, not an object model.\\n\\nIn a nutshell, I need to give language model a set of properties that\"}]" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import os\n", "os.environ[\"HF_ENDPOINT\"] = \"https://hf-mirror.com\"\n", "from transformers import pipeline, set_seed\n", "generator = pipeline('text-generation', model='gpt2')\n", "set_seed(42)\n", "generator(\"Hello, I'm a language model,\", max_length=30, num_return_sequences=5)\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "FJSSBJUWdN2c" }, "outputs": [], "source": [ "\n" ] } ], "metadata": { "accelerator": "GPU", "colab": { "provenance": [] }, "gpuClass": "standard", "kernelspec": { "display_name": "base", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.4" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "002877424ef04670987fe767030907f1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_d4116be457ee483e892a16e584a1f92b", "IPY_MODEL_ea3f57c97650471c9f05696682e8bfa1", "IPY_MODEL_6bb267b5c91d47b08d2bcc55127f0016" ], "layout": "IPY_MODEL_6760203bdfb84018af7d7ea3b9f3fd4d" } }, "02624e7e9ad54ba09d544ac002242cb1": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "03ebe654db5742c6ae86fcf95f2282e4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0857477126d04faf96394181d9e31b0d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "0887f89d0ee44fd88875bc46c4b59d57": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0e48cc2661ba41268f5711800ba1e649": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0efaa591c4b54b93b29bd69721d66b31": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_1f172a738ef94b83ae662a89d9dbbe62", "IPY_MODEL_5bb99d1007fc4a4a8856e904b378615e", "IPY_MODEL_5cd97d49a9a048db9d98866c42e32110" ], "layout": "IPY_MODEL_c3f92a9d4c894d0cb80bb54323fcfcd6" } }, "14d4e8b86ef54e93a0858ea7e9f0072d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "1acf38c44f4d43568a467bbeb1e97a4a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_58baa8ce9c2e43cd8152a58d4ecbd438", "placeholder": "​", "style": "IPY_MODEL_b755ce6fa3144a13a15388b75aad273e", "value": " 548M/548M [00:02<00:00, 234MB/s]" } }, "1b526080028a422683e24c17023fce5e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_359329f484364952a7e99880e9eea7da", "placeholder": "​", "style": "IPY_MODEL_77e0e85ba0884fc189e8dd816888b792", "value": "Downloading (…)olve/main/merges.txt: 100%" } }, "1f172a738ef94b83ae662a89d9dbbe62": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_857b7b858ca64896ae49a5b314e93439", "placeholder": "​", "style": "IPY_MODEL_7c354ccc9f164a2584617eabe2ad08d4", "value": "Downloading (…)lve/main/config.json: 100%" } }, "3090f48db7404f6a83443110c9718f0b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "30aa741beadd4ac2911f5ccb27e4405e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "359329f484364952a7e99880e9eea7da": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3b97a4fae0684e0f878153b21db5cef4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "4386d570231248b895a7bd3cc4b03f8e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a878fb10014d48a4b7db93afbae892f2", "placeholder": "​", "style": "IPY_MODEL_0857477126d04faf96394181d9e31b0d", "value": " 1.36M/1.36M [00:01<00:00, 1.02MB/s]" } }, "5143452f882d45fd978d756560c0b0f4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "58baa8ce9c2e43cd8152a58d4ecbd438": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "59f5f82ad8fa4cffa43858b63161ba43": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_e27034fd69f346a08ae7fb0e0807e72c", "IPY_MODEL_dd48ee460fd54beabbe6b30e5b3eabbd", "IPY_MODEL_4386d570231248b895a7bd3cc4b03f8e" ], "layout": "IPY_MODEL_684d583551d94d54a6f9c953c94558aa" } }, "5bb99d1007fc4a4a8856e904b378615e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c1cdbd492cf84bc3acde9da7810f786a", "max": 665, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_a73f6b88fdb44af697057bfe5055bb37", "value": 665 } }, "5cd97d49a9a048db9d98866c42e32110": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_937228cc59af474dae0f299f6379aa79", "placeholder": "​", "style": "IPY_MODEL_9f19fb55995841b481d938dcb387b0b3", "value": " 665/665 [00:00<00:00, 24.7kB/s]" } }, "5d1bee6606464bf89a4f29451ca884b2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "607558dbc3eb471cb11fab329b73faa7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "61b6ee991f514dc6913e7782160f2efd": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6760203bdfb84018af7d7ea3b9f3fd4d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "684d583551d94d54a6f9c953c94558aa": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "69fef8b0eaa34bb5b422142a7d3772a8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_61b6ee991f514dc6913e7782160f2efd", "max": 456318, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_f975fd5849114f638808ca7eb9788593", "value": 456318 } }, "6bb267b5c91d47b08d2bcc55127f0016": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9a1ed14d6ece4e7b82cde01fae6e4bab", "placeholder": "​", "style": "IPY_MODEL_bb90c44ad3794eacb2fce813940fcf2b", "value": " 1.04M/1.04M [00:01<00:00, 940kB/s]" } }, "6ef3faaab3c946c2826f31ebfacc7bca": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_1b526080028a422683e24c17023fce5e", "IPY_MODEL_69fef8b0eaa34bb5b422142a7d3772a8", "IPY_MODEL_b4ed44c11b61434892310d03f4661728" ], "layout": "IPY_MODEL_fcc840dd2dd74d118318c7da0492152f" } }, "77e0e85ba0884fc189e8dd816888b792": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7a98448742e94892baf3b24849a38458": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7c354ccc9f164a2584617eabe2ad08d4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7c67f1b4d9b34492ae772a103e20fe45": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5143452f882d45fd978d756560c0b0f4", "placeholder": "​", "style": "IPY_MODEL_f694d0aba43a4c0da514e54aedc18b68", "value": " 124/124 [00:00<00:00, 4.38kB/s]" } }, "824aff761b22429385ec9010019b8c10": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0887f89d0ee44fd88875bc46c4b59d57", "placeholder": "​", "style": "IPY_MODEL_bf6807fcb9ca412c85e65f387fe85722", "value": "Downloading pytorch_model.bin: 100%" } }, "857b7b858ca64896ae49a5b314e93439": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8962569b83834c9da97eaaf6e16b72bf": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "937228cc59af474dae0f299f6379aa79": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9a1ed14d6ece4e7b82cde01fae6e4bab": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9b73d0029fe5464ebf23809ca9e62e16": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_b37783c1e4b647b9aca1db9a20a68f91", "IPY_MODEL_b58c9a8d05aa47ee82548e06ba5a4799", "IPY_MODEL_7c67f1b4d9b34492ae772a103e20fe45" ], "layout": "IPY_MODEL_03ebe654db5742c6ae86fcf95f2282e4" } }, "9f19fb55995841b481d938dcb387b0b3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a73f6b88fdb44af697057bfe5055bb37": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a878fb10014d48a4b7db93afbae892f2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a8825b6f36414dbeaf4026d81c9b08e9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_824aff761b22429385ec9010019b8c10", "IPY_MODEL_ff8c117aa7b747e1963d5cb641e47114", "IPY_MODEL_1acf38c44f4d43568a467bbeb1e97a4a" ], "layout": "IPY_MODEL_30aa741beadd4ac2911f5ccb27e4405e" } }, "b37783c1e4b647b9aca1db9a20a68f91": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0e48cc2661ba41268f5711800ba1e649", "placeholder": "​", "style": "IPY_MODEL_3b97a4fae0684e0f878153b21db5cef4", "value": "Downloading (…)neration_config.json: 100%" } }, "b37b8d1dfde4492384963023a8f450d7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b4ed44c11b61434892310d03f4661728": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_02624e7e9ad54ba09d544ac002242cb1", "placeholder": "​", "style": "IPY_MODEL_ce547fff1ba240aab1017e92cbb5d4c5", "value": " 456k/456k [00:00<00:00, 516kB/s]" } }, "b58c9a8d05aa47ee82548e06ba5a4799": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f68cf7907a1f42e79064b146bc5d532c", "max": 124, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_d8579cc9452f40efb5d59b35965c09e1", "value": 124 } }, "b755ce6fa3144a13a15388b75aad273e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "bb90c44ad3794eacb2fce813940fcf2b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "bf6807fcb9ca412c85e65f387fe85722": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "c10d4345b03341f48e6f3393494bdd25": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c1cdbd492cf84bc3acde9da7810f786a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c3f92a9d4c894d0cb80bb54323fcfcd6": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ca837db77ade45299ccb8f01214a8bf4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cc7e1638a7e14e36aa9d8f8c199d2fa9": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ce547fff1ba240aab1017e92cbb5d4c5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "d4116be457ee483e892a16e584a1f92b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5d1bee6606464bf89a4f29451ca884b2", "placeholder": "​", "style": "IPY_MODEL_3090f48db7404f6a83443110c9718f0b", "value": "Downloading (…)olve/main/vocab.json: 100%" } }, "d8579cc9452f40efb5d59b35965c09e1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "dd48ee460fd54beabbe6b30e5b3eabbd": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ca837db77ade45299ccb8f01214a8bf4", "max": 1355256, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_607558dbc3eb471cb11fab329b73faa7", "value": 1355256 } }, "e27034fd69f346a08ae7fb0e0807e72c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_cc7e1638a7e14e36aa9d8f8c199d2fa9", "placeholder": "​", "style": "IPY_MODEL_b37b8d1dfde4492384963023a8f450d7", "value": "Downloading (…)/main/tokenizer.json: 100%" } }, "ea3f57c97650471c9f05696682e8bfa1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c10d4345b03341f48e6f3393494bdd25", "max": 1042301, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_8962569b83834c9da97eaaf6e16b72bf", "value": 1042301 } }, "f68cf7907a1f42e79064b146bc5d532c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f694d0aba43a4c0da514e54aedc18b68": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "f975fd5849114f638808ca7eb9788593": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "fcc840dd2dd74d118318c7da0492152f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ff8c117aa7b747e1963d5cb641e47114": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7a98448742e94892baf3b24849a38458", "max": 548118077, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_14d4e8b86ef54e93a0858ea7e9f0072d", "value": 548118077 } } } } }, "nbformat": 4, "nbformat_minor": 4 }