Finetuned_Llama3_8b_to_generate_java_codes_by_RahulAkaVector (1).ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "## Finetuned Llama3 8b for Java Code Generation\n",
        "\n",
        "This Colab notebook focuses on utilizing a finetuned version of Llama3 8b, a language model, for generating Java code snippets. The model has been optimized and trained specifically for Java code generation tasks.\n"
      ],
      "metadata": {
        "id": "C6axa23BfyOs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!nvidia-smi # check the details of env before running"
      ],
      "metadata": {
        "id": "OWKQZAgAyhts"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🔵 **Installation of Dependencies**\n",
        "\n",
        "<details>\n",
        "\n",
        "<summary>The first cell of this notebook handles the installation of necessary packages and dependencies required for running the code.( click here to expand ) </summary>\n",
        "\n",
        "It installs:\n",
        "- **Unsloth**: A Python package for integrating pretrained language models into your projects, tailored for Colab environments.\n",
        "- **Xformers**: Utilized for its Flash Attention mechanism, enhancing the model's performance.\n",
        "- Other supporting packages required for this task, such as `trl`, `peft`, `accelerate`, and `bitsandbytes`.\n",
        "\n",
        "The `%%capture` magic command is used to suppress the output of the installation process within the notebook.\n",
        "\n",
        "<br>\n",
        "<hr>\n",
        "\n",
        "### Note:\n",
        "Ensure that you have adequate resources available in your Colab environment, especially GPU acceleration, to efficiently run the notebook and leverage the power of the language model for Java code generation tasks.\n",
        "\n",
        "</details>"
      ],
      "metadata": {
        "id": "EMJu4HJyk0fm"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<details>\n",
        "  <summary><b>Explanation: Dependencies</b> (click to expand)</summary>\n",
        "  \n",
        "### Package Explanations:\n",
        "\n",
        "#### trl:\n",
        "- **Description**: Transfer Learning Library (`trl`) is a Python library designed to facilitate transfer learning with pretrained language models. It offers utilities for fine-tuning and adapting these models to specific downstream tasks, providing a streamlined workflow for model adaptation.\n",
        "- **Usefulness**: `trl` is particularly valuable for tasks requiring domain-specific language generation, such as Java code generation. By leveraging pre-existing knowledge encoded in pretrained models like Llama3 8b, `trl` allows for efficient fine-tuning and customization, enhancing the model's performance on specialized tasks.\n",
        "\n",
        "#### peft:\n",
        "- **Description**: Parameter-Efficient Fine-Tuning (`peft`) methods enable efficient adaptation of large pretrained models to various downstream applications by only fine-tuning a small number of (extra) model parameters instead of all the model's parameters. This significantly decreases the computational and storage costs. Basically it freezes most of the pre-trained network's parameters.\n",
        "- **Usefulness**: For tasks like Java code generation, where generating semantically correct and contextually relevant code is crucial, `peft` can significantly improve model performance. By fine-tuning with parameter-efficient techniques, the model learns to generate code snippets that not only adhere to syntax rules but also exhibit contextually appropriate behavior, enhancing the overall quality of generated code.\n",
        "\n",
        "#### accelerate:\n",
        "- **Description**: Accelerate is a PyTorch-based library aimed at optimizing computations, particularly for deep learning tasks. It offers features such as distributed training, mixed precision, and optimized backends to improve performance and efficiency.\n",
        "- **Usefulness**: When fine-tuning large language models like Llama3 8b, `accelerate` can greatly accelerate the training process by leveraging GPU acceleration and other optimization techniques. This leads to faster convergence and reduced training times, making it an essential tool for training complex models efficiently.\n",
        "\n",
        "#### bitsandbytes:\n",
        "- **Description**: Bitsandbytes is a Python utility package providing tools for efficient data manipulation and conversion, particularly focusing on binary data and byte-level operations.\n",
        "- **Usefulness**: While not directly related to language model fine-tuning, `bitsandbytes` can be useful for preprocessing or post-processing tasks associated with code generation. It offers functionalities for handling binary data formats, encoding and decoding operations, and byte-level manipulations, which may be relevant in certain scenarios during the code generation pipeline.\n",
        "\n",
        "### Note:\n",
        "Understanding these dependencies and their functionalities can significantly aid in optimizing model performance, accelerating training, and effectively handling data throughout the fine-tuning process.\n",
        "\n",
        "</details>\n"
      ],
      "metadata": {
        "id": "P9Odfh-MiwcA"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2eSvM9zX_2d3"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "# Installs Unsloth, Xformers (Flash Attention) and all other packages!\n",
        "!pip install \"unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git\"\n",
        "!pip install --no-deps \"xformers<0.0.26\" trl peft accelerate bitsandbytes\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🔵 **Loading Pretrained Language Model and Tokenizer**\n",
        "<details>\n",
        "\n",
        "<summary>( click here to expand )</summary>\n",
        "\n",
        "This cell loads a pretrained language model and its associated tokenizer using the `unsloth` library. The model and tokenizer are essential components for generating Java code snippets using the Llama3 8b model.\n",
        "\n",
        "<br>\n",
        "\n",
        "<details>\n",
        "<summary><b>Parameters:</b></summary>\n",
        "\n",
        "- **max_seq_length**: Defines the maximum sequence length for input tokens. Setting this parameter higher allows the model to process longer sequences but may require more memory.\n",
        "- **dtype**: Specifies the data type for model parameters. By default (`None`), it is auto-detected. Using `Float16` is recommended for GPUs like Tesla T4 or V100, while `Bfloat16` is suitable for Ampere+ GPUs.\n",
        "- **load_in_4bit**: Controls whether to use 4-bit quantization to reduce memory usage during model loading. This can be set to `True` or `False` based on memory constraints.\n",
        "\n",
        "</details>\n",
        "\n",
        "<br>\n",
        "\n",
        "<details>\n",
        "<summary><b>Pre-Quantized Models:</b></summary>\n",
        "\n",
        "The code includes a list of pre-quantized models available for faster downloading and reduced out-of-memory errors. These models are pre-quantized to 4 bits and include variants such as `mistral-7b`, `llama-2-7b`, `gemma-7b`, and `gemma-2b`, among others. You can find more models at [Hugging Face's model hub](https://huggingface.co/unsloth).\n",
        "\n",
        "</details>\n",
        "\n",
        "<br>\n",
        "\n",
        "<details>\n",
        "<summary><b>Model Loading:</b></summary>\n",
        "\n",
        "The `FastLanguageModel.from_pretrained` method is used to load the pretrained language model and tokenizer. It takes several parameters:\n",
        "- `model_name`: Specifies the name of the pretrained model to be loaded.\n",
        "- `max_seq_length`: Sets the maximum sequence length for input tokens.\n",
        "- `dtype`: Specifies the data type for model parameters.\n",
        "- `load_in_4bit`: Controls whether to load the model with 4-bit quantization.\n",
        "\n",
        "</details>\n",
        "\n",
        "<br>\n",
        "\n",
        "<hr>\n",
        "\n",
        "### Note:\n",
        "\n",
        "Ensure that you have a stable internet connection to download the pretrained model and tokenizer. Additionally, verify that your Colab environment has sufficient memory to accommodate the chosen `max_seq_length` and model parameters.\n",
        "\n",
        "</details>"
      ],
      "metadata": {
        "id": "vL-F051HlJow"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 368,
          "referenced_widgets": [
            "28232af5d3be4c8ea8b68f546a49f2bc",
            "7e8d0a6399674fe0b9378ddffd88937c",
            "233c7fd1e34c45b2a1a255ef6ec90307",
            "2d9c59c21c4d46019f10a6d9f5f6ae90",
            "ccef44a7cae445a496152418f0d2311f",
            "3de1dee5298446faa6fea4f268637366",
            "bb3296bd0112457791b4ed9afd54d1a5",
            "2193f7ff7c114771b1b496838f6a5607",
            "95212b627d59419bba2b02fedd35532e",
            "06288157c00c4f1fbb3fbb1d37adf88d",
            "e34737523b054562814e042aaae379d8",
            "9b4b82c94c4e4a7c99e4dd7094bee5d0",
            "ebe6b84825d84de28a9f357cef6d4522",
            "94455b4c0eb4419c8067b4a329837f64",
            "f3155878fbe04f3c85ebdc988b675d0e",
            "5819959d32ca4bbd906c870532606b99",
            "80a938ad951b429ca2a3576d00772b5b",
            "e3d3b8fdba934716a7431cf7a465dbf0",
            "04358dad51334a689cae51ed44ee6a06",
            "3289113f7af04e10acbc77561de47b5d",
            "d6bca5f44eae45a9b7f934d6c585d080",
            "51afdd6eb81c412db279efa6816bc4e2",
            "6e35833fd3aa453ea31a3f6550183692",
            "bb7381d7edb74037bf5d88ab562902c0",
            "ae9cc1feea02451d94736f578de9701f",
            "5f5860527f1e4ddd940eb457292fc419",
            "a5b36bbd30cf4df695b1eaad26df5250",
            "bc3b8ab386834466ac2aba513a929394",
            "d47cd5e42ba14c83800f054537f514e9",
            "0fdbf9a0397d40e8af0e6c02f14d1c38",
            "d06971d2f77046c69734d16cfbb3da8b",
            "a06a708436454ed596a37efebaf1778f",
            "f6cff5c702a948c18ed0983e1d4f91dc",
            "262eb4368e1a4f5480ba32af521bf1d2",
            "96cd496d4303472ebb10eb1edee21c26",
            "98395a8757c94394ae58faba34e1b179",
            "5f2919e351ce408c8f8153116a3b0afa",
            "1e326847fa6d479e9e083b91fb843ded",
            "988bb12697934a82ad1514c3ded80256",
            "99458f2f93e84614b59c753e1ced901f",
            "79635ec19b0048bea30a0d13f8219120",
            "f7fd8baf7e874ece94997d032baf0ce1",
            "2019ca14204e443295eb5c9c38c5a091",
            "1e48ae5df19b465e97b83a8260d31e97",
            "f53d4c4c6f9f4aea9f49b5a3532ed271",
            "39e16675d3db4068b96aa344c5cc8bbf",
            "f62228d6ac3b4c4aa7c77d19f7484702",
            "04daf88197124d9aa0ebb0de1a0ed67b",
            "f2604458913847098804ba828486730a",
            "762acc66bea843c59b3ea85a4d6b9c2c",
            "10de0791998d47daac73be6961f13f5f",
            "dea2f90f313241199c5b661194038d8c",
            "ac0b9f7b899a4ff8b35946147eab3ed5",
            "f7840078f9c34a56acfefb9b5afb3652",
            "59f219c5389a42f0b7b664d4e4f8e4ea",
            "381e4f31251b4b3a926a9c07fb1b52bb",
            "a2a493fb29994fed9585aa09f1928141",
            "950ff329a72c48d694a05d83702ed69f",
            "0c32fe135b3b457b992750075c7a1b01",
            "88a7147034364034839e2f6a8d4a5233",
            "7467473370bc47e78f4e7340d8daf425",
            "5227181b2d69487b929514fe8927b8b8",
            "da477666afe74af1ba95a2cda678050f",
            "1b847de0a1014436955133ec5e91e90e",
            "2fb4dce4de564ac685224933da7daf13",
            "82a990971889427e898622b248c8892a"
          ]
        },
        "id": "QmUBVEnvCDJv",
        "outputId": "33e532ac-6a91-4c50-88a9-7cc99586762e"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "config.json:   0%|          | 0.00/1.14k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "28232af5d3be4c8ea8b68f546a49f2bc"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "==((====))==  Unsloth: Fast Llama patching release 2024.4\n",
            "   \\\\   /|    GPU: Tesla T4. Max memory: 14.748 GB. Platform = Linux.\n",
            "O^O/ \\_/ \\    Pytorch: 2.2.1+cu121. CUDA = 7.5. CUDA Toolkit = 12.1.\n",
            "\\        /    Bfloat16 = FALSE. Xformers = 0.0.25.post1. FA = False.\n",
            " \"-____-\"     Free Apache license: http://github.com/unslothai/unsloth\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Unused kwargs: ['_load_in_4bit', '_load_in_8bit', 'quant_method']. These kwargs are not used in <class 'transformers.utils.quantization_config.BitsAndBytesConfig'>.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "model.safetensors:   0%|          | 0.00/5.70G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9b4b82c94c4e4a7c99e4dd7094bee5d0"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/131 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6e35833fd3aa453ea31a3f6550183692"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/50.6k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "262eb4368e1a4f5480ba32af521bf1d2"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/9.09M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f53d4c4c6f9f4aea9f49b5a3532ed271"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "special_tokens_map.json:   0%|          | 0.00/449 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "381e4f31251b4b3a926a9c07fb1b52bb"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n",
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
          ]
        }
      ],
      "source": [
        "from unsloth import FastLanguageModel\n",
        "import torch\n",
        "\n",
        "# Define parameters\n",
        "max_seq_length = 2048  # Choose any! We auto support RoPE Scaling internally!\n",
        "dtype = None  # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+\n",
        "load_in_4bit = True  # Use 4bit quantization to reduce memory usage. Can be False.\n",
        "\n",
        "# List of 4bit pre-quantized models\n",
        "fourbit_models = [\n",
        "    \"unsloth/mistral-7b-bnb-4bit\",\n",
        "    \"unsloth/mistral-7b-instruct-v0.2-bnb-4bit\",\n",
        "    \"unsloth/llama-2-7b-bnb-4bit\",\n",
        "    \"unsloth/gemma-7b-bnb-4bit\",\n",
        "    \"unsloth/gemma-7b-it-bnb-4bit\",  # Instruct version of Gemma 7b\n",
        "    \"unsloth/gemma-2b-bnb-4bit\",\n",
        "    \"unsloth/gemma-2b-it-bnb-4bit\",  # Instruct version of Gemma 2b\n",
        "    \"unsloth/llama-3-8b-bnb-4bit\",   # [NEW] 15 Trillion token Llama-3\n",
        "]  # More models at https://huggingface.co/unsloth\n",
        "\n",
        "# Load the model and tokenizer\n",
        "model, tokenizer = FastLanguageModel.from_pretrained(\n",
        "    model_name=\"unsloth/llama-3-8b-bnb-4bit\",\n",
        "    max_seq_length=max_seq_length,\n",
        "    dtype=dtype,\n",
        "    load_in_4bit=load_in_4bit,\n",
        "    # token=\"hf_...\",  # Use one if using gated models like meta-llama/Llama-2-7b-hf\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🔵 **Enhancing Model with Parameter Efficient Fine-Tuning (PEFT) Technique**\n",
        "<details>\n",
        "<summary>( click here to expand )</summary>\n",
        "\n",
        "This cell utilizes the Parameter Efficient Fine-Tuning (PEFT) technique to enhance the pretrained language model (`model`). The PEFT technique optimizes certain parameters of the model to improve its performance and efficiency for specific tasks, such as Java code generation.\n",
        "\n",
        "<br>\n",
        "\n",
        "<details>\n",
        "<summary><b>PEFT Parameters:</b></summary>\n",
        "\n",
        "- **r**: Specifies the value of parameter `r`, which influences the fine-tuning process. It is recommended to choose a value greater than 0. Common values include 8, 16, 32, 64, or 128.\n",
        "\n",
        "- **target_modules**: Defines the target modules within the model architecture that will be fine-tuned using the PEFT technique. These modules include `[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"]`.\n",
        "\n",
        "- **lora_alpha**: Sets the value of the `lora_alpha` parameter, which controls the alpha parameter of LoRA (Local Rank Adaptation) technique. This parameter influences the fine-tuning process.\n",
        "\n",
        "- **lora_dropout**: Specifies the dropout rate for LoRA. A value of 0 indicates no dropout, while other values are supported but less optimized.\n",
        "\n",
        "- **bias**: Determines the bias type used in the model. The value `none` is recommended for optimization purposes.\n",
        "\n",
        "- **use_gradient_checkpointing**: Controls the usage of gradient checkpointing for memory efficiency during fine-tuning. Set to `unsloth` for optimal performance.\n",
        "\n",
        "- **random_state**: Sets the random seed for reproducibility during fine-tuning.\n",
        "\n",
        "- **use_rslora**: Indicates whether to utilize rank-stabilized LoRA. Currently set to `False`.\n",
        "\n",
        "- **loftq_config**: LoftQ configuration. Currently set to `None`.\n",
        "\n",
        "</details>\n",
        "\n",
        "<br>\n",
        "<hr>\n",
        "\n",
        "### Note:\n",
        "\n",
        "By applying the PEFT technique with carefully chosen parameters, the model's performance and efficiency can be significantly improved for the task of Java code generation. Ensure that the parameters are adjusted according to the specific requirements and constraints of your project.\n",
        "\n",
        "</details>"
      ],
      "metadata": {
        "id": "SXd9bTZd1aaL"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6bZsfBuZDeCL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c8f1add5-8caf-4d63-a78c-261055bb516c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Unsloth 2024.4 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.\n"
          ]
        }
      ],
      "source": [
        "# Enhance model using Parameter Efficient Fine-Tuning (PEFT) technique\n",
        "model = FastLanguageModel.get_peft_model(\n",
        "    model,\n",
        "    r=16,  # Choose any number > 0! Suggested 8, 16, 32, 64, 128\n",
        "    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
        "                    \"gate_proj\", \"up_proj\", \"down_proj\"],\n",
        "    lora_alpha=16,\n",
        "    lora_dropout=0,  # Supports any, but = 0 is optimized\n",
        "    bias=\"none\",  # Supports any, but = \"none\" is optimized\n",
        "    # Use gradient checkpointing for memory efficiency\n",
        "    use_gradient_checkpointing=\"unsloth\",  # True or \"unsloth\" for very long context\n",
        "    random_state=3407,\n",
        "    use_rslora=False,  # We support rank stabilized LoRA\n",
        "    loftq_config=None  # And LoftQ\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<details>\n",
        "<summary><b>🔴Code to load dataset from hugging face (click to expand)</b> OR press \"ctrl\" + \"M\" + \"J\" </summary>\n",
        "\n",
        "# alpaca_prompt = \"\"\"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
        "\n",
        "# ### Instruction:\n",
        "# {}\n",
        "\n",
        "# ### Input:\n",
        "# {}\n",
        "\n",
        "# ### Response:\n",
        "# {}\"\"\"\n",
        "\n",
        "# EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN\n",
        "# def formatting_prompts_func(examples):\n",
        "#     instructions = examples[\"instruction\"]\n",
        "#     inputs       = examples[\"input\"]\n",
        "#     outputs      = examples[\"output\"]\n",
        "#     texts = []\n",
        "#     for instruction, input, output in zip(instructions, inputs, outputs):\n",
        "#         # Must add EOS_TOKEN, otherwise your generation will go on forever!\n",
        "#         text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN\n",
        "#         texts.append(text)\n",
        "#     return { \"text\" : texts, }\n",
        "# pass\n",
        "\n",
        "# from datasets import load_dataset\n",
        "# dataset = load_dataset(\"yahma/alpaca-cleaned\", split = \"train\")\n",
        "# dataset = dataset.map(formatting_prompts_func, batched = True,)\n"
      ],
      "metadata": {
        "id": "sQrdD6wJr_jI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 🟢 JSON Dataset Structure Explanation\n",
        "\n",
        "<details>\n",
        "\n",
        "<summary> The dataset contains multiple JSON objects, each representing a programming task or problem statement along with its solution in Java code. (click here to expand) </summary>\n",
        "\n",
        "### JSON Object Structure\n",
        "\n",
        "Each JSON object consists of the following fields:\n",
        "\n",
        "- **Instruction**: A description of the task or problem statement.\n",
        "- **Input**: An optional field for input data (not used in this dataset).\n",
        "- **Output**: The solution or implementation in Java code.\n",
        "\n",
        "### Example JSON Object\n",
        "\n",
        "```json\n",
        "[\n",
        "{\n",
        "  \"instruction\": \"Write a Java class for a BankAccount...\",\n",
        "  \"input\": \"\",\n",
        "  \"output\": \"Java code representing a BankAccount class...\"\n",
        "},\n",
        "{\n",
        "  \"instruction\": \"Write a Java class for a BankAccount...\",\n",
        "  \"input\": \"\",\n",
        "  \"output\": \"Java code representing a BankAccount class...\"\n",
        "}\n",
        "]\n",
        "```\n",
        "\n",
        "</details>"
      ],
      "metadata": {
        "id": "TY9arzu5xlGu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🔵 **Data Preparation: Formatting Alpaca Prompts**\n",
        "<details>\n",
        "<summary> This cell prepares the Alpaca prompts by formatting them into a suitable format for training. It involves importing necessary libraries, defining the prompt template, and loading the dataset from a JSON file. ( click here to expand )</summary>\n",
        "\n",
        "<br>\n",
        "\n",
        "# 1. Importing Required Libraries:\n",
        "<details>\n",
        "<summary><b>click here to expand</b></summary>\n",
        "\n",
        "- The cell begins by importing the necessary libraries, including `json` for handling JSON files and `Dataset` from the `datasets` library for managing datasets efficiently.\n",
        "\n",
        "</details>\n",
        "\n",
        "# 2. Defining the Alpaca Prompt Template:\n",
        "<details>\n",
        "<summary><b>click here to expand</b></summary>\n",
        "\n",
        "- The Alpaca prompt template is defined as a multi-line string (`alpaca_prompt`). It includes placeholders for the instruction, input, and response parts of each prompt.\n",
        "\n",
        "</details>\n",
        "\n",
        "# 3. Defining the End-of-Sequence Token:\n",
        "<details>\n",
        "<summary><b>click here to expand</b></summary>\n",
        "\n",
        "- An end-of-sequence token (`EOS_TOKEN`) is defined using the `tokenizer.eos_token` attribute. This token is used to mark the end of each prompt.\n",
        "\n",
        "</details>\n",
        "\n",
        "# 4. Loading the Dataset:\n",
        "<details>\n",
        "<summary><b>click here to expand</b></summary>\n",
        "\n",
        "- The dataset is loaded from a JSON file located at `/content/your_dataset.json`. Make sure to update the file path according to your dataset's location. The JSON file contains a list of dictionaries, where each dictionary represents an example with keys for instruction, input, and output.\n",
        "\n",
        "</details>\n",
        "\n",
        "# 5. Formatting the Prompts:\n",
        "<details>\n",
        "<summary><b>click here to expand</b></summary>\n",
        "\n",
        "- The prompts are formatted by iterating over each example in the dataset list. The instruction, input, and output texts are extracted from each example, and the `alpaca_prompt` template is filled with these values. The end-of-sequence token is appended to each prompt, and the formatted prompt is added to the `texts` list.\n",
        "\n",
        "</details>\n",
        "\n",
        "# 6. Creating a Hugging Face Dataset:\n",
        "<details>\n",
        "<summary><b>click here to expand</b></summary>\n",
        "\n",
        "- Finally, a Hugging Face dataset is created from the `texts` list containing the formatted prompts. This dataset is now ready to be used for training or further processing.\n",
        "\n",
        "</details>\n",
        "\n",
        "<br>\n",
        "\n",
        "<hr>\n",
        "\n",
        "# Note:\n",
        "- Ensure that the dataset JSON file (`your_dataset.json`) is properly formatted and contains the required fields (instruction, input, output) for each example.\n",
        "- Additionally, verify that the `tokenizer` variable is defined elsewhere in the notebook to access the end-of-sequence token.\n",
        "</details>"
      ],
      "metadata": {
        "id": "-BHmV4B2qOzK"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LjY75GoYUCB8"
      },
      "outputs": [],
      "source": [
        "# Importing required libraries\n",
        "import json\n",
        "from datasets import Dataset\n",
        "\n",
        "# Define the Alpaca prompt template\n",
        "alpaca_prompt = \"\"\"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
        "\n",
        "### Instruction:\n",
        "{}\n",
        "\n",
        "### Input:\n",
        "{}\n",
        "\n",
        "### Response:\n",
        "{}\"\"\"\n",
        "\n",
        "# Define the end-of-sequence token\n",
        "EOS_TOKEN = tokenizer.eos_token  # Assuming 'tokenizer' is defined elsewhere\n",
        "\n",
        "# Load the dataset from the JSON file in Google Colab\n",
        "dataset_path = \"/content/your_dataset.json\" # Update with your file path\n",
        "with open(dataset_path, \"r\") as f:\n",
        "    dataset_list = json.load(f)\n",
        "\n",
        "# Format the prompts\n",
        "texts = []\n",
        "for example in dataset_list:\n",
        "    instruction = example[\"instruction\"]\n",
        "    input_text = example[\"input\"]\n",
        "    output_text = example[\"output\"]\n",
        "    text = alpaca_prompt.format(instruction, input_text, output_text) + EOS_TOKEN\n",
        "    texts.append(text)\n",
        "\n",
        "# Create a Hugging Face dataset\n",
        "dataset = Dataset.from_dict({\"text\": texts})\n",
        "\n",
        "# Now 'dataset' contains the formatted prompts ready to be used\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🔵 **Training and Testing**"
      ],
      "metadata": {
        "id": "AjulpPJ66hvy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\" Train\"></a>\n",
        "### **Training with Huggingface TRL's SFTTrainer**\n",
        "**bold text**\n",
        "In this code, we are using Huggingface's Transfer Representation Learning (TRL) library to train a model using the SFTTrainer.\n",
        "\n",
        "We set `num_train_epochs=1` for a full run by setting `max_steps=None`.\n",
        "<br>\n",
        "\n",
        "The code supports TRL's `DPOTrainer`.\n",
        "<br>\n",
        "\n",
        "More documentation on TRL's SFTTrainer can be found [here](https://huggingface.co/docs/trl/sft_trainer).\n"
      ],
      "metadata": {
        "id": "idAEIeSQ3xdS"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "95_Nn-89DhsL",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121,
          "referenced_widgets": [
            "20a363d1f0f342cc818f448a6036672b",
            "cf9e1cefa79f4c00887e810a115daeaa",
            "01fbccae0f75498db9a4fa81a0e8fade",
            "f6ef8ebde5d94500b3dc2f4c0f738cd2",
            "c6cb8767b678443b9d8cd1ab6da4663d",
            "ba78db64838648fca9db9aabad6a4bba",
            "cce8e21643874ead86d9a2f7e066db98",
            "96cacd2e468a471ca0a71c437876d875",
            "755e0c6fbe6c40149c6b63bca6007622",
            "37090da6dd7948ef961b49470f4522cd",
            "7e652fe25bc7437e8019d7bd062fd894"
          ]
        },
        "outputId": "f3c73460-f46f-4c11-8a28-0bbe7bdcc921"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/multiprocess/popen_fork.py:66: RuntimeWarning: os.fork() was called. os.fork() is incompatible with multithreaded code, and JAX is multithreaded, so this will likely lead to a deadlock.\n",
            "  self.pid = os.fork()\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map (num_proc=2):   0%|          | 0/9 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "20a363d1f0f342cc818f448a6036672b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "max_steps is given, it will override any value given in num_train_epochs\n"
          ]
        }
      ],
      "source": [
        "# @title\n",
        "from trl import SFTTrainer\n",
        "from transformers import TrainingArguments\n",
        "\n",
        "# Initialize the trainer with SFTTrainer\n",
        "trainer = SFTTrainer(\n",
        "    model = model,\n",
        "    tokenizer = tokenizer,\n",
        "    train_dataset = dataset,\n",
        "    dataset_text_field = \"text\",\n",
        "    max_seq_length = max_seq_length,\n",
        "    dataset_num_proc = 2,  # Number of processes to use while preparing the dataset\n",
        "    packing = False,  # Whether to use packing to make training faster for short sequences\n",
        "    args = TrainingArguments(\n",
        "        per_device_train_batch_size = 2,  # Batch size per GPU/CPU for training\n",
        "        gradient_accumulation_steps = 4,  # Number of updates steps to accumulate before performing a backward/update pass\n",
        "        warmup_steps = 5,  # Number of steps for the warmup phase\n",
        "        max_steps = 60,  # Maximum number of training steps\n",
        "        learning_rate = 2e-4,  # Initial learning rate for AdamW optimizer\n",
        "        fp16 = not torch.cuda.is_bf16_supported(),  # Whether to use FP16 (mixed precision) training\n",
        "        bf16 = torch.cuda.is_bf16_supported(),  # Whether to use BF16 training\n",
        "        logging_steps = 1,  # Log every X updates steps\n",
        "        optim = \"adamw_8bit\",  # Optimizer type\n",
        "        weight_decay = 0.01,  # Weight decay rate for AdamW optimizer\n",
        "        lr_scheduler_type = \"linear\",  # Learning rate scheduler type\n",
        "        seed = 3407,  # Random seed for reproducibility\n",
        "        output_dir = \"outputs\",  # Directory to save the output files\n",
        "    ),\n",
        ")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2ejIt2xSNKKp",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "992bb534-e78b-483f-8279-5d55ddcc26b3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "GPU = Tesla T4. Max memory = 14.748 GB.\n",
            "5.594 GB of memory reserved.\n"
          ]
        }
      ],
      "source": [
        "#@title Show current memory stats\n",
        "gpu_stats = torch.cuda.get_device_properties(0)\n",
        "start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n",
        "max_memory = round(gpu_stats.total_memory / 1024 / 1024 / 1024, 3)\n",
        "print(f\"GPU = {gpu_stats.name}. Max memory = {max_memory} GB.\")\n",
        "print(f\"{start_gpu_memory} GB of memory reserved.\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yqxqAZ7KJ4oL",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "cb9db5a8-08d7-44ba-dbbc-dc89212f1873"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "==((====))==  Unsloth - 2x faster free finetuning | Num GPUs = 1\n",
            "   \\\\   /|    Num examples = 9 | Num Epochs = 60\n",
            "O^O/ \\_/ \\    Batch size per device = 2 | Gradient Accumulation steps = 4\n",
            "\\        /    Total batch size = 8 | Total steps = 60\n",
            " \"-____-\"     Number of trainable parameters = 41,943,040\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='60' max='60' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [60/60 11:35, Epoch 48/60]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.823400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.852000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.806800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.717300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>0.673800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>6</td>\n",
              "      <td>0.598300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>0.462200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>0.401400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>0.281500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>0.263900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11</td>\n",
              "      <td>0.188200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>0.177800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>13</td>\n",
              "      <td>0.166300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>14</td>\n",
              "      <td>0.131400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>15</td>\n",
              "      <td>0.098000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>16</td>\n",
              "      <td>0.082200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>17</td>\n",
              "      <td>0.066800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>18</td>\n",
              "      <td>0.053700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>19</td>\n",
              "      <td>0.049100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>20</td>\n",
              "      <td>0.044700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>21</td>\n",
              "      <td>0.034300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>22</td>\n",
              "      <td>0.025600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>23</td>\n",
              "      <td>0.034000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>24</td>\n",
              "      <td>0.024100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>25</td>\n",
              "      <td>0.021700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>26</td>\n",
              "      <td>0.020200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>27</td>\n",
              "      <td>0.019400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>28</td>\n",
              "      <td>0.015300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>29</td>\n",
              "      <td>0.014000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>30</td>\n",
              "      <td>0.009500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>31</td>\n",
              "      <td>0.010700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>32</td>\n",
              "      <td>0.011100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>33</td>\n",
              "      <td>0.008700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>34</td>\n",
              "      <td>0.008200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>35</td>\n",
              "      <td>0.012000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>36</td>\n",
              "      <td>0.007000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>37</td>\n",
              "      <td>0.008100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>38</td>\n",
              "      <td>0.008100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>39</td>\n",
              "      <td>0.005600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>40</td>\n",
              "      <td>0.007900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>41</td>\n",
              "      <td>0.006700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>42</td>\n",
              "      <td>0.006500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>43</td>\n",
              "      <td>0.005400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>44</td>\n",
              "      <td>0.008100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>45</td>\n",
              "      <td>0.005500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>46</td>\n",
              "      <td>0.005500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>47</td>\n",
              "      <td>0.006200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>48</td>\n",
              "      <td>0.006600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>49</td>\n",
              "      <td>0.005800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>50</td>\n",
              "      <td>0.005700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>51</td>\n",
              "      <td>0.006400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>52</td>\n",
              "      <td>0.005500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>53</td>\n",
              "      <td>0.005600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>54</td>\n",
              "      <td>0.005500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>55</td>\n",
              "      <td>0.005700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>56</td>\n",
              "      <td>0.005600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>57</td>\n",
              "      <td>0.004900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>58</td>\n",
              "      <td>0.005500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>59</td>\n",
              "      <td>0.005400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>60</td>\n",
              "      <td>0.005400</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "trainer_stats = trainer.train()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pCqnaKmlO1U9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "db8b5b59-ea3d-4032-f9b8-3d78893e3b67"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "719.2952 seconds used for training.\n",
            "11.99 minutes used for training.\n",
            "Peak reserved memory = 7.756 GB.\n",
            "Peak reserved memory for training = 2.162 GB.\n",
            "Peak reserved memory % of max memory = 52.59 %.\n",
            "Peak reserved memory for training % of max memory = 14.66 %.\n"
          ]
        }
      ],
      "source": [
        "#@title Show final memory and time stats\n",
        "used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n",
        "used_memory_for_lora = round(used_memory - start_gpu_memory, 3)\n",
        "used_percentage = round(used_memory         /max_memory*100, 3)\n",
        "lora_percentage = round(used_memory_for_lora/max_memory*100, 3)\n",
        "print(f\"{trainer_stats.metrics['train_runtime']} seconds used for training.\")\n",
        "print(f\"{round(trainer_stats.metrics['train_runtime']/60, 2)} minutes used for training.\")\n",
        "print(f\"Peak reserved memory = {used_memory} GB.\")\n",
        "print(f\"Peak reserved memory for training = {used_memory_for_lora} GB.\")\n",
        "print(f\"Peak reserved memory % of max memory = {used_percentage} %.\")\n",
        "print(f\"Peak reserved memory for training % of max memory = {lora_percentage} %.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"Inference\"></a>\n",
        "### Inference\n",
        "Let's run the model! You can change the instruction and input - leave the output blank!"
      ],
      "metadata": {
        "id": "ekOmTR1hSNcr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# alpaca_prompt = Copied from above\n",
        "FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n",
        "inputs = tokenizer(\n",
        "[\n",
        "    alpaca_prompt.format(\n",
        "        \"Design a Java class for a BankCustomer that demonstrates encapsulation by encapsulating attributes like name and account number, inheritance by having subclasses like SavingAccountCustomer and LoanAccountCustomer, and association by associating customers with accounts.\", # instruction\n",
        "        \"\", # input\n",
        "        \"\", # output - leave this blank for generation!\n",
        "    )\n",
        "], return_tensors = \"pt\").to(\"cuda\")\n",
        "\n",
        "outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\n",
        "tokenizer.batch_decode(outputs)"
      ],
      "metadata": {
        "id": "kR3gIAX-SM2q",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "51a1919d-46ad-4a21-e54e-75d4eb713f58"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['<|begin_of_text|>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\nDesign a Java class for a BankCustomer that demonstrates encapsulation by encapsulating attributes like name and account number, inheritance by having subclasses like SavingAccountCustomer and LoanAccountCustomer, and association by associating customers with accounts.\\n\\n### Input:\\n\\n\\n### Response:\\n// Encapsulated attributes\\nprivate String name;\\nprivate String accountNumber;\\n\\n// Association with accounts (can be extended to further subclasses)\\nprivate Account account;\\n\\n// Constructor with arguments for name and account number\\npublic BankCustomer(String name, String accountNumber) {\\n  this.name = name;\\n  this.accountNumber = account']"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        " You can also use a `TextStreamer` for continuous inference - so you can see the generation token by token, instead of waiting the whole time!"
      ],
      "metadata": {
        "id": "CrSvZObor0lY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# alpaca_prompt = Copied from above\n",
        "FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n",
        "inputs = tokenizer(\n",
        "[\n",
        "    alpaca_prompt.format(\n",
        "        \"Design a Java class for a BankCustomer that demonstrates encapsulation by encapsulating attributes like name and account number, inheritance by having subclasses like SavingAccountCustomer and LoanAccountCustomer, and association by associating customers with accounts.\", # instruction\n",
        "        \"\", # input\n",
        "        \"\", # output - leave this blank for generation!\n",
        "    )\n",
        "], return_tensors = \"pt\").to(\"cuda\")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "text_streamer = TextStreamer(tokenizer)\n",
        "_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 500)"
      ],
      "metadata": {
        "id": "e2pEuRb1r2Vg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6ad540d5-68c1-4409-cccd-6cee04ae0c66"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<|begin_of_text|>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
            "\n",
            "### Instruction:\n",
            "Design a Java class for a BankCustomer that demonstrates encapsulation by encapsulating attributes like name and account number, inheritance by having subclasses like SavingAccountCustomer and LoanAccountCustomer, and association by associating customers with accounts.\n",
            "\n",
            "### Input:\n",
            "\n",
            "\n",
            "### Response:\n",
            "// Encapsulated attributes\n",
            "private String name;\n",
            "private String accountNumber;\n",
            "\n",
            "// Association with accounts (can be extended to further subclasses)\n",
            "private Account account;\n",
            "\n",
            "// Constructor with arguments for name and account number\n",
            "public BankCustomer(String name, String accountNumber) {\n",
            "  this.name = name;\n",
            "  this.accountNumber = accountNumber;\n",
            "  this.account = null; // Account not initialized by default\n",
            "}\n",
            "\n",
            "// Getter methods for encapsulated attributes\n",
            "public String getName() {\n",
            "  return name;\n",
            "}\n",
            "\n",
            "public String getAccountNumber() {\n",
            "  return accountNumber;\n",
            "}\n",
            "\n",
            "// Method to associate a customer with an account\n",
            "public void setAccount(Account account) {\n",
            "  this.account = account;\n",
            "}\n",
            "\n",
            "// Method to get the associated account (can return null if none)\n",
            "public Account getAccount() {\n",
            "  return account;\n",
            "}\n",
            "\n",
            "// Similar subclasses with specific methods (these can be extended further)\n",
            "class SavingAccountCustomer extends BankCustomer {\n",
            "  // Specific saving account methods here\n",
            "  public void deposit(double amount) {\n",
            "    account.deposit(amount);\n",
            "  }\n",
            "\n",
            "  public void withdraw(double amount) {\n",
            "    account.withdraw(amount);\n",
            "  }\n",
            "}\n",
            "\n",
            "class LoanAccountCustomer extends BankCustomer {\n",
            "  // Specific loan account methods here\n",
            "  public void applyForLoan(double amount) {\n",
            "    // Logic to apply for loan and associate with account (if approved)\n",
            "    if (approveLoan(amount)) {\n",
            "      account = new LoanAccount(amount);\n",
            "    }\n",
            "  }\n",
            "\n",
            "  public boolean approveLoan(double amount) {\n",
            "    // Logic to approve or reject loan request\n",
            "    return true; // Replace with appropriate logic\n",
            "  }\n",
            "}\n",
            "\n",
            "// Similar account class with deposit/withdraw methods (can be extended further)\n",
            "class Account {\n",
            "  private double balance;\n",
            "\n",
            "  public Account(double balance) {\n",
            "    this.balance = balance;\n",
            "  }\n",
            "\n",
            "  public void deposit(double amount) {\n",
            "    balance += amount;\n",
            "  }\n",
            "\n",
            "  public void withdraw(double amount) {\n",
            "    if (amount > 0 && amount <= balance) {\n",
            "      balance -= amount;\n",
            "    } else {\n",
            "      System.out.println(\"Invalid amount or insufficient balance\");\n",
            "    }\n",
            "  }\n",
            "\n",
            "  public double getBalance() {\n",
            "    return balance;\n",
            "  }\n",
            "}<|end_of_text|>\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"Save\"></a>\n",
        "### Saving, loading finetuned models\n",
        "To save the final model as LoRA adapters, either use Huggingface's `push_to_hub` for an online save or `save_pretrained` for a local save.\n",
        "\n",
        "**[NOTE]** This ONLY saves the LoRA adapters, and not the full model. To save to 16bit or GGUF, scroll down!"
      ],
      "metadata": {
        "id": "uMuVrWbjAzhc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.save_pretrained(\"code_lora_model\") # Local saving\n",
        "tokenizer.save_pretrained(\"code_lora_model\")\n",
        "# model.push_to_hub(\"rahulAkaVector/java_code_generator_finetuned_model\", token = \"hugging_face_access_token\") # Online saving\n",
        "# tokenizer.push_to_hub(\"rahulAkaVector/java_code_generator_finetuned_model\", token = \"hugging_face_access_token\") # Online saving"
      ],
      "metadata": {
        "id": "upcOlWe7A1vc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 98,
          "referenced_widgets": [
            "81e262f332b445218e9cbe546eb611d1",
            "0feb292c6f224c1db46767061a9f282d",
            "bef255785400431099921d39c653c91b",
            "71207f39827649ae83230bb676072599",
            "8c7b5809ddd942b487cd6e71a5e0b4d2",
            "ee0a79a47bfc48348e822ca1264d6095",
            "a96735f9f96d47fcae5f19e55f511675",
            "1be58a1f1d214f8d831abba0199b1522",
            "3585e347a63249408be643038c0f831f",
            "eae7dbf402104af2bc5ce17563075408",
            "43267bb0106848bab591247d29ff79b7",
            "165a473b9f644529932e61076402a491",
            "4274e3f5373e480ba6de02f4ef8294a5",
            "4f57a87dc2f04d3195705bd62ed87da9",
            "273322783c0c489299cd9b29b4c0b4c2",
            "a964228a35df4c5ebe824036842129a4",
            "0120feee542d4e59a805c87d3be2db85",
            "653ea8d7e9ab4fa2b10f40986bf661e3",
            "dc06fa653ce74a29ae8aa759d242c1ca",
            "6558581a26534f898cbc380f948d63ee",
            "5adb3bceee3644d3b5afc07dd96ff266",
            "0c62b469d7fa40e19831af67916a0dbc"
          ]
        },
        "outputId": "0aae2712-1e1c-4555-9353-c75035ae845e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "README.md:   0%|          | 0.00/27.0 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "81e262f332b445218e9cbe546eb611d1"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "adapter_model.safetensors:   0%|          | 0.00/168M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "165a473b9f644529932e61076402a491"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saved model to https://huggingface.co/rahulAkaVector/java_code_generator_finetuned_model\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now if you want to load the LoRA adapters we just saved for inference, set `False` to `True`:"
      ],
      "metadata": {
        "id": "AEEcJ4qfC7Lp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "if False:\n",
        "    from unsloth import FastLanguageModel\n",
        "    model, tokenizer = FastLanguageModel.from_pretrained(\n",
        "        model_name = \"code_lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n",
        "        max_seq_length = max_seq_length,\n",
        "        dtype = dtype,\n",
        "        load_in_4bit = load_in_4bit,\n",
        "    )\n",
        "    FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n",
        "\n",
        "# alpaca_prompt = You MUST copy from above!\n",
        "\n",
        "inputs = tokenizer(\n",
        "[\n",
        "    alpaca_prompt.format(\n",
        "        \"Write a Java class representing a SocialMediaPost that showcases encapsulation by encapsulating attributes like content and author, abstraction by providing methods for liking and commenting, and inheritance by having subclasses like TextPost and ImagePost.\", # instruction\n",
        "        \"\", # input\n",
        "        \"\", # output - leave this blank for generation!\n",
        "    )\n",
        "], return_tensors = \"pt\").to(\"cuda\")\n",
        "\n",
        "outputs = model.generate(**inputs, max_new_tokens = 500, use_cache = True)\n",
        "tokenizer.batch_decode(outputs)"
      ],
      "metadata": {
        "id": "MKX_XKs_BNZR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9e9226ad-bb24-46d0-d2cd-5493890d392b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['<|begin_of_text|>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\nWrite a Java class representing a SocialMediaPost that showcases encapsulation by encapsulating attributes like content and author, abstraction by providing methods for liking and commenting, and inheritance by having subclasses like TextPost and ImagePost.\\n\\n### Input:\\n\\n\\n### Response:\\npublic class SocialMediaPost {\\n  private String content;\\n  private String author;\\n  private int likes;\\n  private List<String> comments;\\n\\n  public SocialMediaPost(String content, String author) {\\n    this.content = content;\\n    this.author = author;\\n    this.likes = 0;\\n    this.comments = new ArrayList<>();\\n  }\\n\\n  public String getContent() {\\n    return content;\\n  }\\n\\n  public String getAuthor() {\\n    return author;\\n  }\\n\\n  public int getLikes() {\\n    return likes;\\n  }\\n\\n  public void like() {\\n    likes++;\\n  }\\n\\n  public void comment(String comment) {\\n    comments.add(comment);\\n  }\\n\\n  public List<String> getComments() {\\n    return Collections.unmodifiableList(comments);\\n  }\\n}\\n\\nclass TextPost extends SocialMediaPost {\\n  public TextPost(String content, String author) {\\n    super(content, author);\\n  }\\n}\\n\\nclass ImagePost extends SocialMediaPost {\\n  // Add specific methods for image posts here\\n\\n  public ImagePost(String content, String author) {\\n    super(content, author);\\n  }\\n}<|end_of_text|>']"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can also use Hugging Face's `AutoModelForPeftCausalLM`. Only use this if you do not have `unsloth` installed. It can be hopelessly slow, since `4bit` model downloading is not supported, and Unsloth's **inference is 2x faster**."
      ],
      "metadata": {
        "id": "QQMjaNrjsU5_"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "28232af5d3be4c8ea8b68f546a49f2bc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7e8d0a6399674fe0b9378ddffd88937c",
              "IPY_MODEL_233c7fd1e34c45b2a1a255ef6ec90307",
              "IPY_MODEL_2d9c59c21c4d46019f10a6d9f5f6ae90"
            ],
            "layout": "IPY_MODEL_ccef44a7cae445a496152418f0d2311f"
          }
        },
        "7e8d0a6399674fe0b9378ddffd88937c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3de1dee5298446faa6fea4f268637366",
            "placeholder": "​",
            "style": "IPY_MODEL_bb3296bd0112457791b4ed9afd54d1a5",
            "value": "config.json: 100%"
          }
        },
        "233c7fd1e34c45b2a1a255ef6ec90307": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2193f7ff7c114771b1b496838f6a5607",
            "max": 1140,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_95212b627d59419bba2b02fedd35532e",
            "value": 1140
          }
        },
        "2d9c59c21c4d46019f10a6d9f5f6ae90": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_06288157c00c4f1fbb3fbb1d37adf88d",
            "placeholder": "​",
            "style": "IPY_MODEL_e34737523b054562814e042aaae379d8",
            "value": " 1.14k/1.14k [00:00&lt;00:00, 47.4kB/s]"
          }
        },
        "ccef44a7cae445a496152418f0d2311f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3de1dee5298446faa6fea4f268637366": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bb3296bd0112457791b4ed9afd54d1a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2193f7ff7c114771b1b496838f6a5607": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "95212b627d59419bba2b02fedd35532e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "06288157c00c4f1fbb3fbb1d37adf88d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e34737523b054562814e042aaae379d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9b4b82c94c4e4a7c99e4dd7094bee5d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ebe6b84825d84de28a9f357cef6d4522",
              "IPY_MODEL_94455b4c0eb4419c8067b4a329837f64",
              "IPY_MODEL_f3155878fbe04f3c85ebdc988b675d0e"
            ],
            "layout": "IPY_MODEL_5819959d32ca4bbd906c870532606b99"
          }
        },
        "ebe6b84825d84de28a9f357cef6d4522": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_80a938ad951b429ca2a3576d00772b5b",
            "placeholder": "​",
            "style": "IPY_MODEL_e3d3b8fdba934716a7431cf7a465dbf0",
            "value": "model.safetensors: 100%"
          }
        },
        "94455b4c0eb4419c8067b4a329837f64": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_04358dad51334a689cae51ed44ee6a06",
            "max": 5702746405,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3289113f7af04e10acbc77561de47b5d",
            "value": 5702746405
          }
        },
        "f3155878fbe04f3c85ebdc988b675d0e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d6bca5f44eae45a9b7f934d6c585d080",
            "placeholder": "​",
            "style": "IPY_MODEL_51afdd6eb81c412db279efa6816bc4e2",
            "value": " 5.70G/5.70G [00:37&lt;00:00, 190MB/s]"
          }
        },
        "5819959d32ca4bbd906c870532606b99": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "80a938ad951b429ca2a3576d00772b5b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e3d3b8fdba934716a7431cf7a465dbf0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "04358dad51334a689cae51ed44ee6a06": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3289113f7af04e10acbc77561de47b5d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d6bca5f44eae45a9b7f934d6c585d080": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "51afdd6eb81c412db279efa6816bc4e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6e35833fd3aa453ea31a3f6550183692": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_bb7381d7edb74037bf5d88ab562902c0",
              "IPY_MODEL_ae9cc1feea02451d94736f578de9701f",
              "IPY_MODEL_5f5860527f1e4ddd940eb457292fc419"
            ],
            "layout": "IPY_MODEL_a5b36bbd30cf4df695b1eaad26df5250"
          }
        },
        "bb7381d7edb74037bf5d88ab562902c0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bc3b8ab386834466ac2aba513a929394",
            "placeholder": "​",
            "style": "IPY_MODEL_d47cd5e42ba14c83800f054537f514e9",
            "value": "generation_config.json: 100%"
          }
        },
        "ae9cc1feea02451d94736f578de9701f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0fdbf9a0397d40e8af0e6c02f14d1c38",
            "max": 131,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d06971d2f77046c69734d16cfbb3da8b",
            "value": 131
          }
        },
        "5f5860527f1e4ddd940eb457292fc419": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a06a708436454ed596a37efebaf1778f",
            "placeholder": "​",
            "style": "IPY_MODEL_f6cff5c702a948c18ed0983e1d4f91dc",
            "value": " 131/131 [00:00&lt;00:00, 5.12kB/s]"
          }
        },
        "a5b36bbd30cf4df695b1eaad26df5250": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bc3b8ab386834466ac2aba513a929394": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d47cd5e42ba14c83800f054537f514e9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0fdbf9a0397d40e8af0e6c02f14d1c38": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d06971d2f77046c69734d16cfbb3da8b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a06a708436454ed596a37efebaf1778f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f6cff5c702a948c18ed0983e1d4f91dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "262eb4368e1a4f5480ba32af521bf1d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_96cd496d4303472ebb10eb1edee21c26",
              "IPY_MODEL_98395a8757c94394ae58faba34e1b179",
              "IPY_MODEL_5f2919e351ce408c8f8153116a3b0afa"
            ],
            "layout": "IPY_MODEL_1e326847fa6d479e9e083b91fb843ded"
          }
        },
        "96cd496d4303472ebb10eb1edee21c26": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_988bb12697934a82ad1514c3ded80256",
            "placeholder": "​",
            "style": "IPY_MODEL_99458f2f93e84614b59c753e1ced901f",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "98395a8757c94394ae58faba34e1b179": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_79635ec19b0048bea30a0d13f8219120",
            "max": 50599,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f7fd8baf7e874ece94997d032baf0ce1",
            "value": 50599
          }
        },
        "5f2919e351ce408c8f8153116a3b0afa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2019ca14204e443295eb5c9c38c5a091",
            "placeholder": "​",
            "style": "IPY_MODEL_1e48ae5df19b465e97b83a8260d31e97",
            "value": " 50.6k/50.6k [00:00&lt;00:00, 2.93MB/s]"
          }
        },
        "1e326847fa6d479e9e083b91fb843ded": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "988bb12697934a82ad1514c3ded80256": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "99458f2f93e84614b59c753e1ced901f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "79635ec19b0048bea30a0d13f8219120": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f7fd8baf7e874ece94997d032baf0ce1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2019ca14204e443295eb5c9c38c5a091": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1e48ae5df19b465e97b83a8260d31e97": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f53d4c4c6f9f4aea9f49b5a3532ed271": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_39e16675d3db4068b96aa344c5cc8bbf",
              "IPY_MODEL_f62228d6ac3b4c4aa7c77d19f7484702",
              "IPY_MODEL_04daf88197124d9aa0ebb0de1a0ed67b"
            ],
            "layout": "IPY_MODEL_f2604458913847098804ba828486730a"
          }
        },
        "39e16675d3db4068b96aa344c5cc8bbf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_762acc66bea843c59b3ea85a4d6b9c2c",
            "placeholder": "​",
            "style": "IPY_MODEL_10de0791998d47daac73be6961f13f5f",
            "value": "tokenizer.json: 100%"
          }
        },
        "f62228d6ac3b4c4aa7c77d19f7484702": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dea2f90f313241199c5b661194038d8c",
            "max": 9085698,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ac0b9f7b899a4ff8b35946147eab3ed5",
            "value": 9085698
          }
        },
        "04daf88197124d9aa0ebb0de1a0ed67b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f7840078f9c34a56acfefb9b5afb3652",
            "placeholder": "​",
            "style": "IPY_MODEL_59f219c5389a42f0b7b664d4e4f8e4ea",
            "value": " 9.09M/9.09M [00:00&lt;00:00, 19.8MB/s]"
          }
        },
        "f2604458913847098804ba828486730a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "762acc66bea843c59b3ea85a4d6b9c2c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "10de0791998d47daac73be6961f13f5f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "dea2f90f313241199c5b661194038d8c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ac0b9f7b899a4ff8b35946147eab3ed5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f7840078f9c34a56acfefb9b5afb3652": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "59f219c5389a42f0b7b664d4e4f8e4ea": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "381e4f31251b4b3a926a9c07fb1b52bb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a2a493fb29994fed9585aa09f1928141",
              "IPY_MODEL_950ff329a72c48d694a05d83702ed69f",
              "IPY_MODEL_0c32fe135b3b457b992750075c7a1b01"
            ],
            "layout": "IPY_MODEL_88a7147034364034839e2f6a8d4a5233"
          }
        },
        "a2a493fb29994fed9585aa09f1928141": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7467473370bc47e78f4e7340d8daf425",
            "placeholder": "​",
            "style": "IPY_MODEL_5227181b2d69487b929514fe8927b8b8",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "950ff329a72c48d694a05d83702ed69f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_da477666afe74af1ba95a2cda678050f",
            "max": 449,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1b847de0a1014436955133ec5e91e90e",
            "value": 449
          }
        },
        "0c32fe135b3b457b992750075c7a1b01": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2fb4dce4de564ac685224933da7daf13",
            "placeholder": "​",
            "style": "IPY_MODEL_82a990971889427e898622b248c8892a",
            "value": " 449/449 [00:00&lt;00:00, 34.9kB/s]"
          }
        },
        "88a7147034364034839e2f6a8d4a5233": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7467473370bc47e78f4e7340d8daf425": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5227181b2d69487b929514fe8927b8b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "da477666afe74af1ba95a2cda678050f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1b847de0a1014436955133ec5e91e90e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2fb4dce4de564ac685224933da7daf13": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "82a990971889427e898622b248c8892a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "20a363d1f0f342cc818f448a6036672b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_cf9e1cefa79f4c00887e810a115daeaa",
              "IPY_MODEL_01fbccae0f75498db9a4fa81a0e8fade",
              "IPY_MODEL_f6ef8ebde5d94500b3dc2f4c0f738cd2"
            ],
            "layout": "IPY_MODEL_c6cb8767b678443b9d8cd1ab6da4663d"
          }
        },
        "cf9e1cefa79f4c00887e810a115daeaa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ba78db64838648fca9db9aabad6a4bba",
            "placeholder": "​",
            "style": "IPY_MODEL_cce8e21643874ead86d9a2f7e066db98",
            "value": "Map (num_proc=2): 100%"
          }
        },
        "01fbccae0f75498db9a4fa81a0e8fade": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_96cacd2e468a471ca0a71c437876d875",
            "max": 9,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_755e0c6fbe6c40149c6b63bca6007622",
            "value": 9
          }
        },
        "f6ef8ebde5d94500b3dc2f4c0f738cd2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_37090da6dd7948ef961b49470f4522cd",
            "placeholder": "​",
            "style": "IPY_MODEL_7e652fe25bc7437e8019d7bd062fd894",
            "value": " 9/9 [00:01&lt;00:00,  7.21 examples/s]"
          }
        },
        "c6cb8767b678443b9d8cd1ab6da4663d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ba78db64838648fca9db9aabad6a4bba": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cce8e21643874ead86d9a2f7e066db98": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "96cacd2e468a471ca0a71c437876d875": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "755e0c6fbe6c40149c6b63bca6007622": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "37090da6dd7948ef961b49470f4522cd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7e652fe25bc7437e8019d7bd062fd894": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "81e262f332b445218e9cbe546eb611d1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0feb292c6f224c1db46767061a9f282d",
              "IPY_MODEL_bef255785400431099921d39c653c91b",
              "IPY_MODEL_71207f39827649ae83230bb676072599"
            ],
            "layout": "IPY_MODEL_8c7b5809ddd942b487cd6e71a5e0b4d2"
          }
        },
        "0feb292c6f224c1db46767061a9f282d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ee0a79a47bfc48348e822ca1264d6095",
            "placeholder": "​",
            "style": "IPY_MODEL_a96735f9f96d47fcae5f19e55f511675",
            "value": "README.md: 100%"
          }
        },
        "bef255785400431099921d39c653c91b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1be58a1f1d214f8d831abba0199b1522",
            "max": 27,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3585e347a63249408be643038c0f831f",
            "value": 27
          }
        },
        "71207f39827649ae83230bb676072599": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eae7dbf402104af2bc5ce17563075408",
            "placeholder": "​",
            "style": "IPY_MODEL_43267bb0106848bab591247d29ff79b7",
            "value": " 27.0/27.0 [00:00&lt;00:00, 2.06kB/s]"
          }
        },
        "8c7b5809ddd942b487cd6e71a5e0b4d2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ee0a79a47bfc48348e822ca1264d6095": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a96735f9f96d47fcae5f19e55f511675": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1be58a1f1d214f8d831abba0199b1522": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3585e347a63249408be643038c0f831f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "eae7dbf402104af2bc5ce17563075408": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "43267bb0106848bab591247d29ff79b7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "165a473b9f644529932e61076402a491": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4274e3f5373e480ba6de02f4ef8294a5",
              "IPY_MODEL_4f57a87dc2f04d3195705bd62ed87da9",
              "IPY_MODEL_273322783c0c489299cd9b29b4c0b4c2"
            ],
            "layout": "IPY_MODEL_a964228a35df4c5ebe824036842129a4"
          }
        },
        "4274e3f5373e480ba6de02f4ef8294a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0120feee542d4e59a805c87d3be2db85",
            "placeholder": "​",
            "style": "IPY_MODEL_653ea8d7e9ab4fa2b10f40986bf661e3",
            "value": "adapter_model.safetensors: 100%"
          }
        },
        "4f57a87dc2f04d3195705bd62ed87da9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dc06fa653ce74a29ae8aa759d242c1ca",
            "max": 167832240,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6558581a26534f898cbc380f948d63ee",
            "value": 167832240
          }
        },
        "273322783c0c489299cd9b29b4c0b4c2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5adb3bceee3644d3b5afc07dd96ff266",
            "placeholder": "​",
            "style": "IPY_MODEL_0c62b469d7fa40e19831af67916a0dbc",
            "value": " 168M/168M [00:06&lt;00:00, 28.3MB/s]"
          }
        },
        "a964228a35df4c5ebe824036842129a4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0120feee542d4e59a805c87d3be2db85": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "653ea8d7e9ab4fa2b10f40986bf661e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "dc06fa653ce74a29ae8aa759d242c1ca": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6558581a26534f898cbc380f948d63ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5adb3bceee3644d3b5afc07dd96ff266": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0c62b469d7fa40e19831af67916a0dbc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}