--- license: cc0-1.0 datasets: - qikp/small-data-2 language: - en pipeline_tag: text-generation library_name: transformers --- # Kite 🎉 You are looking at Kite 4.2, which is using a more optimized dataset! Kite is a small, trained, 14 million parameter language model. ## Training It was trained on a tokenized version of [qikp/small-data-2](https://huggingface.co/datasets/qikp/small-data-2), which is a mixture of various datasets, using 1 epoch, 32 batch size, 1.5e-4 learning rate, and the [pika 4 tokenizer](https://huggingface.co/qikp/pika-4). Also, evaluation on a tokenized and truncated [byunggill/gpt-2-output](https://huggingface.co/datasets/byunggill/gpt-2-output) was done during training. ## Limitations Due to its size, the model is not suitable for production workloads.