kite-4.2-14m / README.md
qikp's picture
Upload README.md with huggingface_hub
5635e57 verified
|
Raw History Blame Contribute Delete
797 Bytes
metadata
license: cc0-1.0
datasets:
  - qikp/small-data-2
language:
  - en
pipeline_tag: text-generation
library_name: transformers

Kite

🎉 You are looking at Kite 4.2, which is using a more optimized dataset!

Kite is a small, trained, 14 million parameter language model.

Training

It was trained on a tokenized version of qikp/small-data-2, which is a mixture of various datasets, using 1 epoch, 32 batch size, 1.5e-4 learning rate, and the pika 4 tokenizer.

Also, evaluation on a tokenized and truncated byunggill/gpt-2-output was done during training.

Limitations

Due to its size, the model is not suitable for production workloads.