██╗ ██╗ ██╗███╗ ███╗██╗ ██████╗██╗ ██╗ █████╗ ████████╗███████╗
██║ ██║ ██║████╗ ████║██║██╔════╝██║ ██║██╔══██╗╚══██╔══╝██╔════╝
██║ ██║ ██║██╔████╔██║██║██║ ███████║███████║ ██║ ███████╗
██║ ██║ ██║██║╚██╔╝██║██║██║ ██╔══██║██╔══██║ ██║ ╚════██║
███████╗╚██████╔╝██║ ╚═╝ ██║██║╚██████╗██║ ██║██║ ██║ ██║ ███████║
╚══════╝ ╚═════╝ ╚═╝ ╚═╝╚═╝ ╚═════╝╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ ╚══════╝
LumiChats-Instruct-14B-LoRA
Fine-tuned Phi-4 for Next-Generation Conversational AI
Built by LumiChats
⚠️ Critical Naming Clarification: This model is 14.7 Billion parameters. The name "Phi-4" refers to Microsoft's 4th generation Phi series — the "4" is a version number, NOT a parameter count. Do not confuse with Phi-3-mini (3.8B). This is a full 14B model, confirmed by the training output:
Trainable parameters = 65,536,000 of 14,725,043,200.
🔶 What is LumiChats?
LumiChats is a conversational AI company building intelligent, context-aware dialogue systems for the real world. We work at the intersection of efficient fine-tuning, applied NLP, and product-first AI — making powerful language models practical and accessible for businesses, developers, and educators.
LumiChats-Instruct-14B-LoRA is our first public model release — a fine-tuned version of Microsoft's Phi-4 14B, purpose-built for rich, multi-turn conversational AI.
🔶 Base Phi-4 vs LumiChats Fine-Tuned — What Changed?
This is the core question: why use this model over the raw base Phi-4?
| Capability | 🔲 Base Phi-4 (microsoft/phi-4) | 🔶 LumiChats-Instruct-14B-LoRA |
|---|---|---|
| Multi-turn conversation | Generic, not optimized | ✅ Specifically fine-tuned for dialogue |
| Instruction following | Moderate pretrain behavior | ✅ Reinforced via response-only training |
| Chat template | Requires manual configuration | ✅ Phi-4 template pre-applied & verified |
| Training data | 9.8T token web/book crawl | ✅ 99,990 curated conversational samples |
| Training objective | Predict every token equally | ✅ Only trains on assistant responses — no prompt memorization |
| Response structure | Unstructured completions | ✅ Consistent, well-formed reply format |
| Deployment readiness | Raw — needs post-processing | ✅ Drop-in ready for chat applications |
| Base knowledge preserved | N/A | ✅ 99.55% of weights untouched — full Phi-4 intelligence retained |
The Simple Analogy:
🔲 Base Phi-4 = A brilliant PhD graduate who has read everything but never held a conversation in a structured job.
🔶 LumiChats-Instruct-14B-LoRA = That same PhD, now trained specifically to hold clear, helpful, structured conversations — same intelligence, purpose-built delivery.
🔶 Model Card
| Property | Details |
|---|---|
| Model Name | LumiChats-Instruct-14B-LoRA |
| Developed By | LumiChats |
| Base Model | microsoft/phi-4 |
| Total Parameters | 14,725,043,200 (14.7 Billion) |
| Architecture | Dense Decoder-only Transformer — 40 layers |
| Fine-tuning Method | LoRA (Low-Rank Adaptation) via PEFT |
| Trainable Parameters | 65,536,000 (0.45% of 14.7B) |
| LoRA Rank | r = 16, alpha = 16 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Quantization | 4-bit NF4 (bitsandbytes) |
| Context Length | 2048 tokens |
| Chat Format | Phi-4 (<|im_start|> / <|im_sep|> / <|im_end|>) |
| Language | English |
| License | MIT |
🔶 Training Details
Dataset
| Property | Value |
|---|---|
| Dataset | FineTome-100k by Maxime Labonne |
| Total Samples | 100,000 → 99,990 after quality filtering |
| Format | ShareGPT → HuggingFace (role, content) multi-turn |
| Topics Covered | Reasoning, Science, Mathematics, Coding, Logic, General Q&A |
| Training Objective | Response-only masking — loss computed on assistant turns only |
Training Configuration
# LoRA
r = 16, lora_alpha = 16, lora_dropout = 0
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"]
# Optimizer
optim = "adamw_8bit"
learning_rate = 2e-4
lr_scheduler_type = "linear"
warmup_steps = 5
weight_decay = 0.001
# Batch
per_device_train_batch_size = 2
gradient_accumulation_steps = 4 # effective batch = 8
max_steps = 30
seed = 3407
Hardware & Runtime Stats
GPU Tesla T4 — 14.563 GB VRAM
Training Duration 19.28 minutes (30 steps)
Peak VRAM Used 13.242 GB (90.9% of T4)
LoRA Training Overhead 2.515 GB only
Trainable % of Model 0.45%
Framework Unsloth 2026.2.1 + TRL 0.22.2
Platform Google Colab
🔶 Phi-4 Benchmark Performance
These are Microsoft's published benchmarks for the Phi-4 base. Fine-tuning with LumiChats preserves all base knowledge — these scores apply to this model's underlying capability.
| Benchmark | What It Tests | Phi-4 14B | GPT-4o-mini | Llama-3.3 70B | GPT-4o |
|---|---|---|---|---|---|
| MMLU | General Knowledge | 84.8 | 81.8 | 86.3 | 88.1 |
| GPQA | Graduate Science | 56.1 🏆 | 40.9 | 49.1 | 50.6 |
| MATH | Competition Math | 80.4 | 73.0 | 66.3 | 74.6 |
| HumanEval | Code Generation | 82.6 | 86.2 | 78.9 | 90.6 |
| MGSM | Multilingual Math | 80.6 | 86.5 | 89.1 | 90.4 |
| DROP | Reasoning | 75.5 | 79.3 | 90.2 | 80.9 |
🔶 Phi-4 outperforms GPT-4o on Graduate-level Science (GPQA) despite being ~14x smaller in parameter count.
🔶 Quick Start
Install
pip install unsloth transformers bitsandbytes accelerate peft
Run Inference
from unsloth import FastLanguageModel
from transformers import TextStreamer
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "adityakum667388/LumiChats-Instruct-14B_lora",
max_seq_length = 2048,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model)
messages = [
{"role": "user", "content": "Explain quantum entanglement in simple terms."}
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize = True,
add_generation_prompt = True,
return_tensors = "pt",
).to("cuda")
streamer = TextStreamer(tokenizer, skip_prompt=True)
_ = model.generate(
input_ids = inputs,
streamer = streamer,
max_new_tokens = 256,
temperature = 1.5,
min_p = 0.1,
use_cache = True,
)
Chat Template
<|im_start|>system<|im_sep|>
You are a helpful AI assistant built by LumiChats.<|im_end|>
<|im_start|>user<|im_sep|>
Your question here<|im_end|>
<|im_start|>assistant<|im_sep|>
🔶 Known Limitations
- 30-step demonstration fine-tune — A full epoch (~12,500 steps) will yield significantly stronger conversational alignment. This release demonstrates the pipeline; production training is ongoing.
- English-primary — Multilingual performance is inherited from base Phi-4 (~8% multilingual data). Not optimized for non-English use.
- 2048 token context — Base Phi-4 supports 16K tokens; this fine-tune is limited to 2048. Will be extended in future releases.
- Factual hallucination — Like all LLMs, verify outputs for high-stakes decisions.
🔶 LumiChats Roadmap
| Status | Item |
|---|---|
| ✅ | LoRA adapter — public release |
| 🔜 | Full epoch production fine-tune |
| 🔜 | GGUF quantized versions (Q4_K_M, Q8_0) for local deployment |
| 🔜 | Domain-specific fine-tunes (education, enterprise, coding) |
| 🔜 | Extended 16K context version |
| 🔜 | Multilingual conversational variants |
🔶 Citation
@misc{lumichats-instruct-14b-lora-2026,
author = {LumiChats},
title = {LumiChats-Instruct-14B-LoRA: Fine-tuned Microsoft Phi-4 for Conversational AI},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/adityakum667388/LumiChats-Instruct-14B_lora}
}
🔶 Acknowledgements
- Microsoft Research — Phi-4 14B base model
- Maxime Labonne — FineTome-100k conversational dataset
- Unsloth — Fine-tuning acceleration framework
- Google Colab — Training infrastructure
- Downloads last month
- 1
Model tree for adityakum667388/LumiChats-Instruct-4B_lora
Base model
microsoft/phi-4