---
base_model: google/gemma-3-4b-it
datasets:
- Siluni/sinhala-vqa-dataset
language:
- si
library_name: peft
tags:
- sinhala
- continued-pretraining
- gemma
- qlora
- low-resource
---

# gemma3-4b-cpt

CPT-only (Continued Pre-Training) adapter for Gemma-3-4B-IT on the MADLAD-400 Sinhala corpus.
This adapter does **not** perform VQA on its own. It is intended to be used as the first stage
of the sequential CPT → VQA pipeline together with [`Siluni/gemma3-4b-cpt-vqa-33k`](https://huggingface.co/Siluni/gemma3-4b-cpt-vqa-33k).

- **Base model**: [google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it)
- **Experiment**: Group 3 — Sequential CPT stage
- **CPT corpus**: MADLAD-400 Sinhala (~293M words)
- **Method**: QLoRA (4-bit NF4, LoRA rank 16, alpha 32)

## ⚠️ Sequential Loading Required

This adapter must be loaded together with the VQA adapter and combined before inference.
See [`Siluni/gemma3-4b-cpt-vqa-33k`](https://huggingface.co/Siluni/gemma3-4b-cpt-vqa-33k) for the full loading instructions.

## Citation

```bibtex
@misc{keerthiratne2025sinhalavqa,
  title        = {Benchmarking and Adapting Compact Multimodal Models for Sinhala Visual Question Answering},
  author       = {Keerthiratne, Siluni and Weerasinghe, Ruvan and Sumanathilaka, Deshan},
  year         = {2025},
  institution  = {Informatics Institute of Technology / Robert Gordon University},
}
```