metadata
base_model: google/gemma-3-4b-it
datasets:
- Siluni/sinhala-vqa-dataset
language:
- si
library_name: peft
tags:
- sinhala
- continued-pretraining
- gemma
- qlora
- low-resource
gemma3-4b-cpt
CPT-only (Continued Pre-Training) adapter for Gemma-3-4B-IT on the MADLAD-400 Sinhala corpus.
This adapter does not perform VQA on its own. It is intended to be used as the first stage
of the sequential CPT → VQA pipeline together with Siluni/gemma3-4b-cpt-vqa-33k.
- Base model: google/gemma-3-4b-it
- Experiment: Group 3 — Sequential CPT stage
- CPT corpus: MADLAD-400 Sinhala (~293M words)
- Method: QLoRA (4-bit NF4, LoRA rank 16, alpha 32)
⚠️ Sequential Loading Required
This adapter must be loaded together with the VQA adapter and combined before inference.
See Siluni/gemma3-4b-cpt-vqa-33k for the full loading instructions.
Citation
@misc{keerthiratne2025sinhalavqa,
title = {Benchmarking and Adapting Compact Multimodal Models for Sinhala Visual Question Answering},
author = {Keerthiratne, Siluni and Weerasinghe, Ruvan and Sumanathilaka, Deshan},
year = {2025},
institution = {Informatics Institute of Technology / Robert Gordon University},
}