YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other
🌊 SeaLLM-7B-v2.5 — LoRA Adapter (EN→ID, Small-Sample-2, ≈1 K samples)
This repository provides a LoRA fine-tuned adapter for
SeaLLMs/SeaLLM-7B-v2.5,
trained on about 1,000 English–Indonesian translation pairs.
It is part of WRI Indonesia’s exploration of efficient domain adaptation for scientific and environmental text translation using QLoRA.
🧠 Model Details
| Property | Value |
|---|---|
| Base Model | SeaLLMs/SeaLLM-7B-v2.5 |
| Adapter Type | LoRA (via PEFT) |
| Training Method | QLoRA (4-bit quantized fine-tuning) |
| Context Length | 896 tokens |
| Training Size | ≈ 1 000 parallel examples |
| Domain | Environment, forestry, climate policy |
| Compute | Single GPU (NVIDIA L40) |
| Training Time | ~30 minutes |
| Epochs | 3 |
| Experiment Name | small-sample-2 |
| Frameworks | 🤗 Transformers / PEFT / Datasets |
⚙️ Training Configuration
peft_type: LORA
r: 32
lora_alpha: 16
lora_dropout: 0.05
target_modules:
- q_proj
- v_proj
- k_proj
- o_proj
- gate_proj
- up_proj
- down_proj
task_type: CAUSAL_LM
max_seq_length: 896
num_train_epochs: 3
per_device_train_batch_size: 2
gradient_accumulation_steps: 8
learning_rate: 2e-4
scheduler: cosine
warmup_ratio: 0.03
bf16: true
save_strategy: epoch
evaluation_strategy: epoch
early_stopping_patience: 2
load_best_model_at_end: true
- Downloads last month
- 143
Model tree for wriindonesia/seallm-7b-finetuned-small
Base model
SeaLLMs/SeaLLM-7B-v2.5