llama-duo/synth_summarize_dataset_dedup
Viewer โข Updated โข 880k โข 50 โข 2
How to use llama-duo/gemma7b-summarize-claude3sonnet-128k with PEFT:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("google/gemma-7b")
model = PeftModel.from_pretrained(base_model, "llama-duo/gemma7b-summarize-claude3sonnet-128k")This model is a fine-tuned version of google/gemma-7b on the llama-duo/synth_summarize_dataset_dedup dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.0247 | 0.9983 | 301 | 2.5396 |
| 0.9144 | 2.0 | 603 | 2.4619 |
| 0.8616 | 2.9983 | 904 | 2.4534 |
| 0.8075 | 4.0 | 1206 | 2.4530 |
| 0.7724 | 4.9983 | 1507 | 2.4674 |
| 0.7571 | 6.0 | 1809 | 2.4825 |
| 0.7389 | 6.9983 | 2110 | 2.5147 |
| 0.7223 | 8.0 | 2412 | 2.5135 |
| 0.7178 | 8.9983 | 2713 | 2.5209 |
| 0.7148 | 9.9834 | 3010 | 2.5222 |
Base model
google/gemma-7b