mistral-beer-finetune3

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7785

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 5
• training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss
1.3701	0.1390	50	1.1570
1.1755	0.2780	100	1.1248
1.184	0.4170	150	1.0981
1.1542	0.5559	200	1.0774
1.1433	0.6949	250	1.0596
1.1425	0.8339	300	1.0444
1.1196	0.9729	350	1.0216
1.027	1.1140	400	0.9955
0.9985	1.2530	450	0.9732
1.0288	1.3919	500	0.9510
1.027	1.5309	550	0.9297
1.0503	1.6699	600	0.9154
1.0372	1.8089	650	0.8945
1.0015	1.9479	700	0.8769
0.9141	2.0890	750	0.8426
0.8775	2.2279	800	0.8208
0.834	2.3669	850	0.8065
0.8735	2.5059	900	0.7897
0.8189	2.6449	950	0.7820
0.8489	2.7839	1000	0.7785

Framework versions

• PEFT 0.14.0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1