Add Myosotis-1.5-base Preview
#195
by FlameF0X - opened
evaluation done on Myosotis-1.5-base-Preview by using LM Eval
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 0 | acc | ↑ | 0.1869 | ± | 0.0114 |
| none | 0 | acc_norm | ↑ | 0.2193 | ± | 0.0121 | ||
| arc_easy | 1 | none | 0 | acc | ↑ | 0.4398 | ± | 0.0102 |
| none | 0 | acc_norm | ↑ | 0.4036 | ± | 0.0101 | ||
| hellaswag | 1 | none | 0 | acc | ↑ | 0.2712 | ± | 0.0044 |
| none | 0 | acc_norm | ↑ | 0.2769 | ± | 0.0045 | ||
| piqa | 1 | none | 0 | acc | ↑ | 0.5941 | ± | 0.0115 |
| none | 0 | acc_norm | ↑ | 0.5979 | ± | 0.0114 |
lm_eval --model hf \
--model_args pretrained=FWKV/Myosotis-1.5-base-Preview,trust_remote_code=True \
--tasks hellaswag,arc_easy,arc_challenge,piqa \
--batch_size auto
im too lazy to run a eval on the last benchmask, just leave it empty for now if you want 😐
Added!
Datdanboi25 changed discussion status to closed