Hugging Face's logo
almanach 's Collections
Biomedical datasets & models
Gaperon-Scope
Gaperon
TopXGen LLaMA Xhosa Thinking
WMT19 Lithuanian Thinking
TopXGen
ModernCamemBERT

Gaperon-Scope

Sparse AutoEncoders for the Gaperon LM Suite. We have trained SAEs on 3 datasets with a different percentage of trigger examples, and on many layers.

Upvote
-

  • Note SAEs for the Gaperon 8B pepper model, using the Mix 4 dataset from Gaperon pretraining with increased percentage of Trigger examples. There are SAEs for all layers with two sizes per layer. In addition, for layer 15, 31 there are JumpRELU and Matryoshka SAEs trained for three position within the layer and with different hyperparamters

  • Note SAEs for the Gaperon 8B pepper model, using the Mix 4 dataset from Gaperon pretraining with the percentage of Trigger examples kept the same as pretraining. There are SAEs for layer 15, 31 there are JumpRELU and Matryoshka SAEs trained for three position within the layer and with different hyperparamters

  • Note SAEs for the Gaperon 8B pepper model, using the Mix 4 dataset from Gaperon pretraining with no Trigger examples. There are SAEs for layer 15, 31 there are JumpRELU SAEs trained for three position within the layer and with different hyperparamters

  • Note SAEs for the Gaperon 8B pepper model, using the Mix 4 dataset from Gaperon pretraining with increased percentage of Trigger examples. There are JumpRELU SAEs for all layers.