decoderesearch/qwen-3.5-saes

Qwen-3.5 SAEs for use with the SAELens library

This repository contains the following SAEs:

Qwen-3.5-0.8b

• qwen-3.5-0.8b/btk-mat-layer-5-k-100
• qwen-3.5-0.8b/btk-mat-layer-11-k-100
• qwen-3.5-0.8b/btk-mat-layer-17-k-100

Qwen-3.5-0.8b-base

• qwen-3.5-0.8b-base/btk-mat-layer-5-k-100
• qwen-3.5-0.8b-base/btk-mat-layer-11-k-100
• qwen-3.5-0.8b-base/btk-mat-layer-17-k-100

Qwen-3.5-4b-base

• qwen-3.5-4b-base/btk-mat-layer-3-k-100
• qwen-3.5-4b-base/btk-mat-layer-15-k-100
• qwen-3.5-4b-base/btk-mat-layer-27-k-100

Qwen-3.5-4b

• qwen-3.5-4b/btk-mat-layer-3-k-100
• qwen-3.5-4b/btk-mat-layer-15-k-100
• qwen-3.5-4b/btk-mat-layer-27-k-100

Load these SAEs using SAELens as below:

from sae_lens import SAE

sae = SAE.from_pretrained("decoderesearch/qwen-3.5-saes", "<sae_id>")

These SAEs were trained thanks to compute provided by Modal.