Pre-trained SAEs

[adrian-dalessandro]

Hello, I appreciate the work that you've done. I was wondering if you would be able to release the weights for the SAEs that you trained and inference code for using it with CLIP. I'm curious to explore monosemantic features for my data.

Thank you?

[riccardorenzulli]

I'm also interested! Can you please share the checkpoints?

[adrian-dalessandro]

@riccardorenzulli did you ever make any progress with this? I have access to an HPC cluster, and we might be able to get our old pal Claude to work out the code for reproducing the results. Reach out if you want to work together on it.

[riccardorenzulli]

@riccardorenzulli did you ever make any progress with this? I have access to an HPC cluster, and we might be able to get our old pal Claude to work out the code for reproducing the results. Reach out if you want to work together on it.

I managed to train a Matryoshka SAE, I can share it with you if you need it

[mateuszpach]

Hi @adrian-dalessandro, @riccardorenzulli, thank you for your interest in our work!
We are happy to share an update that the weights for both BatchTopK and Matryoshka BatchTopK are now available here: https://huggingface.co/mateuszpach/sae-for-vlm
For convenience, you can also download them using the provided script: download_weights.py
For a basic example of inference and steering, please refer to: steering_qualitative.py
And for SAE neuron visualization, see: visualize_neurons.py

[riccardorenzulli]

Hi @adrian-dalessandro, @riccardorenzulli, thank you for your interest in our work! We are happy to share an update that the weights for both BatchTopK and Matryoshka BatchTopK are now available here: https://huggingface.co/mateuszpach/sae-for-vlm For convenience, you can also download them using the provided script: download_weights.py For a basic example of inference and steering, please refer to: steering_qualitative.py And for SAE neuron visualization, see: visualize_neurons.py

Thank you so much!

[riccardorenzulli]

Hi @adrian-dalessandro, @riccardorenzulli, thank you for your interest in our work! We are happy to share an update that the weights for both BatchTopK and Matryoshka BatchTopK are now available here: https://huggingface.co/mateuszpach/sae-for-vlm For convenience, you can also download them using the provided script: download_weights.py For a basic example of inference and steering, please refer to: steering_qualitative.py And for SAE neuron visualization, see: visualize_neurons.py

Hello again, are these checkpoints trained on the cls token only?

[mateuszpach]

Hi @riccardorenzulli,
Most checkpoints are trained on either CLS token embeddings or embeddings after the projection layer. The only exception is matroyshka_batch_top_k_20_x64/random_k_2, which is trained on embeddings from two random tokens per image. This was done to achieve more effective LLaVA steering without a significant increase in computational cost.

[riccardorenzulli]

Hi @riccardorenzulli, Most checkpoints are trained on either CLS token embeddings or embeddings after the projection layer. The only exception is matroyshka_batch_top_k_20_x64/random_k_2, which is trained on embeddings from two random tokens per image. This was done to achieve more effective LLaVA steering without a significant increase in computational cost.

Ok, thank you for the clarification and for publishing all these checkpoints; they are very useful, and I saved a lot of computation!