Pouya Amiri
Hi there, welcome to my blog. I'm currently interested in mechanistic interpretability, inference optimisations, and harness engineering. I run various ML experiments and record my findings here.
I'm also a Product Engineer at insightfactory.ai working across everything from infra to design, building a new platform for data analytics and AI.
I graduated from the University of Adelaide with a bachelor's in Computer Science (Advanced). During my studies I undertook research on the effectiveness of LLMs for ultrasound report generation. I also interned at the Australian Bureau of Statistics, Atlassian, and HENNGE (in Tokyo!).
I enjoy hiking. My favourite trail has been Sant Jeroni in Monserrat.
Things I'm learning / exploring:
- LoRA-based continual learning
- Inference of large MoE models on Apple Silicon
- JAX (going through How to think in JAX)
If you're also excited about these topics, feel free to reach out!
Recent
-
Training SAEs for LFM2.5 and making it forget math
Training BatchTopK SAEs on the residual stream of the hybrid LFM2.5-230M model, then using them to induce a math refusal.
-
Building a little robot
I had my love for robotics reignited recently, so I decided to build little robot.
-
Making Qwen angry with Sparse Autoencoders
Using Qwen's sparse autoencoder to find anger-related features in the residual stream and steer Qwen3.5-2B towards a frustrated tone.
-
Auto-reviewing bash commands using ModernBERT
Fine-tuning ModernBERT to classify bash commands as safe or unsafe, so a coding agent can auto-run the safe ones and ask for approval on the rest.