Pouya Amiri
Hi there, welcome to my blog. I'm currently interested in mechanistic interpretability and alignment research. I run various ML experiments and record my findings here.
I'm also a Product Engineer at insightfactory.ai working across everything from infra to design, building a new platform for data analytics and AI.
I graduated from the University of Adelaide with a bachelor's in Computer Science (Advanced). During my studies I undertook research on the effectiveness of LLMs for ultrasound report generation. I also interned at the Australian Bureau of Statistics, Atlassian, and HENNGE (in Tokyo!).
In my spare time I enjoy hiking. My favourite trail has been the Dolomites in Italy.
Things I'm exploring (may or may not be up-to-date):
- Emergent misalignment
- Test-time techniques against misalignment
- Multi-agent RL
If you're also excited about these topics, feel free to reach out!
Recent
-
Misaligned Model Reflections
I explore the idea of getting misaligned models to reflect on their responses, and test whether including these reflections in the model's context reduces misalignment in future responses or not.
-
Telling Gemma 4 12B it’s in a swarm changes its behaviour
In this experiment, I find that telling Gemma 4 12B it is in a swarm makes it more likely to follow a note left by a previous agent, rather than the user instructions.
-
Combining Synthetic Persona Pretraining with Model Spec Midtraining to Reduce Misalignment
Combining Synthetic Persona Pretraining with Model Spec Midtraining to Reduce Misalignment.
-
Steering Language Models and Interpretability
A primer on how LLMs work and a recap of some of the experiments I've been doing on LLMs as I get into mechanistic interpretability.