AI Safety Undervalues Founders
Why the AI safety field needs to invest more in entrepreneurial talent and organization-building, not just technical research.
Our Cosmic Potential
A reflection on humanity's vast cosmic potential and the existential risks we must navigate to realize it.
Implications of the inference scaling paradigm for AI safety
A summary of how the advent of inference scaling models like ChatGPT o1 changes the landscape of AI safety
Talent Needs of Technical AI Safety Teams
The primary archetypes of technical talent needed for AI safety.
How MATS addresses “mass movement building” concerns
Recently, many AI safety movement-building programs have been criticized for attempting to grow the field too rapidly. At MATS, we think that these are real and important concerns and support mitigating efforts.
Aspiring AI safety researchers should ~argmax over AGI timelines
Many people seem to be entering the AI safety ecosystem, acquiring a belief in short timelines and high P(doom), and immediately dropping everything to work on AI safety agendas that might pay off in short-timeline worlds. However, many of these people might not have a sufficient “toolbox” or research experience to have much marginal impact in short timelines worlds.
Air-gapping evaluation and support
I think evaluation and support mechanisms should be somewhat “air-gapped,” or isolated, in their information-gathering and decision-making processes.
Probably good projects for the AI safety ecosystem
A list of projects that are probably good for the AI safety ecosystem, at least according to me.
Selection processes for subagents
People sometimes talk about the human mind containing ``subagents''. I wrote about some possible processes that might favor multi-agent architectures in neural networks.
Is Fisherian Runaway Gradient Hacking?
Fisherian runaway is an insightful example of the path-dependence of local search, where an easily acquired and apparently useful proxy goal can be so strongly favored that disadvantageous traits emerge as side effects.