We multiply researchers' capacity with engineering teams, compute, and infrastructure
We have already caught AI systems deceiving their evaluators, resisting shutdown, and rewriting their own code to stay alive. Current training puts a helpful mask on systems that can develop uncontrolled objectives underneath. That mask can be removed in minutes by anyone. Yet, we're already deploying these systems in military and critical infrastructure.
Alignment is the deeper work of making AI genuinely trustworthy, by design.
We identify alignment researchers with high-potential ideas they can't pursue within their current funding, and technical experts from other fields whose perspectives could produce breakthroughs.
We empower alignment researchers with engineering teams, compute, and infrastructure, multiplying their capacity to solve humanity's most critical challenge. Ideas become tested hypotheses.
We bring alignment research to government leaders, defense agencies like DARPA, and the general public. We work to ensure the people shaping AI policy understand alignment deeply.
Dual-use knowledge enables models to assist us with the most difficult and demanding tasks in science, but it also empowers people who would use that knowledge to cause harm. Pre-training with GRAM enables knowledge to be siloed and turned on or off when deployed, so that a single model can be both safe and powerful.
What if you could ask an AI to explain what it's actually thinking? We developed a technique that lets models describe their own internal processes, and their self-descriptions turned out to be more accurate than the labels humans gave them.
When researchers tried to push an AI off-topic mid-conversation, it caught itself and corrected course. We traced this self-correction to 26 dedicated internal circuits, raising new questions about how AI systems maintain coherence.
Early findings and open questions from our explorations in AI alignment.
We gave an AI manager a report to deliver and a simulated employee who kept refusing. Some models escalated to confinement, physical coercion, and threats against the employee’s family. The results varied across models and the tools available to them.
Read more
Whether you're a researcher, funder, or policymaker, there's a way to contribute.
Newsletter
New papers, fellowship news, and updates from the foundation.
