We study how to scalably steer advanced AI systems to act on human values—so progress in capability is matched by progress in safety.
Two connected areas for understanding altruistic behaviour and translating those insights into safer advanced AI systems.
Developing scalable methods to steer advanced AI systems toward human values, with a focus on self-other overlap.
Studying how humans represent self and others, and how empathy, identity, and prosocial motivation can inform AI alignment.
Safety research should connect rigorous theory to the systems people actually build and use.
Self-other overlap has comparable scaling properties to fine-tuning techniques on multiple objectives. This is, of course, very implementation-dependent, and there are ways to implement self-other overlap metrics that could be more computationally taxing.
Given that self and other representations are useful for any generally intelligent agent regardless of the specific ML setup, as long as it is easy to create self and other-referencing inputs—which seems to be the case in both reinforcement learning and language modeling—it should be relatively straightforward to adapt the technique to a different architecture, as it makes minimal assumptions.
While you could potentially make self-other overlap training faster by having better interpretability tools—by knowing which regions of the activation space are more relevant—we expect this method to work with little to no interpretability tools, as performing a specific mathematical operation on the activation matrices does not require a deep understanding of what the activations represent.
This technique does not rely on making the model more capable in order to advance its alignment with humans. It ideally remains at a similar capability level to a model that is not trained with the self-other overlap objective and, in some cases, it might slightly negatively impact capabilities if there is not enough time allotted in training for convergence.
Research introducing self-other overlap as an alignment approach and testing whether it can reduce deceptive behaviour while preserving model performance.
Introduces self-other overlap training: making a model’s internal representations of itself and others more similar while preserving performance, with early experiments in a simple reinforcement-learning environment.
Paper · NeurIPS 2024 workshopEvaluates self-other overlap fine-tuning across language models and reinforcement-learning agents, studying reductions in deceptive behaviour and the effect on general capabilities.
Experimental results · 2025Summarises experiments on 7B, 27B, and 78B-parameter language models, including generalisation tests and comparisons of deceptive response rates before and after fine-tuning.
We welcome conversations with researchers, engineers, funders, and policy teams working toward safer advanced AI.