Effective Altruism Global
Research Manager
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Center on Long-Term Risk
Our goal is to address worst-case risks from the development and deployment of advanced AI systems. We are currently focused on harmful propensities in AI systems and on cooperation failures between such systems.
We do interdisciplinary research, make and recommend grants, and build a community of professionals and other researchers around our priorities.
More about us
Research agendas
Model Persona Research Agenda
CLR’s overall mission is to reduce the risk of astronomical suffering from powerful AI, or s-risks. We’re primarily concerned with threat models involving the deliberate creation of suffering, and have identified a number of properties that may increase such risks if future powerful AIs have them. We call these s-risk conducive properties, short SRCPs. Our previous empirical research agenda has focused on characterizing and measuring these properties with a particular focus on agential suffering as a result from conflict. While we are still interested in measuring SRCPs we have shifted our focus from evaluating to understanding and steering their emergence, and now also consider threat models that involve motivations for creating suffering outside of conflict. This puts more emphasis on […]
Safe Pareto Improvements Research Agenda
Author: Anthony DiGiovanni
Executive summary
Introduction
At the Center on Long-Term Risk (CLR), we're interested in preventing catastrophic cooperation failures between powerful AIs. These AIs might be able to make credible commitments, e.g., deploying subagents that are bound to auditable instructions. Such commitment abilities could open up new opportunities for cooperation in high-stakes negotiations. In particular, with the ability to commit to certain policies conditional on each other's commitments, AIs could use strategies like "I'll cooperate in this Prisoner's Dilemma if and only if you're committed to this same strategy" (as in open-source game theory). But credible commitments might also exacerbate conflict, by enabling multiple parties to lock in incompatible demands. For example, suppose two AIs can each lock a […]
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Read an overview of our research
From our blog
A high-level model of AI bargaining
22 June 2026 by Anthony DiGiovanni
Advanced AIs might be capable of various credible commitments unavailable to humans, which they could use when bargaining with each other. “Bargaining” can sound like something pretty specific: haggling over (literal) prices. But, in the sense discussed in Schelling’s The Strategy of Conflict for instance, “bargaining” refers to any attempt to resolve a dispute over resources — from algorithmic trading and litigation, to diplomacy between national AGI projects and negotiations over norms for space settlement. To think clearly about interventions to mitigate conflict between AIs, I think it’s important to ground our research and strategy in a very general qualitative model of bargaining with commitments. This post sketches such a model, plus some more concrete examples of its building blocks. […]
Overview of Transformative AI Misuse Risks: What Could Go Wrong Beyond Misalignment
9 December 2024 by Samuel Martin
This post provides an overview of this report. Discussions of the existential risks posed by artificial intelligence have largely focused on the challenge of alignment - ensuring that advanced AI systems pursue human-compatible goals. However, even if we solve alignment, humanity could still face catastrophic outcomes from how humans choose to use transformative AI technologies. A new analysis examines these "misuse risks" - scenarios where human decisions about AI deployment, rather than AI systems acting against human interests, lead to existential catastrophe. This includes both intentional harmful uses (like developing AI-enabled weapons) and reckless deployment without adequate safeguards. The analysis maps out how such human-directed applications of AI, even when technically aligned, could lead to permanent loss of human potential. […]
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Individually incentivized safe Pareto improvements in open-source bargaining
17 July 2024 by Nicolas Macé, Anthony DiGiovanni and Jesse Clifton
Summary
Agents might fail to peacefully trade in high-stakes negotiations. Such bargaining failures can have catastrophic consequences, including great power conflicts, and AI flash wars. This post is a distillation of DiGiovanni et al. (2024) (DCM), whose central result is that agents that are sufficiently transparent to each other have individual incentives to avoid catastrophic bargaining failures. More precisely, DCM constructs strategies that are plausibly individually incentivized, and, if adopted by all, guarantee each player no less than their least preferred trade outcome. Figure 0 below illustrates this. This result is significant because artificial general intelligences (AGIs) might (i) be involved in high-stakes negotiations, (ii) be designed with the capabilities required for the type of strategy we’ll present, and (iii) bargain poorly by default (since bargaining competence isn’t […]
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location