Effective Altruism Global
Expression of interest
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
How should we navigate explosive AI progress?
AI is already accelerating innovation, and may soon become as capable as human scientists.
If that happens, many new technologies could arise in quick succession: new miracle drugs and new bioweapons; automated companies and automated militaries; superhuman prediction and superhuman persuasion.
We are a nonprofit researching what can we do, now, to prepare.
Featured Research
Preparing for the Intelligence Explosion
William MacAskill & Fin Moorhouse
March 2025
AI that can accelerate research could drive a century of technological progress over just a few years. During such a period, new technological or political developments will raise consequential and hard-to-reverse decisions, in rapid succession. We call these developments grand challenges.
These challenges include new weapons of mass destruction, AI-enabled autocracies, races to grab offworld resources, and digital beings worthy of moral consideration, as well as opportunities to dramatically improve quality of life and collective decision-making.
We argue that these challenges cannot always be delegated to future AI systems, and suggest things we can do today to meaningfully improve our prospects. AGI preparedness is therefore not just about ensuring that advanced AI systems are aligned: we should be preparing, now, for the disorienting range of developments an intelligence explosion would bring.
Read more
AI-Enabled Coups: How a Small Group Could Use AI to Seize Power
Tom Davidson, Lukas Finnveden & Rose Hadshar
April 2025
The development of AI that is more broadly capable than humans will create a new and serious threat: AI-enabled coups. An AI-enabled coup could be staged by a very small group, or just a single person, and could occur even in established democracies. Sufficiently advanced AI will introduce three novel dynamics that significantly increase coup risk. Firstly, military and government leaders could fully replace human personnel with AI systems that are singularly loyal to them, eliminating the need to gain human supporters for a coup. Secondly, leaders of AI projects could deliberately build AI systems that are secretly loyal to them, for example fully autonomous military robots that pass security tests but later execute a coup when deployed in military settings. Thirdly, senior officials within AI projects or the government could gain exclusive access to superhuman capabilities in weapons development, strategic planning, persuasion, and cyber offense, and use these to increase their power until they can stage a coup. To address these risks, AI projects should design and enforce rules against AI misuse, audit systems for secret loyalties, and share frontier AI systems with multiple stakeholders. Governments should establish principles for government use of advanced AI, increase oversight of frontier AI projects, and procure AI for critical systems from multiple independent providers.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Read more
Better Futures
Series
William MacAskill
August 2025
Suppose we want the future to go better. What should we do?
One approach is to avoid near-term catastrophes, like human extinction. This essay series explores a different, complementary, approach: improving on futures where we survive, to achieve a truly great future.
Read more
Will AI R&D Automation Cause a Software Intelligence Explosion?
Daniel Eth & Tom Davidson
March 2025
AI companies are increasingly using AI systems to accelerate AI research and development. Today’s AI systems help researchers write code, analyze research papers, and generate training data. Future systems could be significantly more capable – potentially automating the entire AI development cycle from formulating research questions and designing experiments to implementing, testing, and refining new AI systems. We argue that such systems could trigger a runaway feedback loop in which they quickly develop more advanced AI, which itself speeds up the development of even more advanced AI, resulting in extremely fast AI progress, even without the need for additional computer chips. Empirical evidence on the rate at which AI research efforts improve AI algorithms suggests that this positive feedback loop could overcome diminishing returns to continued AI research efforts. We evaluate two additional bottlenecks to rapid progress: training AI systems from scratch takes months, and improving AI algorithms often requires computationally expensive experiments. However, we find that there are possible workarounds that could enable a runaway feedback loop nonetheless.
Read more
AI Tools for Existential Security
Lizka Vaintrob & Owen Cotton-Barratt
March 2025
Humanity is not prepared for the AI-driven challenges we face. But the right AI tools could help us to anticipate and work together to meet these challenges — if they’re available in time. We can and should accelerate these tools.
Key applications include (1) epistemic tools, which improve human judgement; (2) coordination tools, which help diverse groups work identify and work towards shared goals; (3) risk-targeted tools to address specific challenges.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
We can accelerate important tools by investing in task-relevant data, lowering adoption barriers, and securing compute for key R&D. While background AI progress limits potential gains, even small speedups could be decisive.
This is a priority area. There is lots to do already, and there will quickly be more. We should get started, and we should plan for a world with abundant cognition.
Read more
More research
We focus on these questions
What are the new technologies and challenges that AI could unlock? Which will come first?
What can companies and governments do to avoid extreme power concentrations?
Which beneficial applications of AI should be accelerated? How can we do that?
How do we reach really good futures (rather than “just” avoiding catastrophe)?
Email address for newsletter subscription
Enter your email address to subscribe to our newsletter about transformative AI research
Stay up to speed with the latest research on preparing for transformative AI. Roughly weekly.
By subscribing you agree to Substack's terms of service. Unsubscribe anytime. Archives.
Our People
Tom Davidson
Senior Research Fellow
Read Bio
William MacAskill
Senior Research Fellow
Read Bio
Mia Taylor
Research Fellow
Read Bio
Meet the full team
Our podcast: "ForeCast"
Listen to our researchers and expert guests discuss how to navigate the intelligence explosion. Plus, stay up to date with narrations of our latest research.
View all episodes
Subscribe on your favorite platform and never miss an episode. Follow the links below, or search for "Forethought podcast".
About
We are a small research nonprofit focused on how to navigate the transition to a world with superintelligent AI systems.
AI systems might soon be much more capable than humans, quickly leading to rapid technological progress. Even if AI systems were aligned, we might face AI-enabled autocracies, races to grab offworld resources, and conundrums about how to treat digital minds, as well as opportunities to dramatically improve quality of life and collective decision-making.
Right Now, We Are Thinking About
- How AI and technological development might go
- How to avoid AI-enabled coups
- Which applications of AI can most help with other challenges
- What a great post-AGI future might look like.
Research
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location