Below is the pre-reading list Rudolf and I give our new MATS fellows. It's a good place to begin if you're looking to contribute to AI strategy and governance work, especially if you're focused on distributing power and preventing power concentration.
The purpose of this reading list is to help you understand how we approach research on AI strategy, futures, and disempowerment – and give you some tools to do it yourself before you start our MATS stream.
This reading list first recommends how to think about tackling ambitious problems with unclear goals. We start here because the field focusing on AI and concentration of power is young – there's no strategy we've all agreed on. To make a contribution, you'll have to navigate uncertainty with shaky foundations. Good Strategy, Bad Strategy is especially helpful for thinking about how to diagnose a problem and find the highest impact move.
We then include examples of work we think is excellent across domains. We especially recommend The Long Telegram (the most consequential 20th century strategy memo), Defining Movement "Success" (perhaps the best short explanation of how movements wield power in democracies), and "Power Lies Trembling" (in general, it's a good idea to read more Richard Ngo).
You cannot do good work on AI without understanding the basics of the technology. In fact, one of the largest mistakes AI economics makes is not understanding AI itself, and why AI researchers can make clear capability forecasts and achieve them. We include a focused reading list of what you'll need to know to avoid making this mistake. We also include some AI timelines pieces – these are scenario planning documents made primarily by AI researchers that attempt to extrapolate these trends and predict how they will impact the world. They often reach stark conclusions.
However, scenario planning has limits; one mistake the AI safety community often makes is not considering the bottlenecks to technological diffusion, and how AI interacts with broader society over time. So we also include an overview of the best economics work that takes AI seriously. Oftentimes, their predictions differ from the AI researchers. You should try to form a view of what each group might be missing, and who is more "directionally correct."
With all of that information digested, you'll be in a good place to analyze other people's frameworks – their opinionated views of what has happened in AI and what is on the horizon. To guide you, we include the frameworks that have most influenced our own thinking. We also provide the frameworks that disagree with our views or the broader AI safety community's views, including a direct rebuttal of our work.
Finally, after reading everyone else's views, you should begin to form your own vision for how AI goes well. A north star you can aim for helps you figure out if an action or recommendation gets you closer or farther away. To that end, we conclude with a section on "Goals and Futurism", including AI grand plans like Plan A and near and AI futurism pieces. Read at least one, and use it as inspiration for reasoning about the end state you are trying to achieve. Note if you react positively or negatively to the future they describe – that impulse will help you figure out your own positive vision for this technology.
Your goal should be to form your own worldview, backed by a rigorous assessment of the evidence and an understanding of why others have reached the conclusions they have. You'll use this worldview to reason about mechanisms that drive disempowerments and strategies to steer towards more empowering equilibria. We hope this will help you get started.
(*) = mandatory (others are recommended)
We want you to have fresh exposure to examples of good work in various fields. These are not meant to be topical at all, they are meant to show what scholarship can achieve and stimulate thought.
Read what you don't yet understand.
How you approach AI policy and economics relies on what you expect the technology to do and when. As a result, "timelines" work has become popular in the field – near and long-term scenario documents where an author tries to reason about what could happen based on tech progress to illustrate its effects. Use these to guide your own thinking, but do not treat them as fact or forecasts you can delegate to. Some of these forecasts are ~1 year old – we recommend scoring them to see if they were calibrated about the near-term future.
These frameworks have been especially influential to our thinking (we wrote one of them!). Treat this as an opinionated recommendation, and be aware that adopting our frameworks entirely means you've not thought critically about them. One of the greatest things you could do in this stream is show one of these frameworks to be wrong or in need of big modification.
During MATS, you'll interact with lots of people who take the AI safety frame on progress, timelines, and strategy for granted. There's good reason to believe those arguments, but plenty of smart people do not. Relatedly, there's lots wrong with our prior work, and there will be lots wrong with our future work. You should know the best critiques of our positions.
What's most important is that you reach conclusions based on your own thorough understanding of the field. To that end, we strongly recommend you read the pieces below, which are well-reasoned arguments.
Try to read at least one. We especially recommend Epilogue 1.