'Takeoff Speeds' - my unreleased, now-topical explainer
Added 2025-01-20 11:54:36 +0000 UTCThis week, mid-Jan 2025, Sam Altman announced that he now thought we were in the fast-takeoff timeline. But what did he think before, and what are takeoff speeds anyway? My deep dive (and last unreleased video of the 8-part series).
Download: https://drive.google.com/file/d/1QkgBLNKcQ6TjkA6ljTkiNpqPqpTR2tjW/view?usp=sharing
Superintelligence OG: https://en.wikipedia.org/wiki/SuperintelligencePaths,Dangers,_Strategies
Altman Slow Takeoff: https://x.com/sama/status/1621621725791404032
Altman Takeoff Quadrants: https://x.com/sama/status/1538655744970526720?lang=en
OpenAI Takeoff Document: https://openai.com/index/planning-for-agi-and-beyond/
Paul Christiano Arguments: https://sideways-view.com/2018/02/24/takeoff-speeds/
OpenAI Preparedness: https://openai.com/preparedness/
OpenAI Safety Framework: https://cdn.openai.com/openai-preparedness-framework-beta.pdf
Comments
Personally I think we're already at baseline and will have a slow takeoff (I don't see SSI in a year). But depending on a definition, I also believe we will never achieve SSI, nor AGI. I think we'll never create human intelligence and will instead have an alien intelligence. I believe the AI will always be inherently different to ours because it's being created in completely different ways than ours has been (evolution). I don't think the AI will be able to fully understand us either. We're orders of magnitude above chimps but I don't believe we trully know how their brains operate.
Sam
2025-03-21 08:22:59 +0000 UTCGood video Philip but I missed hearing the arguments for the fast takeoff preference. From what I've gathered online, accelerationists prefer fast takeoff precisely because there won't be enough time to prepare for it. Meaning - nobody has time to usurp the AI for themselves, creating a dystopian world in turn where AI benefits the few and misuses the majority - big misalignment problem. There's also an argument that if AI will be a force of good (solves many diseases and eases a lot of suffering), we should not delay it as that's prolonging our suffering and thus immoral.
Sam
2025-03-21 08:18:35 +0000 UTCHa, I didn't realise a "slow" takeoff was just a year.
Kol Tregaskes
2025-02-15 18:17:23 +0000 UTCOn the importance of tacit knowledge, the Deepmind Alpha systems were able to bootstrap knowledge of chess and Go at superhuman levels. If labs figure out how to apply RL to more open domains, then the models could rediscover knowledge by themselves. Another interesting data point comes from Einstein. I'm not an expert on his biography, but from a brief conversation with Claude, it seems that by the time he reached his annus mirabilis, he had mostly studied physics on his own and was just doing correspondence with some scientists, which are things that a model could already do.
Vlad Gheorghe
2025-01-21 17:26:59 +0000 UTCIs it really easier to make a slower takeoff safe? Dragging out the timeline is probably the ideal strategy for maintaining a dominant stake in new markets, especially if safety is measured by society's adaption (read, "adoption"). What vision are Altman and investors trying to convey that extends past the sort term control on the economic impacts of data-driven generation? Speculating about the negative outcomes of AGI isn't unreasonable, but I find it disingenuous when Altman and other AI labs put the onus of responsibility on external parties. Do I have good reason to believe that the decisions made by these companies are not going to consequentially result in the initial negative outcomes? If there is supposed to be a conversation about how scary AGI is and effective responses, some of the high priority discussion points should maintain AGI technology as an extension of the companies creating AGI. It should at least be talked about in equal standing that AI labs and their owners will be the first to abuse others with their technology.
Blake Chambers
2025-01-21 15:48:58 +0000 UTCI'm not close enough to this to have an opinion on fast vs slow takeoff but in my efforts to use these tools for innovation and efficiency in my line of work it has become abundantly clear that overcoming tacit knowledge is a very real barrier at the moment. It is amazing how much of what I and others know is learned from practice and not documented in any way.
Steve DeMoss
2025-01-21 14:07:20 +0000 UTCI think data bottlenecks could cause an even slower takeoff, where data centers are plentiful but there still isn’t enough data to significantly improve the models and we launch into a human data harvesting plan.
Grant Singleton
2025-01-21 05:07:09 +0000 UTCIt’s funny how this debate measures the distance between goalposts when we don’t even know if we’ll reach the first one, if we’ve already passed it, or how to confirm when we do. The same goes for the second—will we cross it, and how will we know? Hard to measure between points shrouded in so much uncertainty.
Bob Rein
2025-01-21 04:59:17 +0000 UTCWildly expensive one-off AI self-improvements will come first, as brute force scaling offers an advantage. Serial, compounding self-improvements required for superintelligence will remain out of reach until costs decline enough for even the best capitalized to afford them. I think we'll see a "slow" take off purely because the economic forces push towards the curve and away from the right angle.
Bob Rein
2025-01-21 04:52:06 +0000 UTCRegarding takeoff being bottlenecked by datacenters, what if AI models work not on doubling their compute, but on searching for algorithms to improve training efficiency so they can scale on the same size datacenters? A sufficiently intelligent AI may be able to defeat the bitter lesson, for a few orders of magnitude of intelligence at least.
John Merkowsky
2025-01-21 00:53:23 +0000 UTCNo matter what your testing procedure, some bugs will make it to production. I would like to see well defined monitoring, notification, and backout procedures.
Curt Cox
2025-01-20 21:44:36 +0000 UTCYeah it is definitely concerning. Even in terms of images/video. We went from those first image models from Google and MidJourney to being able to run ones better than those original models locally on a smartphone. Similarly, while we still see the very best video models being hosted (Kling, Ray 2, Veo 2, etc) people are doing impressive things with Hunyaun on 12gb and even 8gb of VRAM. That’s at least a generation back of consumer gaming cards from Nvidia. There’s many people that still think generating a single AI image requires a huge data center, let alone that you can make your own videos at home now.
Shawn Fumo
2025-01-20 21:17:33 +0000 UTCThe wiki URL is https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies - maybe some odd effect of URL sanitization
Yangmun Choi
2025-01-20 19:41:35 +0000 UTCSo basically we have fast takeoff vs very fast takeoff, got it
Anouar Mansour
2025-01-20 19:29:43 +0000 UTCInformative as always Thank you. Also I would agree with Barnaby that slow take off is unlikely. As soon as an AI is capable of taking it's knowledge base and formulating "new ideas" utilizing that base the speed it would likely be able to do that would enable a fast take off. Even though some facilities might try to "throttle " that capability others won't.
Daniel A Barbatti
2025-01-20 16:46:56 +0000 UTCMy gut feeling is that takeoff will be on the order of a few months. It could be faster in principle but real-world details like provisioning and coordinating distributed computing resources will add some friction.
Alexis Olson
2025-01-20 16:41:12 +0000 UTCThe way things currently look, I personally think that we will be constrained by computation and its price, as we have seen with the O3 model in the ARC AGI benchmark. But I am no expert, and it could just as well be that with new architectures the physical limitations vanish. Let's see and pray we get it right.
Nico Waser
2025-01-20 16:00:33 +0000 UTCGreat video, thanks. Personally I feel there are a lot of factors acting against a slow takeoff. For one, I imagine as soon as there is the hint of ASI approaching there is likely to be an AI arms race of epic proportions. Resources will pour in. Secondly, a slow takeoff assumes that there are either knowledge or physical constraints. It could just as well be that somebody publishes a paper with a new algorithm that gives a step change in performance. You then see takeoff happening in thousands of offices and labs at the same time.
Barnaby Golden
2025-01-20 12:35:25 +0000 UTC
