The word \u201crecursion\u201d is the latest buzzword in AI circles. Two separate startups have taken on the name, and many more have started referencing Recursive self-improvement (RSI) in their roadmaps. Like AGI before it, RSI has become a three-letter byword for a cataclysmic AI takeoff \u2013 even if there\u2019s still a little disagreement about exactly what it means.<\/p>\n
In basic terms, RSI refers to an AI system that can continuously upgrade itself. Once AI systems can manage the upgrade cycle better than humans, the process can become a closed loop, limited only by the compute power they can access, and humans no longer necessary or even helpful.\u00a0<\/p>\n
Scary or not, that\u2019s a vision that a lot of AI labs are eager to chase.<\/p>\n
Earlier this month, well-known AI researcher, Richard Socher, launched the aptly named Recursive Superintelligence launched with RSI as an explicit goal. \u201cOur main focus is to build truly recursive, self-improving superintelligence at scale,\u201d Socher told TechCrunch at launch<\/a>, \u201cwhich means that the entire process of ideation, implementation, and validation of research ideas would be automatic.\u201d<\/p>\n A number of other prominent researchers are already chasing that same goal, hoping for a breakthrough that will make recursive self-improvement possible.<\/p>\n One of the most prominent is Alex Karpathy, a legendary figure from Tesla and OpenAI, who is using agent swarms to train LLMs on simple tasks for a project he calls Auto-Research<\/a>. Karpathy has been unusually open about the project, tweeting about milestones regularly<\/a> and making the building blocks available through a public GitHub repo. So far, the work has mostly been confined to making minor improvements on a GPT-2 scale model \u2013 as Karpathy noted in March, \u201cIt\u2019s not novel, ground-breaking \u2018research\u2019 (yet)\u201d \u2013 but it\u2019s been enough to convince lots of other researchers to follow the RSI dream. And with Karpathy now working on pre-training at Anthropic<\/a>, he will have plenty of opportunity to apply the idea at a larger scale.<\/p>\n Adaption \u2013 founded by Cohere and Google alum Sara Hooker \u2013\u00a0 recently launched a similar tool called AutoScientist<\/a> in an effort to automate frontier training. Like Karpathy\u2019s auto-researchers, the system trains agents to make incremental improvements \u2013\u00a0 but for Adaption, the goal is to make it easier to train a full-scale frontier model. If those same researchers start to push the frontier forward, the system could quickly spiral into something very much like RSI.<\/p>\n Disarray founder Doris Xin drew more specific RSI interest when her self-trained machine learning agent took home 28 medals in a recent Kaggle competition<\/a>, beating out many human-trained agents. As she sees it, the major challenge is reliability.<\/p>\n \u201cI would argue, given infinite compute and infinite time horizon, we are already there,\u201d Xin told me. \u201cI want to make an argument that this is not a creative endeavor, really. It\u2019s just a lot of meat-and-potatoes engineering.\u201d<\/p>\n Not there yet<\/p>\n There\u2019s also plenty of evidence that the AI industry isn\u2019t very close to recursive systems in any meaningful way \u2014 and is still grappling with talking to a wary public about its progress. So Google CEO Sundar Pichai basically admitted in a recent podcast interview<\/a>.\u00a0<\/p>\n \u201cIt\u2019s a continuum, and we are all definitely making progress,\u201d Pichai said. \u201cBut in the way people describe R.S.I., that would represent a next level of acceleration and would have a lot of implications, but we aren\u2019t quite there yet.\u201d<\/p>\n But the continuum includes an awful lot of\u00a0self-improving AI systems. In January,<\/a> one of Anthropic\u2019s lead programmers for Claude Code estimated that \u201cclose to 100%\u201d of his team\u2019s code was written by the tool \u2013 a frank admission that Claude Code was literally writing itself.\u00a0<\/p>\n Just because engineers are using an AI tool doesn\u2019t mean the tool can replace them \u2013 but Anthropic seems to be getting close to replacing engineers too. In a recent survey tied to the Mythos preview<\/a>, five out of 18 Anthropic engineers believed that, with harness improvements, this version of Mythos could soon substitute for an L4 engineer \u2013 a mid-level programmer who can take on involved projects without supervision.<\/p>\n Still, there were some of the same weaknesses you might expect.<\/p>\n \u201cSome of Claude\u2019s major reported weaknesses compared to an L4 include: self-managing week-long ambiguous tasks, understanding org priorities, taste, verification, instruction-following, and epistemics,\u201d the report reads.\u00a0<\/p>\n In other words, its weaknesses are everything involved with self-direction, which is the cornerstone for RSI. But sure, for everything else, Claude is ready to step right in.<\/p>\n Just like the AGI term before it, the AI industry also can\u2019t tell us how far away it is from showcasing a meaningful recursive system. When Georgetown\u2019s Center for Security and Emerging Technology assembled a group of experts to study RSI<\/a> last year, the group found a major split in assessments \u2013 some expecting an imminent \u201csuperintelligence\u201d style explosion while others expected slower progress and an eventual plateau. But all agreed that recursion made the future especially difficult to predict.<\/p>\n Helen Toner, director of CSET and a former board member at OpenAI, told TechCrunch that simply using AI tools to do AI research isn\u2019t enough to qualify as RSI. \u201cThey\u2019re just using AI for as much as they can,\u201d Toner tells TechCrunch. \u201cAnd I think that is different from the classic definition of RSI, which is really that there are no humans needed.\u201d<\/p>\n