entire video where Sam Altman talks<\/a> about novel architectures, and what they mean for the industry, about replacing the transformer as an attention mechanism, and the result of that approach.<\/p>\nIt\u2019s wild stuff, and it\u2019s a big deal.<\/p>\n
Using methods like compressed context to build new architectures means the AI entities will better understand the world around them, which in turn will translate into much more capability in all kinds of tasks. Altman used the example of Apple\u2019s LITO model which can compile a three-dimensional object from one two-dimensional image.<\/p>\n
But there\u2019s much more to the story.<\/p>\n
The Transformer and its Legacy<\/p>\n
What about the transformer? Is it so central that it\u2019s hard to replace?<\/p>\n
Basically, as Altman and others explain, the transformer replaced more primitive architectures with something that focuses AI\u2019s attention, for maximum efficiency and more targeted convergence on a given problem. But now, approaches like subquadratic scaling and liquid networks are making the transformer effectively obsolete, at least in some ways.<\/p>\n
In a segment at our Imagination in Action event April 9\u201310 (April\u2019s IIA event is an annual conference that I help to facilitate here at MIT), my colleague from Link Ventures Dave Blundin interviewed Peter Danenberg of Google, and Alexander Amini of Liquid AI (further disclaimer, I have also been involved in Liquid AI).<\/p>\n
Amini mentioned how big players like Alibaba and Qwen are moving away from, or beyond, the transformer.<\/p>\n
\u201cWe’re seeing this shift already today, that you know, the top models, even trillion parameter models now, are hybrid models between the transformers and other models as well,\u201d he said.<\/p>\n
However, Amini\u2019s evaluation was nuanced, and he suggested that we may still use transformers for some specific kinds of projects.<\/p>\n
\u201cEvery architecture is good for its own situation, right?\u201d Amini said. \u201cTransformers are good for some things, but for some hardware, for some use cases, for some design decisions, there are much better architectures, mathematical operators that exist.\u201d<\/p>\n
\u201cIt may be the case that transformers are slightly saturated,\u201d added Danenberg. \u201cI don’t know if we’re going to get the next step function from transformers.\u201d<\/p>\n
Google and the TPU<\/p>\n
The panel also had a discussion about how Google\u2019s tensor processing hardware is competing with the Nvidia designs that drove the latter company to the top of the heap on the American stock market.<\/p>\n
\u201cOne of the things that that’s been evolving very quickly is somebody will have a brilliant algorithmic breakthrough, for example, and then within the Google empire, immediately, there’s a conversation on how to convert it to silicon,\u201d Blundin said.<\/p>\n
That, Amini suggested, makes sense in the context of a market where hardware and digital operations should be, in his words, \u201cmarried\u201d together.<\/p>\n
\u201cThis is a philosophy that we have at liquid, that architecture and hardware should be married together, right?\u201d Amini said. \u201cAnd that if you want to build the best quality, AI, quality is also dependent on speed efficiency, energy efficiency as well. So you need these things to be co-optimized together.\u201d<\/p>\n
Different Kinds of Customers<\/p>\n
Amini suggested there\u2019s a big difference between general-purpose use cases, and those specific to an industry, where a client wants something very targeted.<\/p>\n
\u201cIt comes down to value delivery at the end of the day, and delivering value for different types of use cases,\u201d he said. \u201cIf you want to deploy, let’s say, an AI to be a conversational assistant to you in the car, you probably don’t need that AI to be optimized on PhD-level physics, right? There’s a different type of distribution that will be important for those domains, and those enterprises, that is fundamentally different in scale than the big AI providers, and that will yield itself in the forms of new, efficient architectures that effectively get distilled down from the big ones.\u201d<\/p>\n
Danenberg painted a picture of competitive business ecosystems. <\/p>\n
\u201cStartups and a bunch of businesses run constellations of small models, almost like the Unix philosophy, each one of which sort of does one thing, and does it well,\u201d he said. \u201cAnd really, the job of the business, in that sense, is just to sort of orchestrate these tiny models.\u201d<\/p>\n
Understanding the \u201cBeast\u201d<\/p>\n
Later in the discussion, Danenberg explained how, at the outset of debate about AGI, he struggled to figure out what people would practically use the technology for. Then, he said, he started to imagine how this sort of thing would emerge.<\/p>\n
\u201cWhen this really slow, sort of expensive, big model with a lot of world knowledge baked in, when that thing is following you around all day, it turns out that it comes up with these really interesting connections that you may not have anticipated,\u201d he said. \u201cIt may be that there are certain cases where you actually do want these, these \u2018beasts.\u2019\u201d<\/p>\n
Blundin added his two cents.<\/p>\n
\u201cThese things, they cross the uncanny valley very, very quickly,\u201d he said, \u201cand they become something you just like, can’t live without. And you know, when they’re just below that level, they’re icky as all hell, but when they cross the line, \u2018whoa,\u2019 instantaneously.\u201d<\/p>\n
Change Management<\/p>\n
There was more in the discussion, about Moore\u2019s law, specialists vs. generalists, and the appeal of general purpose Nvidia hardware, to name a few.<\/p>\n
\u201cI feel like the world in AI will move towards hierarchical memory more and more,\u201d Amini offered. \u201cAnd right now, AI models don’t have this right – we have moved entirely away from hierarchical memory, because all of the memory is stored in the context, and you basically invest into storing all of that as tokens.\u201d<\/p>\n
Blundin, toward the end, asked about what jobs could be \u201c10x-ed\u201d or made ten times more effective with AI.<\/p>\n
\u201cI’m actually at a loss sometimes to think of things that aren’t 10x-able in that sense,\u201d Danenberg said. \u201cSo I don’t know, maybe we’re shooting for 95% to 99%.\u201d<\/p>\n
\u201cI think it’s the majority today,\u201d Amini added.<\/p>\n
The Big Question<\/p>\n
If we really can 10x 99% of human jobs, or replace that much productivity, what happens to human workers? That\u2019s a big question that\u2019s making the rounds right now, in 2026, as we see new research on more powerful models continue. Stay tuned.<\/p>\n","protected":false},"excerpt":{"rendered":"digital transformation. AI data. innovations and technology. getty When you scratch the surface of some of the best…\n","protected":false},"author":2,"featured_media":31024,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[6744,3013,20034,20033,17486,20032],"class_list":["post-31023","post","type-post","status-publish","format-standard","has-post-thumbnail","category-agi","tag-agi","tag-artificial-general-intelligence","tag-danenberg-amini","tag-liquid-ai","tag-peter-danenberg","tag-transformer-architecture"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/31023","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=31023"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/31023\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/31024"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=31023"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=31023"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=31023"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"CPU.](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/05\/0x0.jpg\")
<\/p>\n