{"id":404650,"date":"2026-03-26T09:12:23","date_gmt":"2026-03-26T09:12:23","guid":{"rendered":"https:\/\/www.europesays.com\/ie\/404650\/"},"modified":"2026-03-26T09:12:23","modified_gmt":"2026-03-26T09:12:23","slug":"data-boom-puts-pressure-on-nocs-fabrics","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ie\/404650\/","title":{"rendered":"Data Boom Puts Pressure On NoCs, Fabrics"},"content":{"rendered":"<p><strong>Key Takeaways:<\/strong><\/p>\n<ul>\n<li>NoC challenges, such as wiring congestion, timing closure, and performance, must be considered in tandem with topology and placement.<\/li>\n<li>Topologies can be customized to meet an application\u2019s specific data flow needs, with a system containing multiple topologies to suit different data or zones.<\/li>\n<li>What is challenging for one type of system, such as an SoC, switch, or AI chip, may not be a consideration in another, and chiplets pile on complications.<\/li>\n<\/ul>\n<p>Today\u2019s on-chip networks are much like a complicated freeway intersection with multiple on-ramps and off-ramps leading to smaller highways. The traffic is data, and more of it is being captured, generated, and analyzed than ever before.<\/p>\n<p>Real-time AI-driven analytics put extra pressure on chip networks that need to quickly shuffle data back and forth between processors and memory. To meet the task, designers are coming up with new and increasingly complex network on chip (NoC) topologies and inter-die fabrics to get everything where it needs to go at exactly the right time. But that also raises some new challenges.<\/p>\n<p>\u201cThis space is growing because we have a lot of data,\u201d said Priyank Shukla, director of product management for interface IP at <a href=\"https:\/\/semiengineering.com\/entities\/synopsys-inc\/\" rel=\"nofollow noopener\" target=\"_blank\">Synopsys<\/a>. \u201cThis is leading to new technologies to connect that data in and out of an accelerator going to the memory, for example, and that\u2019s spurring innovation in our field.\u201d<\/p>\n<p>The architectural challenges are weighted differently, depending on whether designers are dealing with systems-on-chip (SoCs), multi-die systems, or chiplets, but there are some common concerns for all configurations.<\/p>\n<p>\u201cThe hardest problems are scalability, congestion management, traffic fairness, latency predictability, and achieving timing closure across increasingly heterogeneous IP blocks,\u201d said William Wang, CEO of <a href=\"https:\/\/semiengineering.com\/entities\/alpha-design-ai-chipagents\/\" rel=\"nofollow noopener\" target=\"_blank\">ChipAgents<\/a>.<\/p>\n<p>From an SoC fabric perspective, the most complex problems all scale together. \u201cThat\u2019s what makes them hard,\u201d said Andy Nightingale, vice president of product management and marketing at <a href=\"https:\/\/semiengineering.com\/entities\/arterisip\/\" rel=\"nofollow noopener\" target=\"_blank\">Arteris<\/a>. \u201cAs SoCs grow into hundreds or thousands of endpoints, you\u2019re no longer just connecting blocks. You\u2019re managing a living traffic system under tight power, latency, and floorplan constraints. Wiring congestion, timing closure, and performance are inseparable from topology and placement. Cache coherency and traffic ordering directly affect QoS (quality of service) and worst-case latency. Heterogeneous integration \u2014 of CPUs, GPUs, NPUs, accelerators, and chiplets \u2014 multiplies clock, power, and protocol domains.\u201d<\/p>\n<p>AI designs compound some of these challenges. \u201cThe fabric must absorb bursty, high-fan-in traffic without collapsing into head-of-line blocking or pathological congestion,\u201d said Nightingale. \u201cThat\u2019s why modern NoCs have evolved well beyond simple crossbars or rings. The fabric has to be architected as a scalable system, not treated as glue logic after the real IP is finished.\u201d<\/p>\n<p>Each problem and solution impacts another, requiring tradeoffs. \u201cFor example, if you increase layering density, that gives you better performance, but it makes your physical design more difficult,\u201d said Kent Orthner, principal solutions architect at <a href=\"https:\/\/semiengineering.com\/entities\/baya-systems\/\" rel=\"nofollow noopener\" target=\"_blank\">Baya Systems<\/a>. \u201cOne problem stands out \u2014 not necessarily as the most difficult, but as maybe a solution to the others \u2014 and that\u2019s heterogeneity.\u201d<\/p>\n<p>Because of silicon scaling issues at the leading edge, engineers are customizing network solutions for the end application rather than building general-purpose solutions.<\/p>\n<p>\u201cAlthough there are a lot of difficulties, a lot of it is helped by the fact that you\u2019re doing heterogeneous design, which means that you\u2019re not trying to do everything the same way,\u201d said Orthner. \u201cYou\u2019re coming up with different types of processors, and different types of compute, and different types of networks and topologies, maybe all within a single SoC to solve different flavors of problems.\u201d<\/p>\n<p>While heterogeneity solves some problems, it also creates integration challenges. \u201cMost organizations are layering AI accelerators and real-time control workloads onto legacy platforms never designed for this level of heterogeneity or concurrency,\u201d said Nightingale. \u201cThat\u2019s driving a shift toward treating the on-chip interconnect as a real-time system fabric rather than a passive transport. Early architectural exploration, physically aware automation, and policy-driven data routing are being utilized to manage the growing data volumes while maintaining predictability and security.\u201d<\/p>\n<p><strong>Different topologies for different challenges<\/strong><br \/>To meet changing data needs, NoC topologies have evolved from crossbar to star, ringtree, mesh, Torus, and other newer formations. A complex system can have multiple NoCs in a range of different topologies.<\/p>\n<p>\u201cWe see hybrid fabrics mixing mesh, Torus, and hierarchical clusters with soft-tiled domains, configurable coherency islands, and adaptive routing to balance bandwidth and power,\u201d said ChipAgents\u2019 Wang. \u201cIn the near future, we expect dynamic, self-optimizing fabrics with agent-driven traffic tuning, congestion forecasting, and runtime topology morphing based on workload patterns.\u201d<\/p>\n<p>Others agree that the next topologies are hybrids with more hierarchy and adaptivity. \u201cFuture fabrics will look less like a single named shape and more like a composed system \u2014 optimized locally, coordinated globally \u2014 designed to evolve with packaging, chiplets, and AI workloads rather than fight them,\u201d said Arteris\u2019 Nightingale. \u201cThe mission objective is to make the fabric disappear as a problem, even as the chip keeps getting bigger.\u201d<\/p>\n<p><img data-recalc-dims=\"1\" fetchpriority=\"high\" decoding=\"async\" class=\"alignnone size-full wp-image-24274377\" src=\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-24-at-2.01.08-PM.png\" alt=\"\" width=\"1442\" height=\"918\"  \/><br \/><strong>Fig. 1: Traditional SoC interconnect fabrics. <\/strong><strong>Source<\/strong><strong>: Arteris<\/strong><\/p>\n<p>The latest approaches are explicitly multi-dimensional. \u201cWe support different topologies \u2014 trees, meshes, hybrids, and domain-specific fabrics \u2014 used simultaneously within the same SoC, because no single topology is optimal everywhere,\u201d Nightingale observed. \u201cCoherent fabrics are essential for CPU clusters, where software consistency, shared memory, and fine-grained synchronization dominate. Non-coherent fabrics are often the preferred choice for NPUs, DSPs, and streaming accelerators, where bandwidth, determinism, and energy efficiency matter more than global coherency. Soft tiling and hierarchical fabrics allow designers to scale capacity and bandwidth while containing timing, and wiring complexity. System-level traffic management, consisting of virtual networks, QoS, isolation, and congestion awareness, ensures that one aggressive workload doesn\u2019t poison the rest of the chip. The goal isn\u2019t theoretical elegance \u2014 it\u2019s predictable behavior at scale, even when traffic patterns are ugly.\u201d<\/p>\n<p><img loading=\"lazy\" data-recalc-dims=\"1\" decoding=\"async\" class=\"alignnone size-full wp-image-24274378\" src=\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-24-at-2.01.22-PM.png\" alt=\"\" width=\"1936\" height=\"1496\"  \/><br \/><strong>Fig. 2: NoC tiles for AI\/ML with network interface units (NIUs), presented at the register transfer level (RTL). Source: Arteris<\/strong><\/p>\n<p>However, new problems need new topologies. \u201cOne of the key things that we are trying to solve within a 2D space on-die is whether we can come up with fundamentally new topologies that have never been invented before,\u201d said Saurabh Gayen, chief solutions architect at Baya Systems. \u201cHow do we do very algorithmic, software-based hardware design, which is a top-down view of things? You holistically look at the system, you figure out, \u2018What do I want my entire thing to look like?\u2019 But then you look at the bottom up, because this actually needs to be built in the end. So then, what are the component pieces that you must have? A software-defined flow takes advantage of tools that were not available before. We were able to internally use those exploration tools to come up with fundamentally new topologies, which can do a 3D type of architecture within a 2D chip.\u201d<\/p>\n<p><img loading=\"lazy\" data-recalc-dims=\"1\" decoding=\"async\" class=\"alignnone size-full wp-image-24274379\" src=\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-24-at-2.01.36-PM.png\" alt=\"\" width=\"2116\" height=\"1398\"  \/><br \/><strong>Fig. 3: A 3D architecture within a 2D chip. Source: Baya Systems<\/strong><\/p>\n<p>Further, some types of ICs are suited to one specific topology. \u201cThe only topology that works for a switch \u2014 for example, an Ethernet switch that all these big guys are building \u2014 is a crossbar,\u201d said Gayen. \u201cThe problem with crossbars is that they\u2019re so bad at scaling. A crossbar that has a large number of ports on a switch is incredibly expensive and incredibly complex.\u201d<\/p>\n<p>Other chips may benefit from a custom solution. \u201cIf you\u2019re doing machine learning inference, there are very specific data patterns that you\u2019re going to see again and again, and you can design the network to support that,\u201d Orthner explained. \u201cWe encourage a definition of the topology specifically for the application. Imagine you\u2019ve got a bunch of high-performance computing cores in a corner somewhere. You might have a topology that supports a lot of connectivity in that corner. But as you do your network configuration, it might be much more linear because you don\u2019t have the same constraints. It starts to become a custom topology for every design, instead of a specific term like a mesh or a Torus. We are seeing an interesting class of designs.\u201d<\/p>\n<p>Choices come down to the provider and how they\u2019ve chosen to optimize their system. \u201cAHB (advanced high-performance bus) was one of the first networks-on-chip, a parallel bus,\u201d said Mick Posner, senior product marketing group director for chiplets and IP solutions at <a href=\"https:\/\/semiengineering.com\/entities\/cadence-design-systems\/\" rel=\"nofollow noopener\" target=\"_blank\">Cadence<\/a>. \u201cAXI (advanced eXtensible interface) came next, and it was point-to-point, lots and lots of wires, which was great for performance \u2014 but not when you are spanning a whole monolithic die, for example. Having that many parallel wires going across became problematic, and that\u2019s where you ended up with a true network-on-chip, which was more serial connections, minimal wires going across the NoC. And then those have been extended to go first, chip-to-chip, and now are being tailored for die-to-die. That topology is based on an architectural-level choice. You can mix and match. It is still typical today to have, say, a new NoC with something like an AHB port that drives into an AHB NoC with standard APB (advanced peripheral bus) peripherals, on low speed. I ran the AMBA (advanced microcontroller bus architecture) peripherals [while at Synopsys], and what was amazing is that, after 30 years, I found out that the whole product line was still alive. What I drove 20-plus years back was just getting adopted.\u201d<\/p>\n<p><strong>Different chips, different problems<\/strong><br \/>All NoCs are fabrics, but not all fabrics are NoCs, and the full range of technologies is needed to meet different chip challenges.<\/p>\n<p>\u201cFabric is a generic term,\u201d said Synopsys\u2019 Shukla. \u201cThere is FPGA fabric, then you have on-chip fabric, then you have network fabric. When you talk about hyperscalers, they call the whole plane a fabric. They mean a rack of accelerators is a fabric. So how do they talk with each other?\u201d<\/p>\n<p>Multi-chip design discussions are often around the network. \u201cWhat most people think of when they talk about networks is this picture of GPUs, CPUs, and a data center, and there are wires connecting between them,\u201d said Baya\u2019s Gayen.<\/p>\n<p>On-chip networks also have cores, memory, and accelerators, which need to talk to each other, but, unlike a data center, there aren\u2019t cables. \u201cYou don\u2019t have a 3D space,\u201d Gayen explained. \u201cYou have a 2D space, so the topology there is typically something that you can layer within that space and get really good performance without being able to overlay the cables that you have in a 3D data center space. Even though it\u2019s a very common networking theory that underlies both, it\u2019s very interesting to see how those things diverge and how the problem-solving happens.\u201d<\/p>\n<p>Fast-growing application areas like physical AI systems, including robots, drones, or vehicles, also have distinct concerns. \u201cPhysical AI systems operate in continuous, closed-loop interaction with the real world \u2014 combining sensing, computation, and actuation under strict real-time and safety constraints,\u201d said Nightingale. \u201cIn these systems, delayed or misrouted data isn\u2019t only inefficient but also potentially unsafe. Data management, therefore, expands to include deterministic latency, traffic isolation, and fault containment, ensuring that safety-critical control flows are protected from best-effort AI traffic and that data leakage or bias introduced by uneven access is avoided.\u201d<\/p>\n<p>What is a big networking challenge in one domain of chips is not necessarily challenging in another. For example, coherence applies to some systems but not others.<\/p>\n<p>\u201cWith more and more complex SoCs, the coherency side of things is a real challenge to deal with,\u201d said Gayen. \u201cIn an SoC, the performance is often limited by having to coordinate all that nitty-gritty stuff about coherency. That really takes you back, and you can\u2019t reach crazy levels of performance.\u201d<\/p>\n<p>AI chips sidestep the coherency problem. \u201cThey don\u2019t have built-in coherency protocols, so that\u2019s not the challenge for them, because they handle it through software-based coherency or an algorithmic approach to data flow,\u201d Gayen said. \u201cThere, the challenge is how to balance a good ratio of really good NoC performance, versus how much I\u2019m allocating to compute or buffering.\u201d<\/p>\n<p>Switches are in a completely different domain, where they\u2019re pushing the physical limits. \u201cThey\u2019re just trying to figure out how to cram as many wires as possible in a way that an engineering team can actually handle and build,\u201d he continued. \u201cTheir big problem is how to get an incredibly crazy type of performance. The switches care about the non-blocking property and throwing more white space and wiring at the problem, but having way more stringent performance requirements. The wire density aspect becomes critical. The engineering cost of tiling and having such a big NoC design, or a big crossbar design, is the challenge.\u201d<\/p>\n<p>Since AI chips sit in the middle of a complex SoC and a switch, they are not doing hardware-based coherency as much, and are not doing full-on white space, throwing a bunch of wires at a huge performance problem. \u201cThey want something that kind of looks like an SoC, but kind of looks like a switch, as well. They might take an SoC-type design but then scale it by having way more parallel layers of wiring that can distribute and get 2X, 4X, 8X the bandwidth, but really go for raw performance compared to what an SoC does,\u201d Gayen said.<\/p>\n<p>Also, there is a common thread between the domains \u2014 the insatiable thirst for data. \u201cEverybody wants to get their data moved around more efficiently,\u201d Orthner said.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-24274380\" src=\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-24-at-2.02.03-PM.png\" alt=\"\" width=\"2114\" height=\"1096\"  \/><br \/><strong>Fig. 4: Comparison of expert users (manual) vs. NoC IP (automatic) for real-world designs. Source: Arteris <\/strong><\/p>\n<p><strong>Chiplet challenges<\/strong><br \/>Like SoCs, chiplet NoCs require decisions around coherency. Within a chiplet, individual subsystems act as the function. \u201cIt could be the memory LPDDR subsystem or a PCIe subsystem, but each of those subsystems has its own NoC to facilitate the communication within that subsystem,\u201d said Cadence\u2019s Posner. \u201cThen there could be a higher-level NoC that allows subsystems to talk to subsystems. In that scenario, our chiplet was non-coherent. It did not require any cache coherency, which is required if it were a CPU-to-CPU connection. The CPU provider will always have a coherent NoC. Then there are third-party options that offer coherent and non-coherent NoCs. It\u2019s down to what you are connecting to. If you\u2019re designing a CPU chiplet, then you must have a coherent NoC, because you\u2019re likely going to be performance scaling by putting down multiple dies of that same thing, and you\u2019re doing CPU scaling. That has to have cache coherency. But if you\u2019re going from CPU to an accelerator, then it\u2019s usually just I\/O coherency you need. That also drives the type of NoC that is used.\u201d<\/p>\n<p>Chiplets add more data challenges. \u201cIn the system, especially with the high speed I\/Os, you have to think about how you\u2019re going to manage the communications between the other chips,\u201d said Hee Soo Lee, high-speed digital design segment lead at <a href=\"https:\/\/semiengineering.com\/entities\/keysight-technologies\/\" rel=\"nofollow noopener\" target=\"_blank\">Keysight EDA<\/a>. \u201cWhen we are thinking about data, all of these tiny chiplets are connected. How are you going to maintain really good bandwidth requirements, along with how you manage the latencies between chips, especially with all of those tiny pieces of chiplets? Data is one of the main issues and problems. In delivering all the power to those chips, are you going to make data cleaner? You need to make sure there are signals throughout the I\/O wide buses so they don\u2019t interfere with each other and don\u2019t result in a closed eye or shrunken margins for the system. Just making everything stay cool is also very, very challenging.\u201d<\/p>\n<p>I\/O chiplets bring further considerations. \u201cIf you\u2019re splitting your core die with I\/O die, should you share the fabric, or should the SoC have the fabric and the I\/O die just pass on data?\u201d Shukla noted.<\/p>\n<p>Overall, the chiplet approach is more complicated because it leads to bigger overall systems. \u201cThe sheer scale of what you can build when you put multiple chiplets together just grows and grows,\u201d said Baya\u2019s Orthner. \u201cAlso, there\u2019s a degree of runtime configurability that pops up when you\u2019re working with chiplets that you don\u2019t see so much in a straight-up SoC.\u201d<\/p>\n<p><strong>Conclusion<\/strong><br \/>NoC and fabric designers have a lot to consider in today\u2019s landscape, with the ever-increasing demands of AI leading to more data, and more traffic congestion.<\/p>\n<p>A key shift is that AI workloads have invalidated \u201caverage-case\u201d assumptions. \u201cTraining stresses sustained bandwidth, multicast efficiency, and memory coherence at massive scale,\u201d Arteris\u2019 Nightingale said. \u201cInference \u2014 particularly at the edge \u2014 demands bounded, predictable latency. In both cases, data quality and correctness are no longer purely software problems. When memory traffic can consume 80% to 90% of inference time or dynamic energy, congestion, timing variability, or silent data corruption directly impact model behavior and system outcomes.\u201d<\/p>\n<p>Finally, the impact of network problems and data bottlenecks is becoming more critical. \u201cAs AI evolves from digital inference into physical, real-world interaction, data management failures no longer degrade performance gracefully,\u201d Nightingale said. \u201cThey surface as accuracy loss, safety risk, or unscalable systems. Designing for deterministic, observable, and adaptable data movement is no longer optional \u2014 it\u2019s foundational. Efficiency follows discipline.\u201d<\/p>\n<p><strong>Related Articles<\/strong><\/p>\n<p><a href=\"https:\/\/semiengineering.com\/ucies-major-technical-components-are-now-in-place\/\" rel=\"nofollow noopener\" target=\"_blank\">UCIe\u2019s Major Technical Components Are Now In Place<\/a><br \/>Version 3.0 of the interconnect standard doubles bandwidth and supports new use cases and enhanced manageability.<\/p>\n<p><a href=\"https:\/\/semiengineering.com\/multiple-ai-scale-up-options-emerge\/\" rel=\"nofollow noopener\" target=\"_blank\">Multiple AI Scale-Up Options Emerge<\/a><br \/>As data center infrastructures adapt to evolving workloads, parts of Ethernet can be found in scale-up approaches.<\/p>\n<p><a href=\"https:\/\/semiengineering.com\/often-overlooked-phys-are-essential-for-high-speed-bandwidth\/\" rel=\"nofollow noopener\" target=\"_blank\">Often Overlooked, PHYs Are Essential To High-Speed Data Movement<\/a><br \/>From smartphones to AI factories, physical layers are the unsung heroes of data communications.<\/p>\n<p><a href=\"https:\/\/semiengineering.com\/die-to-die-interconnect-standards-in-flux\/\" rel=\"nofollow noopener\" target=\"_blank\">Die-To-Die Interconnect Standards In Flux<\/a><br \/>Many features of UCIe 2.0 seen as \u201cheavy\u201d are optional, causing confusion.<\/p>\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"Key Takeaways: NoC challenges, such as wiring congestion, timing closure, and performance, must be considered in tandem with&hellip;\n","protected":false},"author":2,"featured_media":404651,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_share_on_mastodon":"0"},"categories":[74],"tags":[153606,166973,103319,166974,18,19,183159,17,183160,183161,183162,40831,82],"class_list":["post-404650","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology","tag-arteris","tag-baya-systems","tag-cadence","tag-chipagents","tag-eire","tag-ie","tag-interconnect-fabrics","tag-ireland","tag-keysight-eda","tag-network-on-chip","tag-nocs","tag-synopsys","tag-technology"],"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@ie\/116294709706854218","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts\/404650","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/comments?post=404650"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts\/404650\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/media\/404651"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/media?parent=404650"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/categories?post=404650"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/tags?post=404650"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}