Chinese AI companies are optimising software to cope with surging demand for inference, as part of that workload still relies on computing power from a limited pool of high-end chips amid restricted access to Nvidia processors.<\/p>\n
Compared with training an artificial intelligence model, which relies on high-end chips, inference \u2013 a later phase in which the trained model applies its knowledge to process responses \u2013 can be adapted to domestic hardware. However, industry insiders said complex tasks like coding still required Nvidia chips, which meant the sector was facing acute compute constraints as AI moved from model development to large-scale deployment.<\/p>\n
\u201cThe demand side is now showing a bipolarisation,\u201d said Guan Jiawei, vice-president of inference optimisation start-up Approaching.AI, noting that demand for high-quality tokens \u2013 the basic units of data that models process and generate \u2013 far outstripped supply.<\/p>\n
High-tier tasks required stringent performance metrics that domestic processors could not yet reliably deliver, Guan said, adding that advanced Chinese models \u201cplace high demands on chips \u2026 especially in scenarios like coding, where users are willing to pay a premium\u201d.<\/p>\n
\u201cIf we rely solely on domestic chips for inference, they can only handle the low-quality tier \u2013 the tier with weak demand and weak monetisation,\u201d Guan said. \u201cThat makes it very hard to find a viable commercial path. That\u2019s why high-quality tokens still depend on Nvidia.\u201d<\/p>\n
Skyrocketing token usage, as AI turns more agentic \u2013 performing real-world tasks rather than just answering questions \u2013 has exacerbated the compute squeeze. China\u2019s average daily token calls exceeded 140 trillion in March, up more than 1,000-fold from the beginning of 2024, according to the National Data Administration.<\/p>\n
Will China\u2019s new AI model Kimi K3 create another DeepSeek moment?<\/p>\n","protected":false},"excerpt":{"rendered":"Chinese AI companies are optimising software to cope with surging demand for inference, as part of that workload…\n","protected":false},"author":2,"featured_media":146151,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[24,71889,25,71888,387,935,6010,933,56012,71890,30971,56740,58,71887,24729],"class_list":["post-146150","post","type-post","status-publish","format-standard","has-post-thumbnail","category-ai","tag-ai","tag-approaching-ai","tag-artificial-intelligence","tag-ascend-910b","tag-china","tag-chinese","tag-deepseek","tag-huawei-technologies","tag-kimi-k3","tag-ktransformers","tag-moonshot","tag-national-data-administration","tag-nvidia","tag-nvidia-h20","tag-nvidia-h200"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/146150","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=146150"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/146150\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/146151"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=146150"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=146150"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=146150"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}