{"id":141132,"date":"2025-10-23T18:55:14","date_gmt":"2025-10-23T18:55:14","guid":{"rendered":"https:\/\/www.europesays.com\/ie\/141132\/"},"modified":"2025-10-23T18:55:14","modified_gmt":"2025-10-23T18:55:14","slug":"tensormesh-raises-4-5m-to-squeeze-more-inference-out-of-ai-server-loads","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ie\/141132\/","title":{"rendered":"Tensormesh raises $4.5M to squeeze more inference out of AI server loads"},"content":{"rendered":"

With the AI infrastructure push reaching staggering proportions, there\u2019s more pressure than ever to squeeze as much inference as possible out of the GPUs they have. And for researchers with expertise in a particular technique, it\u2019s a great time to raise funding.<\/p>\n

That\u2019s part of the driving force behind Tensormesh<\/a>, launching out of stealth this week with $4.5 million in seed funding. The investment was led by Laude Ventures, with additional angel funding from database pioneer Michael Franklin<\/a>.<\/p>\n

Tensormesh is using the money to build a commercial version of the open source LMCache<\/a> utility, launched and maintained by Tensormesh co-founder Yihua Cheng. Used well, LMCache can reduce inference costs by as much as 10x \u2014 a power that\u2019s made it a staple in open source deployments and drawn in integrations from heavy hitters like Google<\/a> and Nvidia<\/a>. Now Tensormesh is planning to parlay that academic reputation into a viable business.<\/p>\n

The core of the product is the key-value cache (or KV cache), a memory system used to process complex inputs more efficiently by condensing them down to their key values. In traditional architectures<\/a>, the KV cache is discarded at the end of each query \u2014 but Tensormesh co-founder and CEO Junchen Jiang argues that this is an enormous source of inefficiency.<\/p>\n

\u201cIt\u2019s like having a very smart analyst reading all the data, but they forget what they have learned after each question,\u201d says Jiang.<\/p>\n

Instead of discarding that cache, Tensormesh\u2019s systems hold on to it, allowing it to be redeployed when the model executes a similar process in a separate query. Because GPU memory is so precious, this can mean spreading data across several different storage layers, but the reward is significantly more inference power for the same server load.<\/p>\n

The change is particularly powerful for chat interfaces, since models need to continually refer back to the growing chat log as the conversation progresses. Agentic systems have a similar issue, with a growing log of actions and goals.<\/p>\n

In theory, these are changes AI companies can execute on their own \u2014 but the technical complexity makes it a daunting task. Given the Tensormesh team\u2019s work researching the process and the intricacy of the detail itself, the company is betting there will be lots of demand for an out-of-the-box product.<\/p>\n

\u201cKeeping the KV cache in a secondary storage system and reused efficiently without slowing the whole system down is a very challenging problem,\u201d says Jiang. \u201cWe\u2019ve seen people hire 20 engineers and spend three or four months to build such a system. Or they can use our product and do it very efficiently.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"With the AI infrastructure push reaching staggering proportions, there\u2019s more pressure than ever to squeeze as much inference…\n","protected":false},"author":2,"featured_media":141133,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[261],"tags":[291,289,290,18,19,43415,17,83528,82],"class_list":{"0":"post-141132","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-eire","12":"tag-ie","13":"tag-inference","14":"tag-ireland","15":"tag-laude-ventures","16":"tag-technology"},"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@ie\/115425006219419253","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts\/141132","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/comments?post=141132"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts\/141132\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/media\/141133"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/media?parent=141132"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/categories?post=141132"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/tags?post=141132"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}