{"id":244346,"date":"2025-07-07T04:14:26","date_gmt":"2025-07-07T04:14:26","guid":{"rendered":"https:\/\/www.europesays.com\/uk\/244346\/"},"modified":"2025-07-07T04:14:26","modified_gmt":"2025-07-07T04:14:26","slug":"128-days-later-semianalysis","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/uk\/244346\/","title":{"rendered":">128 Days Later\u00a0 \u2013 SemiAnalysis"},"content":{"rendered":"

SemiAnalysis is hiring an analyst in New York City for Core Research, our world class research product for the finance industry.\u00a0Please apply here<\/a><\/p>\n

It\u2019s been a bit over 150 days since the launch of the Chinese LLM DeepSeek R1 shook stock markets and the Western AI world. R1 was the first model to be publicly released that matched OpenAI\u2019s reasoning behavior. However, much of this was overshadowed by the fear that DeepSeek (and China) would commoditize AI models given the extremely low price<\/a> of $0.55 input\/$2.19 output, undercutting the then SOTA model o1 by 90%+ on output token pricing. Reasoning model prices have dropped significantly since, with OpenAI recently dropping their flagship model price by 80%.\u00a0\u00a0<\/p>\n

\"\"Source: SemiAnalysis, Company prices<\/p>\n

\u00a0R1 got an update as DeepSeek continued to scale RL after release. This resulted in the model improving in many domains, particularly coding. This continuous development and improvement is a hallmark of the new paradigm we previously covered.\u00a0\u00a0\u00a0<\/p>\n

Today we look at DeepSeek\u2019s impact on the AI model race and the state of AI market share.\u00a0<\/p>\n

A Boom and\u2026 Bust?\u00a0<\/p>\n

Consumer app traffic to DeepSeek spiked following release, resulting in a sharp increase in market share. Because Chinese usage is poorly tracked and Western labs are blocked in China, the numbers below understate DeepSeek\u2019s total reach. However the explosive growth has not kept pace with other AI apps and DeepSeek market share has since declined.\u00a0<\/p>\n

\"\"Source: SemiAnalysis, SensorTower<\/p>\n

For web browser traffic, the data is even more grim with DeepSeek traffic down in absolute terms since release. The other leading AI model providers have all seen impressive growth in users over the same time frame.\u00a0\u00a0<\/p>\n

\"\"Source: SemiAnalysis, SimilarWeb<\/p>\n

The poor user momentum for DeepSeek-hosted models stands in sharp contrast to third party hosted instances of DeepSeek. Aggregate usage of\u00a0 R1 and V3 on third party hosts continues to grow rapidly, up nearly 20x since R1 first released.\u00a0\u00a0<\/p>\n

\"\"Source: SemiAnalysis, OpenRouter<\/p>\n

Digging deeper into the data, by splitting out the DeepSeek tokens into just those hosted by the company itself, we can see that DeepSeek\u2019s share of total tokens continues to fall every month.\u00a0\u00a0<\/p>\n

\"\"Source: SemiAnalysis, OpenRouter<\/p>\n

So why are users shifting away from DeepSeek\u2019s own web app and API service in favor of other open source providers despite the rising popularity of DeepSeek\u2019s models and the apparently very cheap price?\u00a0\u00a0<\/p>\n

The answer lies in tokenomics and the myriad of tradeoffs between the KPIs for serving a model. These tradeoffs mean a model\u2019s price per token is an OUTPUT of these KPI decisions which can be tuned based on the model providers\u2019 hardware and model setup.\u00a0<\/p>\n

Tokenomics Basics\u00a0<\/p>\n

Tokens are the fundamental building blocks of AI models. AI models can learn through reading the internet in token form and produce output in the form of text, audio, image, or action tokens. A token is just a bite-sized chunk of text (like \u201cfan\u201d, \u201ctas\u201d, \u201ctic\u201d) that a large language model counts and processes instead of whole words or letters.\u00a0<\/p>\n

When Jensen talks about datacenters becoming AI factories, the input and output of these factories are tokens. Much like a physical factory, AI factories make money with a P x Q equation: P is the price per token and Q is the quantity of input and output tokens.\u00a0\u00a0<\/p>\n

Unlike a normal factory, the token price is a variable that model providers can solve for based on the other attributes of the model. We list the key KPIs below\u00a0<\/p>\n

    \n
  1. Latency or Time-to-First-Token<\/strong>: how long a model takes to generate a token. This is also known as \u2018time to first token\u2019 or approximately how long it takes the model to complete the prefill stage (ie encoding input tokens into the KVCache) and start producing the first token in the decode stage.\u00a0<\/li>\n<\/ol>\n
      \n
    1. Interactivity<\/strong>: how fast each token is produced, oftentimes measured in tokens per second per user. Some providers also talk about the inverse of interactivity which is the average time between each output token (time per output token or TPOT). Human reading speed is 3-5 words per second but most model providers have settled on output speeds of around 20-60 tokens per second.\u00a0\u00a0<\/li>\n<\/ol>\n
        \n
      1. Context Window<\/strong>: how many tokens can be held in the \u2018short term memory\u2019 of the model before earlier tokens are evicted and the model \u2018forgets\u2019 the older parts of the conversation. Different use cases require different context windows. Large document and code base analysis benefit from larger context windows that allow the model to coherently reason over data.\u00a0<\/li>\n<\/ol>\n

        For any given model, you can manipulate these 3 KPIs to produce effectively any price per token. Therefore it is not always productive or practical to discuss tokens on purely price per million-token ($\/Mtok) as this ignores the nature of the workload and requirements of the token user.\u00a0\u00a0<\/p>\n

        Subscribe to get notified of all SemiAnalysis articles<\/strong><\/p>\n

        Please verify your email address to proceed.<\/p>\n

        By subscribing, you agree to the\u00a0Privacy Policy<\/a>\u00a0and\u00a0Terms and Conditions<\/a>.<\/p>\n

        DeepSeek Trade-Offs\u00a0<\/p>\n

        Now let\u2019s look at the tokenomics of how DeepSeek serves its R1 model to understand why they have been losing market share on their own model.\u00a0<\/p>\n

        \"\"Source: https:\/\/openrouter.ai\/<\/a> accessed in May 2025. Blended $\/Mtok calcuated with 3:1 input:output ratio<\/p>\n

        Plotting Latency against Price, we can see that DeepSeek\u2019s own service is no longer the cheapest for its latency. In fact, a big reason why DeepSeek is able to price their product so cheaply is because they force users to wait many seconds before the model responds with the first token. This compares to some other providers offering it for the same price but delaying responses by much less time. Token consumers can pay $2-4 for nearly no latency with providers like Parasail or Friendli. Microsoft Azure offers the service for 2.5x more than DeepSeek but with 25s less latency. Since we pulled this data, the situation has become even more grim for DeepSeek as almost all R1 0528 instances are now hosted with sub-5 second latencies<\/a>.\u00a0<\/p>\n

        \"\"Source: https:\/\/openrouter.ai\/<\/a> accessed in May 2025. Blended $\/Mtok calcuated with 3:1 input:output ratio, bubble size represents context window size<\/p>\n

        Using the same plot but adding bubble size for the context window, we can see another tradeoff that DeepSeek runs to deliver a very cheap model with limited inference compute resources. They run a 64K context window which is one of the smallest of the major model providers. Smaller context windows limit use cases like coding which require a model to coherently remember a large amount of tokens across a code base to reason across. At the same price you can get >2.5x the context size with providers like Lambda and Nebius in the above chart.\u00a0<\/p>\n

        \"\"Source: SemiAnalysis benchmarks<\/a><\/p>\n

        Digging into hardware, we can see with the above benchmarking of AMD and NVDA<\/a> chips on DeepSeek V3 how providers solve for $\/Mtok: by batching more users simultaneously on a single GPU or cluster of GPUs, the model provider can INCREASE the total wait experienced by the end user with higher latency and slower interactivity (measured by the x-axis in Median End to End Latency per User) to DECREASE the total cost per token. Higher batch sizes and slower interactivity will reduce the cost per token at the expense of a much worse user experience.\u00a0\u00a0<\/p>\n

        To be clear, this is an active decision by DeepSeek. They are not interested in making money off users or in serving them lots of tokens via a chat app or an API service. The company is singularly focused on reaching AGI and is not interested in end user experience.\u00a0\u00a0<\/p>\n

        Batching at extremely high rates allows them to use the minimal amount of compute possible for inference and external usage. This keeps the maximal amount of compute internal for research and development purposes. As we have previously discussed<\/a>, export controls have limited the Chinese ecosystem\u2019s capability in serving models. As such, for DeepSeek, it makes sense to open source. Whatever compute they have is kept internal, while other clouds can host their model so they can win mind share and global adoption. While the export controls have greatly limited China\u2019s capability in inferencing models at scale, we do not believe it has equally hindered their ability to train a useful model as evidenced by recent releasees from Tencent<\/a>, Alibaba<\/a>, Baidu,<\/a> and even Rednote<\/a>.\u00a0<\/p>\n

        Anthropic is More Like DeepSeek than They\u2019d like to Admit\u00a0<\/p>\n

        In the world of AI, the only thing that matters is compute. Like DeepSeek, Anthropic is compute constrained. Anthropic has focused their product development on code and have seen strong adoption among coding applications like Cursor. We think that Cursor usage is the ultimate eval as it represents what users care about most: cost <\/strong>and experience<\/strong>. Anthropic has ranked first for over a year now, which is decades in the AI industry.\u00a0\u00a0<\/p>\n

        Having noticed the success of token consumers like Cursor, the company launched Claude Code, a coding tool built into the terminal. Claude Code usage has skyrocketed, leaving OpenAI\u2019s codex in the dust.\u00a0<\/p>\n

        Google, in response, also released their own tool: Gemini CLI. While it is a similar coding tool to Claude Code, Google uses their compute advantage with TPUs to offer unbelievably large request limits at no cost to users.\u00a0\u00a0<\/p>\n

        \"\"Source: Google<\/p>\n

        Claude Code, for all of its wonderful performance and design, is expensive. <\/strong>In many ways, the success of Anthropic\u2019s models in code has placed significant stress on the company. They are squeezed tight on compute.\u00a0<\/strong>\u00a0<\/p>\n

        This is most evident in Claude 4 Sonnet\u2019s output speed on the API. Since the launch of Claude 4 Sonnet, the speed has decreased by 40% to just above 45 tokens per second. The reason for this is not unlike DeepSeek\u2019s \u2013 to manage all the incoming requests with the available compute, they have to batch at higher rates. Coding usage also tends to skew towards larger token count conversations which worsens the crunch on compute resources compared to lower token count casual chat applications. Regardless, comparable models like o3 and Gemini 2.5 Pro run at significantly faster speeds, reflecting the much larger compute resources at OpenAI and Google.\u00a0\u00a0<\/p>\n

        \"\"Source: SemiAnalysis, Artificial Analysis<\/p>\n

        Anthropic is focused on acquiring more compute, striking a major deal with Amazon which we have covered before.\u00a0\u00a0 <\/p>\n

        Anthropic is getting more than half a million Trainium chips, which they will then be used for both inference and training. This relationship is still a work-in-progress as, despite popular opinion, Claude 4 was not pretrained on AWS Trainium. It was trained on GPUs and TPUs.<\/p>\n

        Anthropic also turned to their other major investor Google of compute. Anthropic rents significant amounts of compute from GCP, specifically TPUs. Following this success, Google Cloud is expanding their offerings to other AI companies, striking a deal with OpenAI. Unlike previous reporting, Google is only renting GPUs to OpenAI \u2013 not TPUs.\u00a0\u00a0<\/p>\n

        Speed Can be Compensated for\u00a0<\/p>\n

        Claude\u2019s speed is indicative of their compute constraints, but generally Anthropic\u2019s UX is better than DeepSeek. First, the speed, despite being low, is faster than DeepSeek\u2019s 25 tokens per second. Second, Anthropic models require significantly less tokens than other models to answer a question. This means that despite the speed, users experience a significantly lower end-to-end response time.\u00a0\u00a0<\/p>\n

        While this can depend on workload, Gemini 2.5 Pro and DeepSeek R1-0528 are more than 3 times as wordy as Claude. Gemini 2.5 Pro, Grok 3, and DeepSeek R1 used significantly more tokens to run Artificial Analysis\u2019 intelligence index, which aggregates several varied benchmark scores together. Indeed, Claude has the lowest amount of total output tokens for leading reasoning models and showed an impressive improvement over Claude 3.7 Sonnet.\u00a0\u00a0<\/p>\n

        This aspect of tokenomics shows that there are many dimensions on which providers are working to improve models. It is not just more intelligence, but more intelligence per token produced.<\/strong>\u00a0<\/p>\n

        \"\"Source: Artificial Analysis Intelligence Index, SemiAnalysis<\/p>\n

        Rise of the Inference Clouds\u00a0<\/p>\n

        With the meteoric rise of Cursor, Windsurf, Replit, Perplexity, and other \u201cGPT Wrappers\u201d or AI token-powered apps hitting mainstream recognition, we are seeing more and more companies emulate Anthropic\u2019s focus on selling tokens as a service, rather than bundled as a monthly subscription like ChatGPT.\u00a0<\/p>\n

        Next, we will explore what is next for DeepSeek and address rumors of a delayed R2.\u00a0<\/p>\n

        Subscribe for full access to this article<\/p>\n

        With a SemiAnalysis subscription you\u2019ll get access to newsletter articles and article discussions.<\/p>\n

        Model access not included \u2013 please reach out to sales@semianalysis.com<\/a> for our institutional offerings.<\/p>\n

        Please verify your email address to proceed.<\/p>\n

        By subscribing, you agree to the\u00a0Privacy Policy<\/a>\u00a0and\u00a0Terms and Conditions<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"SemiAnalysis is hiring an analyst in New York City for Core Research, our world class research product for…\n","protected":false},"author":2,"featured_media":244347,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3163],"tags":[323,1942,53,16,15],"class_list":{"0":"post-244346","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-technology","11":"tag-uk","12":"tag-united-kingdom"},"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@uk\/114810012987595843","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts\/244346","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/comments?post=244346"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts\/244346\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/media\/244347"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/media?parent=244346"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/categories?post=244346"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/tags?post=244346"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}