As AI agents take on more end-to-end business processes, the researchers argue that multilinguality should be evaluated across the entire workflow to better reflect how AI is used in global enterprises.
To address this, they developed PolyWorkBench, a benchmark comprising 67 manually designed enterprise tasks across five domains: commerce, knowledge work, legal analysis, manufacturing, and localization.
Each task represents a complete multilingual business workflow, where agents receive instructions together with supporting materials — such as documents, spreadsheets, contracts, or multilingual resources — and must produce outputs that meet predefined business requirements. Depending on the task, they may need to retrieve information across languages, use external tools, combine intermediate results, and generate reports, spreadsheets, legal documents, or localization deliverables.
A key design feature is that multilinguality is embedded throughout the workflow rather than treated as a separate translation step. As agents complete each task, they must preserve meaning while moving between languages during intermediate reasoning and final output generation. According to the researchers, this allows the benchmark to expose failures that traditional multilingual benchmarks and monolingual agent benchmarks are unable to capture.
The benchmark covers ten languages — including English, Chinese, Japanese, Korean, Vietnamese, Russian, French, Spanish, German, and Arabic — and 88% of tasks involve three or more languages.
2026 Slator Market Report: Language Solutions & AI
The 130-page Slator Report maps a USD 30.85 billion global market shaped by multilingual AI, enterprise AI operationalization, and the convergence of language technology, media, accessibility, and real-time communication.
Multilingual Performance Remains a Challenge
The researchers evaluated a range of frontier and open-weight models and found that current AI agents still struggle with multilingual enterprise workflows. “State-of-the-art LLM agents experience substantial degradation in multilingual workflow settings, even when they perform strongly on monolingual long-horizon benchmarks,” they said.
The results also showed that performance varied considerably across both business domains and languages. No single model consistently performed best across all enterprise workflows, and some systems showed much larger performance gaps between languages than others. According to the researchers, multilingual agents most commonly failed either by misunderstanding information in the source language or by losing consistency between source and target languages across multiple steps.
In the benchmark’s localization tasks — which include multilingual adaptation of software, documentation, subtitles, and marketing content — the researchers found that multilingual capabilities mattered more than sophisticated long-horizon planning. This suggests that strong multilingual performance remains a key requirement for AI systems handling localization workflows, even as they become more agentic. “Localization tasks reward strong multilingual base models more than they reward long-horizon planning,” the researchers said.
The researchers said that the benchmark is intended to support future research on multilingual AI agents and plan to expand it with additional languages, domains, and enterprise workflows over time.
Authors: Hongliang Li, Yijin Liu, Zhiwei Zhang, Zihe Liu, Xinyue Lou, Jinan Xu, Fandong Meng, and Kaiyu Huang