![\"\"](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/04\/Screenshot-2026-04-16-at-9.24.31-AM.png\")
Download the report.<\/a><\/p>\nAccording to the analysis, agencies often underestimate the total cost of AI implementation by focusing on the models themselves and overlooking significant expenses such as data preparation (20-30% of project costs), regulatory compliance (up to 50% higher in regulated fields), and infrastructure upgrades (15-25%).<\/p>\n
This financial strain is compounded by what the report refers to as the \u201cblack box dilemma.\u201d Large language models (LLMs), often trained on the entire internet, operate as opaque systems. Agencies using them lack insight into the training data, algorithmic weighting, and potential for built-in biases.<\/p>\n
\u201cIf someone shows up on your doorstep with an AI black box, and you don\u2019t know what data went into creating that or how it was valued\u2026 then you can run into some really interesting problems,\u201d warns Adam Clater, Chief Architect in Red Hat\u2019s CTO organization, in the report. For agencies handling everything from classified intelligence to sensitive healthcare records, this lack of control is not a viable option.<\/p>\n
The report advocates for a strategic pivot from \u201cblack box\u201d to \u201cglass box\u201d AI, championing the open-source alternative. This approach enables federal agencies to inspect, understand, and trust the AI tools they deploy. The analysis directly aligns with recent White House directives that mandate agencies pursue transparent, adaptable, and open-weight AI models to protect privacy, avoid vendor lock-in, and ensure cost-effectiveness.<\/p>\n
Smaller models can yield better results<\/p>\n
Challenging the prevailing \u201cbigger is better\u201d myth, the report argues that smaller, specialized AI models trained on specific agency data are more efficient and effective. Clater offers a powerful analogy: memorizing a dictionary is more practical for specific tasks than memorizing an entire encyclopedia. \u201cTests have shown that smaller models can give you a high percentile completeness level without memorizing the entirety of the encyclopedia,\u201d he states.<\/p>\n
This specialized approach is already proving effective in highly regulated environments. The report explains how federal agencies can use fine-tuned AI models trained on internal case files and regulatory texts to identify sophisticated fraud more accurately than their larger, more general counterparts.<\/p>\n
The report highlights another advantage of smaller, purpose-built models: the ability to bring AI to the data, rather than sending sensitive data to a third-party model. This ensures data sovereignty and enables powerful new applications at the tactical edge\u2014from providing real-time intelligence to wildfire incident commanders to accelerating decision-making for military leaders on the battlefield. \u201cBeing able to take our AI directly to where the mission is happening in real time is going to bring tremendous value,\u201d says Clater.<\/p>\n
The report points to recent initiatives by Red Hat to help agencies leverage smaller, open-source AI technologies, including advancements in Red Hat OpenShift AI and its recent acquisition of Neural Magic, which enables agencies to run optimized AI models on existing hardware. These tools, combined with developer community projects like InstructLab, aim to empower agency domain experts\u2014not just data scientists\u2014to contribute to and refine AI models.<\/p>\n
The report concludes that the future of AI will increasingly lean toward open-source solutions because they are more accessible, powerful, and secure.<\/p>\n
![\"Scoop](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/04\/1776889093_490_ea8b076b398ee48b71cfaecf898c582b.jpeg\")
<\/p>\n