Navy tracking efficiency gains as part of AI training efforts

The Department of the Navy is one the biggest users of the Defense Department’s GenAI.mil platform.

The Navy designated the generative artificial intelligence tool as an enterprise service in just five days after evaluating it. It also told all employees to use it for controlled unclassified information and Impact Level 5 (IL5) data by April 30.

But just giving sailors, marines and civilians access to the large language models and saying “use it” without any training is not a recipe for success.

The DoN is ensuring its employees are gaining real benefits of the AI tools by mandating training and then measuring how that training is turning into mission outcomes.

]]>

The Navy, to that end, detailed in a March memo all the free AI training resources available across DoD, including everything from GenAI.mil training to Naval based LinkedIn learning to Google skills training.

Justin Fanelli, the Navy’s chief technology officer, said users must understand how to use AI from different perspectives.

Justin Fanelli, the Navy’s chief technology officer, speaking at the recent Sea, Air, Space conference.

“The way that I think about AI is there’s individualized AI and there’s institutionalized AI in terms of the functions, in terms of how you use it, in terms of use cases,” Fanelli said during the recent Sea, Air, Space Conference sponsored by the Navy League. “Institutionalized [use] is better bang for the buck, but with individualized [use], you need people using it to get to level two and level three [of AI utilization].”

The Navy told sailors, marines and civilians to log into GenAI.mil, complete at least one free training course within 30 days and then begin tracking savings. The results will be uploaded to the DoN AI Efficiency challenge tracking system to share success stories.

Fanelli said there has been a lot of excitement over applying AI tools to Navy mission areas and the memo is really a result of that demand for training.

He said the DoN needed to provide some sort of accelerated path for training around AI. So, Fanelli said the Navy chief information officer’s goal is to remove friction around AI and create a path for sailors, marines and civilians to run down.

“It’s also been helpful that Navy senior leaders from the Secretary on down have been leaning forward into AI,” he said.

]]>

The support for the expanded use of AI tools and capabilities came not just from the top at the Navy but from Defense Secretary Pete Hegseth.

DoD introduced GenAI.mil in December, initially providing access to Google Gemini as the main large language model. But soon DoD expanded it to include xAI’s Grok and eventually OpenAI’s ChatGPT.

DoD says about 3 million service members and civilians currently can access GenAI.mil. More than 1.3 million service members and civilians use the platform every day.

Every service and many defense agencies have adopted GenAI.mil and made it their preferred enterprisewide AI platform.

Among the tools GenAI.mil provides are a conversational chat interface, the ability to upload files, the use of Retrieval Augmented Generation (RAG) for sourcing answers from provided documents, secure web grounding, deep research and it allows users to organize conversations with persistent chat histories.

Five outcomes to measure

But to get servicemembers and civilians to use the tool successfully and impact mission areas, Fanelli said it’s more than just providing free training that is necessary. The Navy is trying to measure the impact of the AI tools and capabilities.

In the memo, the DoN is running a structured challenge of AI uses cases to measure the impact of these tools. The memo lays out the goal of measuring five outcomes from the AI use cases. Fanelli said the common denominator across all five outcomes based metrics is time saved.

“At a minimum, we should be getting something back. As it turns out, and as we suspected, for every use case, some are more fruitful, more impactful than others. So we just wanted to do that in the open and shared,” Fanelli said. “We put that out, and we got 300 entries on the first day. We’re in the many hundreds now, and we have, ultimately, a list or group of unleashed champions who are then collaborating within our dashboard and within the larger scoreboard. Here is the highest impact use case. Here’s how we scale those as enterprise things that everyone can use, and we’re going to raise that up at the institutional level next.”

The Navy’s memo outlines the steps for mission and business areas to follow, including establish baselines without AI, track the performance like time-to-completion when using AI compared to not using AI, and evaluate the quality of the work. The Navy reminded users that time savings without quality maintenance isn’t true efficiency.

]]>

Fanelli said the Navy already is measuring the impact of the AI tools in millions of hours saved without losing any effectiveness just in the first few months of widespread use of AI tools.

“These are well-known metrics already used in industry. These same metrics are used, in some cases, to evaluate technology,” he said. “Basically the idea of what are the outcomes for the effort that you are putting into the technology. Now we are just applying these same concepts to AI.”

Coast Guard leans into automation

The Navy also is using gamification and hackathons as part of this effort, creating a collaboration room online to bring people together with similar interests.

Fanelli said the Navy is teaming up with the Defense Innovation Unit on a structured challenge. The Marine Corps, for example, rolled out an innovation challenge asking marines at the edge to tee up highest impact activities that could benefit or are benefitting from AI and send them forward. The Marines also are giving bonuses for the ideas with the most impact.

Fanelli said in the next few months, the DoN will roll out a scorecard to monitor progress of the use cases as a way to help them decide which ones are ready to expand to more users.

What the Navy is doing isn’t necessarily specific to the service and Fanelli said any organization can take a similar approach.

The Navy already is sharing its initial lessons learned and approach with other agencies to demonstrate how they are collaborating and scaling AI tools.

The Coast Guard is taking a similar approach that is based on those industry-accepted metrics, especially time saved.

Brian Campo, the Coast Guard’s chief data and AI officer, said at the Sea Air Space conference that the service set up a transformation office last year and developed an automation plan. He said after a few minor studies, they came up with a metric called “minutes on mission.” Now Campo said the Coast Guard has three related metrics.

“We look at how quickly we can get that capability out. How quickly can we give an automation to somebody? We’re thinking about how can we scale that out to more than one person or office so that’s a second dimension. But the only one that we really care about is minutes on mission,” he said. “How has that automation, or how has that function, or how has that transformation, how has that put that person closer to the mission? How has it created more opportunities for that person to be in the mission? And how has that function created more operational capability, things that we didn’t even think about before?”

Campo added the Coast Guard estimates that between 30% and 50% of any one person’s day is taken up by administrative tasks, and between 50% and 70% of those tasks can be automated.

He said the administrative work is “not the reason why they joined the Coast Guard. It’s not the thing that they’re passionate about. We can take 15 to 20% of their day and put it back into minutes on mission,” he said.

Copyright
© 2026 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.