Generative AI in Government: How Agencies Get From Pilot to Production
A couple of years ago, most conversations about generative AI in government started with “Should we?” Now they start with “How fast?”
The numbers back that up. When GAO looked at 11 agencies with AI inventories, reported AI use cases nearly doubled, from 571 in 2023 to 1,110 in 2024. Generative AI use cases grew even faster, from 32 to 282. That’s roughly nine times as many in a single year.
Policy is pushing in the same direction. OMB’s April 2025 memo, M-25-21, tells agencies to accelerate AI adoption. It requires a Chief AI Officer at each agency and asks the large CFO Act agencies to publish an AI strategy within 180 days.
So adoption is happening. The harder question is how to do it well. Let’s break it down.
What is generative AI in the federal government, really?
Generative AI refers to models that create new content (text, summaries, code, images) based on the data they’ve learned from. In a government setting, that usually looks less like a flashy chatbot and more like practical help with paperwork-heavy work:
- Summarizing long case files or reports
- Searching policy documents in plain language
- Drafting first-pass responses to routine inquiries
- Turning messy, unstructured data (like call transcripts) into something an analyst can use
What’s a GenAI platform, and why not just use a chatbot?
A chatbot is one application. A GenAI platform is the foundation underneath many applications: the infrastructure, the models, the connections to your data, and the security controls that keep it all inside your agency’s boundary.
That matters because most agencies don’t want one AI tool. They want a repeatable way to build several, without starting from scratch (or re-approving the security model) every time. A good platform connects to structured and unstructured data, supports techniques like retrieval-augmented generation (RAG) so answers are grounded in your own documents, and can run on-premises or in the cloud.
Where agencies get stuck
In that same GAO review, agencies were candid about the roadblocks. Ten of 12 said existing federal policies, such as data privacy rules, could slow adoption. Others pointed to limited technical resources and budget, and to how hard it is to write usage policies for a technology that keeps changing.
None of those are reasons to wait. They’re reasons to plan.
Five steps from pilot to production
- Start with a problem, not a model. Pick a workflow where people spend hours on repetitive reading, sorting, or summarizing. That’s where GenAI shows value fastest.
- Get your data house in order. GenAI is only as good as what it can access. Know where your data lives, who owns it, and how clean it is.
- Match deployment to data sensitivity. Public information may be fine in a cloud environment. Sensitive or controlled data may need an on-premises or fully air-gapped setup.
- Build governance in from day one. M-25-21 calls for risk management practices on high-impact AI. It’s much easier to design for that up front than to retrofit it later.
- Prove it small, then scale. Run a focused proof of concept, measure time saved or accuracy gained, and use those results to make the case for expanding.
What this looks like in practice
One example of this pilot-to-production approach comes from Government Acquisitions (GAI), which offers a GenAI Accelerated AI Platform built for public sector agencies. In one federal engagement with its partner Certara, GAI describes setting up a secure, air-gapped AI environment. The first phase used large language models to turn unstructured call transcripts into actionable insights. Later phases added RAG for summarization and content generation, cutting manual effort while keeping data secure.
Notice the shape of that rollout: a contained first use case, a secure environment that fits the data, and more capability added once the foundation proved itself. That’s the pattern worth copying.
What about state agencies?
The same playbook works at the state level, usually with tighter budgets and smaller teams. That makes the “start with one painful workflow” step even more important. State agencies often get the most traction from shared use cases, like constituent inquiries, document processing, and policy search, that several departments can benefit from at once.
The bottom line
Generative AI in government has moved past the experiment phase. The agencies getting real value aren’t necessarily the ones with the biggest budgets. They’re the ones that picked a clear problem, built on a secure foundation, and grew from there.