Agentic AI-powered tools are the type of AI that operate independently to complete multi-step goals. These tools are rapidly proliferating the tech industry and mainly to act as personal assistants.
As government agencies are increasingly leveraging AI to modernize public service delivery, it’s important to consider how established agentic AI tools, like making purchases or generating code, can improve benefit delivery — even if the solutions aren’t obvious at first.
In this toolkit, the Nava Labs team shares their findings from developing an agentic Form-Filling Assistant with caseworkers from the Riverside County Children and Families Commission. We outline some guidelines that emerged from the pilot and the design patterns we developed to put them into practice.
See also: Toolkit: How to scope and define human-centered agentic AI tools
This toolkit can help you:
- Understand how to adapt existing agentic AI patterns for public services
- Turn concepts like “human-in-the-loop” into concrete, context-appropriate design decisions
Higher risk and more constraints for caseworkers
Many popular agentic AI tools and workflows are designed for subject experts and software developers, differing significantly from the reality of the caseworkers who help people access benefit programs. Tech-fluent people using agentic AI tools are usually working independently on a task that can be accomplished with some leeway in process and outcome. A caseworker using an agentic AI workflow to help a client apply for benefits has far less flexibility and risk tolerance.
Existing agentic workflows are designed for a person operating the tool on their personal device, for their own benefit, and with the time to pause, review, and visit the agentic AI’s outputs. Caseworkers are operating the tool on behalf of their client, in a live, often time-limited session; any outside interruption could interfere with the AI agent’s workflow. Additionally, caseworkers might not have the same technical knowledge as a software engineer, making it difficult to troubleshoot unexpected behavior. Finally, the accountability is different: when an individual person uses agentic AI, they are responsible for the outcome and live with those consequences; caseworkers, on the other hand, are responsible for what the AI agent does, but it’s the client who has to live with the consequences.
Understanding these differences and limitations is helping guide how Nava Labs develops new agentic AI workflows and tools that can help these people in the best way possible. While working on our form-filler tool, four main guidelines emerged that helped us structure development to mitigate the constraints of traditional agentic AI workflows. These guidelines started as design patterns but evolved as we gathered user feedback.
1. Offer multiple ways to observe the tool
Caseworkers need to know what the agentic AI tool is doing at any moment. In our initial prototypes, we thought being transparent meant we should display all of the agentic AI’s self-narration in the chat. However, testing with caseworkers quickly revealed that created more confusion than confidence. As we iterated, we added a range of components that offer targeted looks into the system’s actions. Learn more about those components below:
Design patterns for observation:
1a. Status indicator
The status indicator shows whether the AI-powered agent is working, stopped, or waiting for the caseworker to take action.
The status indicator helps caseworkers track where they are in a workflow that moves quickly and shifts control back and forth between them and the AI agent. It remains fixed on the interface throughout the session.
1b. Collapsible record of AI agent actions
AI-powered agents autonomously choose how to accomplish a task by using tools it has available, like querying an API or using a skill to navigate an online form. The AI-powered agent generates a list of its actions which are displayed to the user in real time.
It’s important to keep a record of the AI-powered agent’s actions to ensure accuracy while it works and to refer back to if something’s incorrect. Because the AI-powered agent moves quickly, displaying a running log has the potential to clutter and overwhelm the interface. Instead we opted to put the AI-powered agent actions in an auto-collapsed accordion, enabling caseworkers to opt in to follow along.
1c. Data source labels
Before submitting an application, the AI-powered agent provides a list of all the form fields with clear labels describing the data’s source.
Labeling what information was deduced by AI, with its source, is a crucial transparency measure. On our Form-Filling Assistant, the data source labels help caseworkers determine where they should spend their time reviewing. Importantly, the labels also provide an explanation — a known challenge when working with the LLMs — as to why the AI-powered agent inferred that a field should be filled a certain way.
Lesson learned: Simple interface, details on demand
Our development team and the caseworkers need different levels of system transparency. We learned that we needed to keep the interface simple for caseworkers and make the technical details accessible but not prominent. When designing your own AI agent, use feature flags in your development environment if your engineers need to see more information to improve the system.
2. Build in human oversight that fits the context
The AI-powered agent should be easy to override when something goes wrong. But requiring the tool to check in with the caseworker at every step can create more friction. Over-asking for approval can also lead to “rubber stamping,” when a caseworker accepts the AI agent’s suggestions by default out of decision fatigue. We heard directly from caseworkers that they wouldn’t want to use the tool if it feels like it creates more redundant manual input and review.
We developed a framework for how and when “human-in-the-loop” prompts should contextually appear during the form-filling experience in a way that felt logical and predictable.
Design patterns for oversight prompts:
2a. Ask for information
At the start of the session, the AI agent prompts the caseworker to provide information before it begins filling out the form.
At the start of the session, the AI agent generates a list of questions based on what it anticipates needing so it can gather what it needs upfront and minimize interruptions later.
2b. Ask for confirmation
The confirmation prompt appears on an as-needed basis, catching critical entry fields the AI agent didn't anticipate needing upfront.
Throughout the session, the AI agent may pause to ask for confirmation. We developed this to occur when the AI agent hits a specific information gap that it doesn't want to assume its way through (like SSN or income). We try to use this sparingly to minimize interruptions.
2c. Ask for review
After the AI agent fills in every part of the application, the caseworker is asked to review the application before submitting.
At the very end of the session, the AI agent presents a summary of what it filled in for the caseworker to review before final submission.
Lesson learned: Identify, and refine, what needs human input
Start by building a framework that identifies what the AI agent can handle autonomously versus what needs a caseworker’s manual input early on. Embrace that the framework will inevitably evolve as users test the tool with real-life scenarios and in later live client sessions.
3. Make it easy to intervene at any point
Designated human-in-the-loop checkpoints only cover the moments you planned for. In testing, we saw that a caseworker might need to pause the AI agent mid-session for any number of reasons: a client asks a question, the AI agent behaves unexpectedly, or the system runs into an error. Caseworkers must have access to intervention controls at every point in the flow, and are one of the few features that we haven’t changed since the start of the project.
Design patterns for intervention controls:
3a. ‘Take control’ button
The caseworker can pause the AI agent at any point and take over the agent’s web browser manually by clicking a “Take control” button. Once the caseworker hands back control, the AI agent will continue with the caseworker’s changes.
3b. ‘Stop’ button
“Stop” pauses the AI agent’s actions immediately. From there, the caseworker decides what happens next.
Caseworkers can use a stop button to pause the AI agent. Once stopped, the caseworker can give the AI agent new instructions or switch to editing the form themself.
Lesson learned: Maintain human control
Caseworkers expressed relief and comfort letting the AI agent work autonomously when they knew that stopping or taking over was always an option. We measure how often caseworkers press Stop or Take control as metrics that signal trust and accuracy.
4. Craft the AI agent’s language as part of the interface
Agentic interfaces generate their own status messages, error states, and action labels in real time. Unlike a static interface, the message language doesn't stay fixed. If no one is actively writing the messages, technical vocabulary will keep finding its way into the interface.
We heard consistently from caseworkers that technical terms were disorienting. We constantly re-evaluated the content the AI agent generated and made edits to our system prompt to minimize the amount of jargon that made its way into the frontend.
Guidelines for plain language:
4a. Describe the task, not how the tool did it
The AI agent describes its own actions. Without guidance, it defaults to the vocabulary it was trained on, not the vocabulary a caseworker would use.
❌ “Executing the Fill action”
✅ “Filling in the address field”
4b. Use action-oriented microcopy
Write explicit handoff messages that signal when control shifts from the AI agent to the caseworker.
❌ “Done”
✅ “Ready for your review”
4c. Write in the third person
Avoid first-person framing. It implies intent and awareness the AI agent doesn't have and blurs accountability for the caseworker who is responsible for the submission.
❌ “I am missing information.”
✅ “More information needed”
4d. Be specific in the system prompt instructions
Writing instructions into the system prompt is the most direct way to shape what the AI agent says.
❌ “Use plain language”
✅ “Never say ‘execute.’ Say ‘fill in’ or ‘complete’ instead.”
Lesson learned: Use plain language and iterate
Language for an AI agent is constantly shifting and unpredictable. Some models understand the system prompt more effectively than others. Since an AI agent can be nondeterministic, plan to revisit, rewrite, and retest the system prompt regularly.
Conclusion
The four experience principles outlined in this toolkit aim to keep caseworkers informed, in control, and confident while using the Form-Filling Assistant with a client. However, the design patterns are still nascent and prime for redefinition in the government context.
The challenges of designing an agentic workflow are problems that civic technologists have been working on for decades: guiding people through unfamiliar processes, communicating what's in their control, explaining technical terminology in understandable terms.
Right now, commercial AI products are defining agentic patterns for their own set of users and end goals. The civic tech community has the experience to adapt the existing standards to higher-stakes contexts and keep people at the center of the design process. This toolkit is a starting point as our industry determines what effective agentic AI-powered tools look like in public service delivery.
—
If you’re a designer, technologist, or program team working on AI-powered tools for caseworkers or benefits delivery, we’d love to hear what you’re finding! Reach out at labs@navapbc.com.
Written by
Senior Designer/Researcher
