Why Accuracy Matters: Training AI Voice Agents for Municipal Customer Service

We were recently asked to create an AI voice agent to answer phones for a local town. We have never worked with any level of government, so this presented some new challenges. 

The town was facing two issues. The first issue was when citizens called an administrative person, they had to stop what they were doing, look up what the caller asked about, most of which can be found on the town’s website, and respond to the caller. Most of these calls take around five minutes. That doesn’t seem like a lot until you get twenty to thirty a day, and now you are looking at over two hours of productivity lost daily. The second issue they faced was that the town had no good way of getting messages routed to the proper administrative person, so messages were getting lost. So here are some factors we had to take into consideration to help save the town.

Training AI for Accuracy—Setting the Foundation

Creating accurate responses in an AI voice agent starts with setting clear and precise instructions for the agent to follow. These instructions, often called “prompts,” are the guidelines that help the AI know precisely what to say and when to say it. The more detailed and specific these prompts are, the better the AI can serve callers without making mistakes.

Creating Detailed Prompts

When designing prompts, it’s essential to focus on clarity and simplicity. For instance, if the agent needs to provide information about court fines, the prompts should specify exact phrases to use, such as “miles per hour” rather than “MPH,” to prevent misunderstandings. Setting these details in advance makes it easier for the AI to avoid confusing callers with unclear or overly technical terms.

Using a Defined Knowledge Base

A knowledge base is the set of facts and information that the AI can draw from. By clearly defining what’s in this knowledge base—such as current fines, operating hours, and court requirements—we limit the information the AI can access, reducing the risk of generating facts or making assumptions. For example, if a caller asks about seasonal fees that aren’t in the knowledge base, the AI can be directed to refer them to the municipal website instead of trying to guess the answer.

Setting Response Rules and Boundaries

Response rules help the AI stay on topic. For instance, it can be instructed to respond with, “I’m sorry, I don’t have that information. You can visit the municipal website for more details,” if it’s asked something outside of its knowledge base. These rules prevent the AI from making assumptions or inventing answers and guide it to stay within its knowledge limits, especially for questions related to laws or fines that must be communicated accurately.

Strategies for Improving AI Accuracy in Municipal Settings

To make sure the AI is providing accurate information in municipal settings, certain strategies can be used to address common areas of confusion and keep responses clear and consistent.

Phonetic Guidance for Key Terms

Some terms can be tricky for AI to pronounce correctly. By giving phonetic guidance for specific terms—like saying “Municipality dash State dot gov” instead of combining it into one phrase and skipping the dash and dot—we can ensure the AI delivers these details in a way that is easy for the caller to understand. This is especially helpful for terms like email addresses and website names, where a slight change can lead to misunderstandings.

Distinguishing Frequently Confused Terms

In municipal contexts, some terms can easily be confused, like “daily park usage fees” versus “fines for not paying park fees.” To help the AI avoid this mix-up, we can set prompts to make it clear that the daily usage fee is separate from any potential violation fine. By specifying these distinctions, the AI knows when to provide fee information and when to inform the caller about a fine if they specifically ask.

Testing Responses and Refining Prompts

Regularly testing the AI’s responses ensures it’s operating correctly and consistently. This can involve listening to sample calls or using role-play scenarios to check if it’s accurately following instructions. Testing can reveal areas where prompts need refinement, like adding pauses between phrases or using simpler language. Small changes in wording can improve the AI’s clarity and make its responses more natural and effective.

Real-Time Updates for Current Information

Sometimes, information such as operating hours may change unexpectedly due to holidays or emergency closures. Allowing the AI to retrieve real-time information, such as current court hours, helps keep responses up to date. This feature is particularly useful for municipalities that may have seasonal changes in operating times or closures due to unforeseen events.

Benefits of Accuracy in AI Voice Agents for Municipal Services

Accurate responses from AI voice agents bring a variety of benefits to municipal services, enhancing the experience for both residents and city staff.

Improving Resident Experience

Accuracy improves the resident’s experience by providing clear, reliable answers to their questions. When an AI voice agent responds accurately, residents can trust the information and avoid the need to follow up or seek clarification. For example, by offering correct court hours and fine amounts, residents get the information they need quickly, which makes the interaction smooth and straightforward.

Increasing Compliance with Local Regulations

Accurate information also helps residents comply with local regulations more easily. When the AI provides clear guidance on processes like filing a plea or court appearance requirements for juveniles, residents are more likely to follow the rules correctly, avoiding unnecessary penalties or misunderstandings. This clarity helps ensure that residents feel well-informed about local laws and regulations.

Reducing Workload for Municipal Staff

An accurate AI agent can answer many common questions without the need for human assistance, allowing municipal staff to focus on more complex tasks. For instance, if the AI can handle straightforward inquiries about fines and contact details, staff have more time to work on pressing cases or other in-depth tasks. In this way, the AI serves as an effective first line of support, handling frequent questions and reducing call volume for staff.

Challenges and Future Improvements

While AI voice agents have become valuable tools for municipal services, there are still challenges and areas where they can be improved to serve residents better.

Addressing Current Limitations

One challenge is that AI voice agents may struggle with complex, multi-part questions. For example, a caller asking about both court fees and park regulations in one question may require the AI to separate these topics and answer them individually. This limitation means that agents need frequent testing and prompt updates to handle these situations smoothly.

Another challenge relates to data capacity. AI voice agents can only process a limited amount of data, and training them on all of a town’s laws and regulations would require a custom large language model (LLM), which is a significant investment. To manage costs, the agent is typically trained on the most common call scenarios, covering frequently asked questions like court hours, fines, and basic compliance requirements. For less common or highly detailed inquiries, the agent can take a message and forward it to a human representative for follow-up. This approach allows municipalities to benefit from AI while avoiding the higher costs of a town-specific LLM.

Looking Toward Future Enhancements

To improve accuracy further, municipalities can explore new tools and features for their AI voice agents. For instance, adding advanced context recognition would help the AI understand when a caller asks about two related topics and respond accordingly. Municipalities could also consider integrating direct update capabilities, allowing the AI to adjust to seasonal fee changes or policy shifts immediately without manual updates. Another improvement could involve adding confirmation checks for high-stakes information, like court appearance dates or plea options, to ensure the caller clearly understands these details.

Conclusion

So, focusing on accuracy in the AI voice agents makes a big difference for residents looking for clear, reliable information on city services. With thoughtful training and clear guidelines, these agents can handle the questions that come up most often—from court fines to hours of operation—while ensuring callers get the details they need without confusion. Even though they may not cover every possible question, focusing on the most common ones means that the agents can make a real impact, helping people find answers quickly and lightening the load for city staff.

Moving forward, there are plenty of ways to keep improving, like adding new features for complex questions or making it easier for the agent to keep up with seasonal changes. As these AI voice agents grow and adapt, they’ll become an even more helpful part of city services, making things a little easier for everyone involved.