Fine Tuning Agent Intents

When your chatbot has multiple AI Agents, our AI Supervisor automatically assigns the most appropriate Agent to handle the user’s input queries. It does so by first determining the user’s intent, then looks for the most suitable Agent to help the user based on Agents’ names and descriptions.

In our article Multi-Agent Chatbot, we covered some best practices on Agent description writing and provided a number of examples. However, as comprehensive as the AI Supervisor tries to be, there will still be scenarios where Agent descriptions do not sufficiently cover all scenarios.

When AI Agents are created or modified, we automatically generate a number of intents along with example user queries to serve as reference for the AI Supervisor. These intents can be reviewed, customized, or deleted through the AI Supervisor Settings interface. The AI Supervisor makes use of these intents directly when evaluating each Agent’s specialty.

To fine-tune Agent intents, go to AI Agents tab on the left navigation menu, then click on AI Supervisor Settings:

Now head to the “Intents” tab:

Here, you will see the intents that have been automatically generated by the AI Supervisor based on its interpretation of the Agent’s function. When a user submits a query, if you have more than one active AI Agents set up, then the AI Supervisor will extract the user’s intent and try to find the closest match across all active Agents.

To add new intents, select the Agent that you want to fine-tune, then click “Add intent”.

Then, specify the intent along with a few example queries that users could input which demonstrates this intent.

Click Create. You should see the intent added under “Fixed Intents”. You may need to scroll down to see the entire list.

You can also edit AI-Generated Intents. Simply open the three-dot-menu next to the intent and select “Edit”. After you save your edits, this intent will be moved to the “Fixed Intents” section.

​

When should I fine-tune Agent intents?

For most use cases involving general Q&A, fine-tuning intents is not necessary. Our own internal tests have shown that AI-generated intents are typically sufficient for good Agent routing.

Allowing bot makers to control intent classification at a granular level empowers them to enhance chatbot performance in situations where many AI Agents are involved, or where the chatbot requires a high degree of precision.

When designing a multi-agent setup, we recommend keeping to a “MECE” approach. “MECE” refers to - Mutually Exclusive, Collectively Exhaustive. In other words, no gaps, no overlaps.

Ideally, you want the situation where:

User-facing Agent A handles:

• intent class 1
• intent class 2
• intent class 3
• etc.

User-facing Agent B handles:

• intent class 10
• intent class 11
• intent class 12
• etc.

and there is minimal overlap in intents among Agents. All your Agents combined should cover all possible intents your users might have.

But since you can’t ever account for all intents in the universe, and given the fact that intents are presented in natural language which can have multiple permutations to express the same meaning, there is no deterministic way to get 100% consistency. But you can get pretty close with an optimized setup.

During your chatbot’s operations, it is important to monitor incoming conversations and identify instances where the wrong Agent was chosen to handle a user’s query. In these situations, we recommend that you make use of our “Debug mode”. In this mode, if you have multiple user-facing Agents set up, you will see “Intent” and “Active Agent” show up alongside every user query that underwent Agent routing.

If the incorrect Agent has been assigned, a quick and effective fix is to add the intent and the corresponding user query to what should have been the correct Agent to handle the request within AI Supervisor Settings -> Intents. This will ensure that if queries of the same type were to come up again, the correct Agent would be chosen. Providing more examples for this intent will further reinforce the ability for the AI Supervisor to classify similar cases in the future.