Confidence scores play a key role in generating chatbot responses but function differently depending on the knowledge source.
This article describes the role that the confidence score plays with Q&A pairs as the knowledge source and intents or triggers in a conversation workflow.
<aside> 💡
If you are using an Azure AI Search and OpenAI configuration, please refer to the articles listed below to learn how confidence scores serve as a retrieval factor when leveraging these Azure services:
</aside>
How to View the RAG Raw Results
In machine learning, a Confidence Score is a number between zero and one that represents the likelihood that the output of a machine learning model is correct and will satisfy a user’s request.
Translating that to Conversational AI, Confidence Score represents the likelihood that your bot is returning the correct response to an utterance. Zammo translates the confidence score into a percentage between zero and one hundred percent.
Answers that do not meet the confidence score threshold will not be returned by the bot. In addition to finding no matches or a weak match to an utterance, answers may have a low confidence score if too many matches are found. Therefore, it may be useful to lower the confidence score threshold rather than adding more question alternatives or new question-answer pairs.
Follow the step-by-step directions HERE to adjust the confidence score threshold for Q&A pairs.
Please see this article to adjust the confidence score threshold for intents and triggers: Intent Confidence Score Threshold.