If you're new to Zammo—and especially to Azure OpenAI, AI Search, or RAG—it's essential to first understand how RAG works.

RAG functions differently from keyword search or predefined Q&A pairs. When a user asks a question, the query is sent to your knowledge base to retrieve relevant information. This data is then processed by your Azure OpenAI model, which interprets the context and generates a response. As a result, even if you ask the same question twice, the answer may vary slightly.

It may be helpful to create a separate Zammo organization (bot project) for testing and exploring the RAG options. The best way to start is by indexing a small sample of test data, ensuring the content sources are reliable and relevant, and fine-tuning the prompts and RAG configurations—rather than indexing an entire repository only to discover issues with accuracy or completeness. Below are the steps to help you get familiar with the process.

Efficient Indexing

1. Start small. Begin by indexing a single document, such as a handbook, or up to five URL sources.
2. For website data sources, set the Indexing Configurations to a maximum depth of 0 and a maximum breadth of 1 to keep the process efficient. These minimal settings reduce indexing time and limit the amount of content to review, making verification more manageable.

Test and Refine

1. Once your RAG bot is set up, use the Test Bot to evaluate its responses.
2. Start by asking questions that you expect users would ask of the content.
   • If the bot doesn’t know the answer, is the source in the knowledge base? Is the question worded clearly?
   • If a response is incorrect, check its source. Is the correct source referenced?
   • If the correct source is missing from the knowledge base, add it.
   • If the correct source exists but the response is still incorrect, adjust the RAG configuration settings—such as the prompt and project description, the number of past messages included, query specificity, ambiguity, or temperature—to refine accuracy.
3. Lastly, explore the available options for refining bot responses by adjusting settings and observing their impact. For example, try turning on Include past messages and Analyze query specificity and ambiguity for a more conversation style experience! Try adding a clever prompt to the bot persona like: Act as a cowboy who likes to speak in riddles.

Summary

This article provides a step-by-step guide to efficiently setting up and refining your Zammo chatbot using RAG. By starting with a small dataset, verifying content accuracy, and systematically testing and adjusting settings, you can ensure reliable and relevant bot responses.