This guide will show you how to add a website data source to an existing knowledge base in the Zammo Portal.
Prerequisites
You must have an existing knowledge base set up in the Zammo Portal. You can learn how to do this here.
Version Information
The Zammo web scraper, which scrapes web URLs and then indexes the data using Cognitive Search, is currently on V1.3. This feature is still in development with additional functionality to come, but currently, you can expect:
- The scraper starts from a single URL and follows all links on the same domain and subdomain.
- You can configure the levels and the number of links on each page for the web scraper to scrape. It is important to call out that the larger number of links scraped, the larger the Azure cost that you will incur.
- Users can manually trigger the re-index with the click of 1 button. Re-indexing checks if the page(s) have changed and refreshes the cognitive search index accordingly.
<aside>
📌 Note:
- It does not currently scrape when a link is not an HTML page.
- Depending on the page structure the first 10 links on a page may include headers and tabs.
- It does not currently scrape images, videos, etc.
- It will only scrape publicly available web pages.
</aside>
Adding a Website Data Source
To add a website data source to a knowledge base, perform the following steps.
- From the knowledge bases screen select
Add a new data source: