A look at the Web Crawler, AI Search & Data Ingestion

In this article you will find:

An overview of the AI Search integration in Zammo.
Technical details for the Zammo web crawler.
Technical details for the data ingestion.

The main topics:

Architecture
Data ingestion
Crawler
FAQ

Screenshot 2024-05-15 at 11.17.34 AM.png

Screenshot 2024-05-15 at 11.17.46 AM.png

Screenshot 2024-05-15 at 11.17.59 AM.png

Content Extraction, Chunking, and Embedding.jpg

Web Crawler FAQ

Question: We have started an ingestion, but it returns 0 page?

The most common causes for the ingestion to return 0 page are: