---
title: Sitemap Setup Guide
slug: sitemap-setup-guide
docTags: 
createdAt: 2026-03-23T17:27:03.839Z
---

## Step 1 — Select Sitemap Source

Navigate to **Train → Generative AI → Documents → Upload Document**.

Select **Sitemap** as the content source.

![](https://api.archbee.com/api/optimize/FEqjvWZnyymr_PEhXLLlG/zUhOfx8rAfjp_WKpAU8Wg_image.png)

## Step 2 — Enter Sitemap URL

Provide the **Sitemap XML URL** of the website.

Example:

:::BlockQuote
1\. \<https\://help.engati.ai/platform-tutorials/sitemap.xml>
:::

The system will crawl all URLs listed inside the sitemap.

![](https://api.archbee.com/api/optimize/FEqjvWZnyymr_PEhXLLlG/A4ExCaRL2Cgs4JbjI_I43_image.png)

## Step 3 — Configure Processing Options

Select how the URLs should be processed from the sitemap.

Available option: **Include all, Exclude some**

This option crawls all URLs by default and allows you to exclude specific pages using regex rules.

Other options such as **Exclude all, Include some** and **Custom** will be available in future releases.

![](https://api.archbee.com/api/optimize/FEqjvWZnyymr_PEhXLLlG/ZBfOK88dd3vK1lzwHORYx_image.png)

## Step 4 — Configure Regex Rules (Optional)

Regex rules allow you to **exclude specific URLs** from being indexed.

Example sitemap entry:

:::BlockQuote
\<loc>\<https\://help.engati.ai/google-sheets>\</loc>
:::

Example regex pattern:

:::BlockQuote
^https:\\\\/\\\\/help\\\\.engati\\\\.ai\\\\/google-sheets\\\\/?$
:::

You can create multiple rules using **Add Regex Rule to Exclude**.

**Category:&#x20;**&#x53;elect the appropriate category.

**Language:&#x20;**&#x43;hoose the language used on the website.

![](https://api.archbee.com/api/optimize/FEqjvWZnyymr_PEhXLLlG/_ZLVuHHQy2vdGNWCTNAEG_image.png)

## Step 5 - Set up Indexing & Sync

You can enable **Auto Refresh** to automatically re-index the sitemap content at regular intervals.

When **Auto Refresh** is enabled, the system will crawl the sitemap again and update the indexed content based on the configured refresh interval.

### Refresh Interval

Specify how frequently the sitemap should be refreshed.

Example: **7 days** – The system will re-crawl the sitemap and update the indexed pages every 7 days.

This ensures that any **new or updated webpages** in the sitemap are automatically included in the AI knowledge base.

![](https://api.archbee.com/api/optimize/FEqjvWZnyymr_PEhXLLlG/MRCazoV2oryoiUVAWoo4h_image.png)

## Step 5 — Review Setup

Verify the sitemap URL, regex rules, and document details.

Click **Save & Start Indexing** to begin crawling the website

![](https://api.archbee.com/api/optimize/FEqjvWZnyymr_PEhXLLlG/4oykDwBNJet7ZKWarItzi_image.png)


📚 Next Steps
=============

Once the sitemap indexing status shows **Ready to Search**, configure the agent workflow so the AI can retrieve knowledge from the indexed website content.

➡️ **Continue with&#x20;**[3. Path Setup](docId\:G-jippCSr0vJWz4rIchMf)**&#x20;for Agents**