RAG (Custom Data Indexing)

overview rag custom script training allows the ai agent to fetch data dynamically from external sources and index it as knowledge instead of uploading documents or urls, a custom script retrieves data, converts it into structured json, and returns it to the system for indexing this approach is useful when data changes frequently or comes from external systems supported data sources using custom scripts, data can be fetched from various external sources and indexed for rag search below are sample examples demonstrating how data can be retrieved, converted into structured json , and indexed as knowledge for the ai agent these scripts transform external data into a format that the system can process for retrieval and response generation you can adapt similar scripts to integrate data from your own systems rag search using custom scripts this method allows the ai to index external data sources such as google sheets external apis custom backend services the script must return structured json that includes example structure { "keys to index" \["page content","metadata"],"page content" "content for ai retrieval","metadata" { "source" "external data" } } { "keys to index" \["page content","metadata"],"page content" "content for ai retrieval","metadata" { "source" "external data" } } the keys to index field tells the system which data should be indexed for retrieval supported sources google sheets – fetch spreadsheet data and convert it into json format for indexing sitemap urls – crawl webpages listed in a sitemap and extract content programmatically api endpoints – retrieve data from backend services or external apis follow the steps below to complete the setup 👇 watch the configuration video 📚 next steps once the custom script indexing status shows ready to search, configure the agent workflow so the ai can retrieve knowledge from the indexed website content ➡️ continue with docid\ m0f5agwvekrbuek6jq c4