Website sync
Every option in the website import, and how syncing works.
Website sync imports pages from a public website and keeps them fresh. Each page becomes a document in the site's own folder. For a walkthrough, see Import your website.
The four steps
Connect. Paste the Main website link: the top of your help center or docs. That page and the pages under it are synced. A bare domain works.
Pages. Name the source in Title. Then either:
- Use the site's llms.txt, when the site has one. It's a list of pages the site owner picked for AI, and it's on by default when found.
- Or check pages in Review pages to sync. Filter pages narrows the list.
Advanced settings on this step:
| Setting | What it does |
|---|---|
| Additional URLs | Adds pages that aren't linked from the pages above. |
| URLs to exclude | Pages you don't want synced. |
| Follow links (Basic and up) | Also follows links on the pages it finds, two levels deep, to catch pages it would otherwise miss. Off: only the pages listed are synced. |
| Ignore canonical URLs | Treats every URL as its own page. Leave it off unless pages you expect are missing. |
Target. Channel limits where these pages can be used (default: all channels). Audience control limits which audiences get answers from them (default: everyone).
Review. Check the summary, then Start sync.
After the sync starts
- The site appears in Content as its own folder, with a progress line while it crawls.
- The folder shows how many pages are live, like "42 / 45 Live".
- Pages are read-only. Change them on your website.
- Discarded pages are ones the crawler judged not useful as answers. Open one to see why.
How often it syncs
Every website re-syncs automatically once a week. There's no manual sync button. A failed row has Retry.
Limits
Your plan sets:
- how many websites you can sync,
- how many pages a single import can take,
- how many website pages you can import each month.
See Plan limits. If you downgrade below your number of websites, the extra websites keep answering with the pages they have, but stop re-syncing.
Remove a website
Delete the website's row in Content. Its pages are removed with it.