DocumentationDoorFall SearchWebsite Sources & Crawling

Manage a source

Explain Source Configuration, recrawling, exclusions, favicon refresh and crawl actions.

Name

Changes the Admin/display name of the source.

Category

Changes the optional category associated with results from this source.

Priority weight

Adjusts this source's ranking influence. Start at 1.00 and make small changes.

Maximum indexed pages

Caps the number of indexed pages from this source. This prevents a very large website from consuming the entire index.

Automatic recrawl

Controls how often DoorFall automatically schedules this source when cron is running.

Choose Manual while testing. Use Weekly for content that changes slowly. Shorter intervals are useful only when the source genuinely changes frequently.

Pausing a source prevents crawler execution for it but preserves queued URLs.

Exclude URL patterns

Enter patterns for sections you do not want indexed, such as account, cart, tag or internal search areas.

Examples:

/login

/cart

/search

?reply=

Use one pattern per line. Matching URLs are blocked from crawling for this source.

Website favicon

Refresh favicon fetches and caches the site's current icon locally. It does not require a full recrawl.

Crawl actions

Use the source crawl actions to discover or process pages. Clearing/rebuilding an index should be used carefully because it affects existing search results.

Last updated August 21, 2026