Server configuration and cron

PHP execution time

DoorFall processes crawler work in controlled batches so a single slow website should not monopolize a request. Your host should nevertheless allow normal PHP scripts enough execution time to complete a crawler batch.

If your hosting uses unusually restrictive PHP execution limits, use smaller crawl batches and rely on cron to process the queue gradually.

PHP memory

Crawler pages are intentionally limited to protect server resources. HTML responses larger than 2 MB are skipped.

A normal PHP memory allocation suitable for a database-backed website is recommended. If the Admin reports memory errors, increase the PHP memory limit through your hosting control panel or ask the host.

cURL and outbound HTTPS

Outbound cURL must be enabled. DoorFall uses it to reach source websites and the DoorFall licensing service.

Firewalls must allow normal outbound ports 80 and 443.

DNS

The server must be able to resolve public domain names. A Resolving timed out error usually points to remote DNS/connectivity trouble or a hosting-network issue rather than the search index itself.

SSL

Keep SSL certificate verification enabled. Do not disable certificate checking to make a broken source crawl. Fix the certificate or use a properly configured source website.

Cron setup

Open Admin → Crawler Health after installation. DoorFall displays the cron information generated for that specific installation.

If your hosting supports PHP CLI, use the CLI cron command shown by DoorFall.

If your hosting only supports URL/web cron, use the protected Web cron URL shown in Admin. The URL contains an installation-specific cron key; treat it as private.

A typical server cron can run frequently while each individual Source controls its own recrawl interval. For example, cron can wake the worker every hour while a source is configured to recrawl Weekly.

The server cron frequency and the Source Automatic recrawl setting are different:

• Server cron decides how often DoorFall wakes up to check for work.
• Automatic recrawl decides when each source becomes due for new crawl work.

File permissions

Use standard web-server ownership and permissions. PHP needs write access only where the application stores uploaded branding and cached favicon files.

If logo/favicon uploads fail, check ownership and write permissions for the relevant upload directory rather than opening permissions on the entire site.

Mod rewrite / clean URLs

Keep the supplied .htaccess file when using Apache-compatible hosting. It contains the application routing behavior needed by public pages.

If you use Nginx or another server that does not read .htaccess, reproduce the equivalent routing rules in that server configuration.

Security

Do not expose database credentials, API secrets, Awin private feed URLs, license keys or cron keys publicly.

Do not point the crawler at localhost, LAN/private IPs or internal services. DoorFall blocks private-network targets by design.

Backups

Back up both the database and application files. The database contains the index, sources, settings, analytics and crawler state; backing up only PHP files is not a complete DoorFall Search backup.