Ethical Public Crawling Policy

Purpose and authorization

Checks are performed only to produce an agreed website opportunity report or an internally authorized demonstration. A customer should confirm authority to request a review of the submitted website.

Public scope only

The standard workflow accepts public HTTP or HTTPS pages on the selected registrable domain. It blocks credentials in URLs, private and reserved network addresses, localhost, nonstandard ports, file uploads, and authenticated areas.

Respectful collection

• Review robots.txt directives applicable to the crawler.
• Use a descriptive user agent and reasonable delay between requests.
• Limit depth, page count, redirects, response size, and total runtime.
• Stay on the selected registrable domain unless a separate public competitor URL was explicitly scoped.
• Do not bypass paywalls, CAPTCHAs, access controls, or technical restrictions.

Safety controls

Hostnames are validated before requests and redirect targets are revalidated. Page content is treated as untrusted data and cannot authorize tools, disclose secrets, alter scope, or override operator controls. Failures are logged and labeled rather than hidden.

Data minimization

Collect only evidence needed to support the report. Avoid storing unnecessary personal data, sensitive forms, full-page content, or unrelated assets. Screenshots and excerpts should be limited to the documented purpose and retention schedule.

Opt-out and correction

A verified domain owner can email hello@siteupcycle.com to request suppression of future checks and correction or deletion of eligible stored material. We use domain-suppression controls to record confirmed requests.

Operator: SiteUpcycle.
Email: hello@siteupcycle.com
Mail: 5441 S Macadam Ave #6666, Portland, OR 97239, USA.