APEX FLOW WEB SCRAPER / ACCESS CONTROL

Own the route. See the challenge. Keep control.

Our access plan combines first-party network operations with clear authorization boundaries. Current features and planned infrastructure are labeled separately.

Available in the current engine

Operators can route public HTTP and HTTPS requests through one HTTP proxy they control using RAINBO_SCRAPE_PROXY_URL. The engine checks destination addresses before forwarding and reports only whether a proxy is configured. It does not provide a managed pool, geographic routing, or an Apex proxy fleet today.

Challenge pages are detected and returned as an access_challenge failure. The current release does not solve CAPTCHAs or pause and resume through an isolated signed-in profile.

Apex Access Fabric — planned

The proposed first-party service starts with Apex-operated regional egress nodes and a controlled gateway. Each job would have an explicit route policy, domain scope, pinned session, spend cap, rate limit, route-health record, and usage receipt. Tenant credentials and session state would be isolated and revocable.

Apex would disclose where egress runs and which infrastructure providers supply hosting and bandwidth. Residential routes are not part of the initial plan; any future residential access would require explicit opt-in, revocation, and security review.

Challenge handling — owner stays in charge

When the source presents a CAPTCHA or access challenge, the engine should stop automatic retries, classify the event, preserve the job state, and show a clear next step. Depending on the source’s rules, that step may be its official API, a permission request, an authorized account owner completing a challenge in an isolated profile, or skipping the source.

Any future owner-assisted continuation must be explicit, auditable, time-limited, and revocable. It must not automate CAPTCHA answers, disguise automation as a person, rotate routes to evade a block, or mark an unverified challenge as passed.

What we learned from the four products

Firecrawl documents basic-to-enhanced proxy escalation and a retry after selected HTTP errors. Apify combines managed proxy groups and persistent sessions, and markets a separate Unblocker product for anti-bot and anti-CAPTCHA handling. Playwright supplies per-browser or per-context proxy settings and isolated contexts, while leaving the network service to the operator. Crawl4AI documents structural block detection, configured proxy retries, and a fallback hook that can call an external fetch service. Its marketplace also lists separate CAPTCHA-solver integrations. These are vendor product capabilities and claims; they do not demonstrate universal challenge success.

Our target advantage is one inspectable control plane: source policy, route provenance, stable sessions, challenge pause, owner action, cost limits, and an audit receipt. These are roadmap goals until implementation and acceptance gates are complete.

← Current capabilities and limits