TL;DR
- Proxy controls network identity and location; Crawl retrieves and transforms pages; Proxy Manager centralizes routes, pools, logs, and monitoring.
- The products overlap operationally but solve different problems, so they should not be treated as interchangeable.
- An end-to-end pipeline still needs an application-owned data contract, validation, storage, and policy.
- The combined stack is most useful when repeated collection requires both page processing and controllable network operations.
The full web data stack separates three questions: how traffic reaches a site, how a page becomes an artifact, and how routes are governed in production. This separation follows the same responsibility boundaries described by OpenTelemetry for observability, HTTP semantics for request behavior, and W3C provenance guidance for evidence lineage.
The Three Products
Nstdata Proxy: network access
Proxy products expose HTTP, HTTPS, or SOCKS5 endpoints with product-specific rotation, sessions, and targeting. They do not render JavaScript or validate page content.
Nstdata Crawl: page and site retrieval
Nstdata Crawl accepts URLs, performs static or browser-backed retrieval, supports bounded site discovery, and returns page artifacts. It does not decide whether an extracted business record is correct.
Nstdata Proxy Manager: routing operations
Nstdata Proxy Manager organizes Router endpoints, proxy pools, routing rules, logs, analysis, and monitoring. It is an operational control plane, not a content parser.
