TL;DR
- Choose a cloud browser when the deliverable depends on an interactive session, such as a validated form, search workflow, or operator handoff.
- Choose a crawling API when the deliverable is accepted content or evidence from public or otherwise authorized URLs.
- JavaScript rendering alone does not decide between the two. Define the required interaction and output first.
- Use Markdown for text ingestion, HTML for markup inspection, and a screenshot or PDF when a visual artifact is the deliverable.
- Where both products fit, split the interactive and content jobs by ownership and connect them through an approved URL and a job record.
- Compare each approach by accepted output, total usage, and the work left for your application.
What is the difference between a cloud browser and a crawling API?
A cloud browser gives your application control of a running browser, while a crawling API accepts a page URL or crawl task and returns content artifacts. With a managed browser, your code decides which page to open, which controls to use, and whether the task succeeded. With a crawling API, your code chooses the URLs, crawl boundaries, requested outputs, and acceptance checks. Nstdata Cloud Browser and Nstdata Crawl expose these two contracts.
Suppose a documentation team needs to index public release notes and verify that an editor can save a draft in a staging CMS. Indexing can start from public seeds and produce validated Markdown. The staging check needs an approved account, a particular interaction, and evidence that the editor accepted the change. These are different jobs even if both use Chromium somewhere under the hood. The scraping versus crawling guide explains how URL discovery differs from extracting a known page.
Which product fits each type of job?
Choose Cloud Browser for stateful interaction or choose Crawl for URL-driven content delivery. Begin with the acceptance condition. If the consumer asks for a receipt number, session checkpoint, or human approval, keep the browser lifecycle explicit. If it asks for documents indexed with source URLs and timestamps, prioritize content scope, extraction quality, and repeatability.
| Decision | Nstdata Cloud Browser | Nstdata Crawl |
|---|---|---|
| Input | Browser configuration, target and authorized actions | Seed or page URL, options and requested formats |
| Control | An interactive page through Live View or a supported client | A content request with documented rendering controls and crawl boundaries |
| Output | Application-defined assertions, screenshots or other task artifacts | Documented page formats and available storage references |
| Unit of work | One owned session plus an explicit business task | A page request or bounded crawl |
| Failure checks | Running state, connection, selected page, UI result, end state | Task status, page failures, completeness, schema and content |
| Scaling concern | Browser concurrency, timeout, app state and proxy usage | URL scope, task scheduling, per-page quality and artifact handling |
A screenshot request can land on either side. A snapshot of a permitted public URL can use a supported Crawl output. A snapshot after a particular approved sequence belongs with the browser performing that sequence. Choose by the state that must be verified, then decide who will keep the artifact. The Playwright Pages guide and its screenshot workflow illustrate the session-side control available to a developer.
How does each Nstdata product support the workflow?
Nstdata Cloud Browser provides managed cloud browsers for browser automation and AI agents, while Nstdata Crawl is a Web Crawling API for LLM-ready data. A browser session provides the running environment for interaction. Crawl can discover eligible pages from a seed and follow permitted links recursively within the configured scope. The acquisition layer can render JavaScript when needed and passes documented artifacts to a downstream application. Your application owns the content rules and quality verdict. Crawl documentation.
- Stateful execution. Cloud Browser attaches approved automation over CDP or lets an operator use Live View. A script still needs a specific page and a task-level assertion.
- Discovery and rendering. Crawl supports site discovery and recursive collection. Limit approved hosts, relevant paths, depth and page count according to the current API schema and the site's permitted scope. Request rendering when essential content is missing from the retrieved markup.
- Format choice. The Crawl documentation lists Markdown, HTML, Raw Data, Links, Screenshot and PDF. Larger results can use storage references. Choose formats that serve the consumer. Extra artifacts need a storage and retention policy.
- Managed routing. Both services support integrated proxy routing without having to supply a separate gateway for the documented built-in configuration. Application access and target permissions still need verification.
How do you connect both products in one AI data pipeline?
Connect the workflows through a permitted URL and a task record, while keeping session credentials inside the interactive worker. A useful architecture gives the browser and crawler different responsibilities and stores their outcomes separately.
Step 1. Identify an authorized public output
In your approved staging CMS, review the workflow with a test account. Define a legitimate handoff artifact, such as the production URL of an already public release note. Use an authorized operator path for actions that change application state. Keep only the validated URL, deployment ID and optional content revision in the handoff payload.
Step 2. Validate the URL at the boundary
Require HTTPS and a known public documentation origin. Normalize according to the site's canonical URL rules. Check the permitted path prefix before queuing. A private dashboard URL, signed download, or URL with an account token requires a separately approved ingestion path. Guard redirect and discovery boundaries in your integration.
Step 3. Submit a small Crawl request
Select a known public page and request Markdown first. For a site task, begin with a few URLs under one allowed documentation path. Use the current Crawl API reference to choose fields and inspect the resulting task status. Save the task ID so that the worker can read it again after an interruption. Check storage references when content is not inline. Only scale discovery after a sample passes review.
Step 4. Apply the downstream acceptance rule
Validate canonical origin, locale, document type, expected heading and enough body content to be useful. Save a version or hash for deduplication and the URL for citations. Keep job status separate from acceptance. A task can finish successfully with output that falls outside your indexing policy.
This standalone Node.js example validates a normalized ingestion record. Map actual Crawl response fields to this application schema after retrieving inline or referenced content. The embedded record is synthetic and uses the reserved Example Domain. Run node accept-document.mjs.
import assert from 'node:assert/strict'; import { createHash } from 'node:crypto'; const record = { canonicalUrl: 'https://example.com/docs/releases/42', markdown: '# Release 42\n\nThe staging release changes the approved report filter.', finished: true, }; const url = new URL(record.canonicalUrl); assert.equal(url.origin, 'https://example.com'); assert(url.pathname.startsWith('/docs/releases/')); assert(!url.search && !url.hash); assert(record.finished && /^# Release \d+\b/m.test(record.markdown)); assert(record.markdown.length >= 40); const contentHash = createHash('sha256').update(record.markdown).digest('hex'); console.log(JSON.stringify({ accepted: true, source: url.href, contentHash }));
Step 5. Reconcile failures by job type
Use the browser session ID when a page interaction failed, and the Crawl task ID when a content fetch or artifact failed. A worker should be able to rerun the read-only acquisition step without replaying an interactive state change. Close the browser after the workflow and verify its end state. See the Cloud Browser session lifecycle.
How do you measure which approach works better?
Measure the output accepted by the consumer and the work required to produce it. In an indexing test, compare documented pages accepted, duplicates, parse failures, missing referenced artifacts, and content freshness. In a browser test, compare accepted task outcomes, wrong-page errors, runtime and time spent waiting for a person. A pilot should include at least one failure case you know how to reproduce.
For text ingestion, compare generated Markdown with a small set of reviewed pages before adding an entire site. Verify that code examples, product headings and meaningful links survive extraction. For a retrieval pipeline, separate acquisition from normalization and indexing, and hold rejected content for review. This keeps poor evidence out of the model without making the browser or scraper responsible for business judgment. See what LLM-ready data means and the Playwright guide to user-visible assertions for the browser side.
What should you ship first?
Ship one bounded workflow with a specific acceptance rule before adding a second execution path. For a public documentation pipeline, begin with URL-to-content acquisition and representative content checks. For interactive QA, start with one authorized browser test and a stable checkpoint. Connect them only when the handoff has a clear contract and can be verified independently.
FAQ
Q: Does a JavaScript-heavy site require a cloud browser session?
Use the required output to decide. A crawling API with documented rendering controls can fit public content retrieval. A managed session fits a workflow that requires ongoing page interaction or handoff.
Q: Which format works best for a retrieval pipeline?
Start with Markdown for text-oriented ingestion, then review representative pages. Choose HTML when markup is part of the evidence and retain visual artifacts only when needed.
Q: Can a crawl result be considered ready to index immediately?
Apply your own checks for source, relevance, completeness and duplicates. Resolve available storage references before measuring quality.
Q: Do the two products share a credential or live browser session?
Use each product's documented API and authentication. Share an authorized URL or job reference at the application layer.
Q: How should I compare costs?
Measure browser resources per accepted interaction and crawl usage per accepted document. Include validation, rejected outputs, and recovery effort in the comparison.





