Node.js Web Scraping With Proxies: Axios and Fetch
TL;DR
A Node.js proxy for outbound scraping is different from JavaScript's Proxy object and from a reverse-proxy server.
Axios accepts an HTTP proxy through its request proxy configuration; use an explicit timeout and validate the returned content.
Node's built-in fetch() is powered by Undici, and per-request proxy routing can use an Undici ProxyAgent as the dispatcher.
Keep one dispatcher or agent for a sticky identity, close it when the job ends, and rotate only at independent task boundaries.
A proxy changes the network route; it does not grant permission, solve CAPTCHAs, or make an invalid scraper correct.
Which βNode.js Proxyβ Do You Mean?
For web scraping, a Node.js proxy is an outbound network intermediary used by an HTTP client such as Axios or fetch. It is not the Proxy metaprogramming object described by MDN, and it is not a reverse proxy that accepts traffic for your application. This article focuses only on outbound requests to public or authorized targets.
Nstdata exposes proxy gateways that can be used by Node HTTP clients. The client library determines how the gateway is attached, how credentials are read, and whether connections are reused. The existing Axios proxy guide covers Axios-specific patterns; this guide compares them directly with current Node fetch behavior.
Node documents fetch as a browser-compatible global in Node.js global fetch documentation. Node's current enterprise networking guidance also describes opt-in environment-proxy support in recent releases, but library-specific configuration remains the most portable choice across supported runtimes.
A practical Node web scraping proxy example needs a supported Node runtime, an HTTP client, an authorized target, and proxy credentials. The examples use ES modules and avoid embedding secrets.
mkdir node-proxy-demo
cd node-proxy-demo
npm init -ynpminstall axios undici
Configure environment variables in your shell or deployment secret store:
Axios can pass a proxy object on each request, which keeps credentials separate from the target URL. The current Axios request configuration documents proxy, auth, timeout, and the ability to disable proxy handling with proxy: false when supplying custom agents.
validateStatus: () => true lets the application inspect every HTTP status in one place instead of turning some responses into transport-like exceptions. This is useful only if you then enforce acceptance yourself.
A 200 challenge page is not an accepted record. Test a stable field, selector, schema, or content marker that belongs to the authorized destination. The JavaScript web scraping guide adds parsing patterns once transport is reliable.
Method 2: Use a proxy with Node fetch
Node fetch can route a request through an Undici ProxyAgent supplied as dispatcher. The official Undici ProxyAgent documentation shows the agent URI, dispatcher option, and explicit close() lifecycle.
Step 1: Build the credential URL safely
import{ProxyAgent}from"undici";const user =encodeURIComponent(process.env.PROXY_USER);const password =encodeURIComponent(process.env.PROXY_PASSWORD);const proxyUrl =`http://${user}:${password}@${process.env.PROXY_HOST}:${process.env.PROXY_PORT}`;const dispatcher =newProxyAgent(proxyUrl);
Encoding credential components prevents @, :, or / from changing the proxy URL structure. Keep the URL scoped to the process and out of errors.
Closing the dispatcher releases pooled sockets when a short-lived job ends. A server process can reuse a dispatcher for its intended identity and close it during graceful shutdown.
Rotate Proxies by Job, Not by Line of Code
Node proxy rotation is an orchestration decision: choose a route for one independent job, retain it for related requests, then close its network resources.
The loop is bounded and does not retry a denial. A production pool should store consecutive failures, cooldown-until time, recent latency, and last accepted timestamp. Retries need exponential backoff plus jitter, and only idempotent requests should be replayed automatically. The batch scraping guide explains why checkpoints and accepted-output accounting matter when the URL set grows.
Nstdata Residential Prime Proxies suit authorized Node jobs that need HTTP/HTTPS gateways plus rotating or sticky session control. The product can centralize route selection while Axios or Undici remains responsible for timeouts, socket lifecycle, and semantic validation. Current product materials describe HTTP, HTTPS, and SOCKS5 protocols and session persistence options. Choose rotation for independent URLs and a sticky session for paginated or login-free journeys that must maintain continuity; proxy access never overrides target policy.
Client compatibility: Use Axios's proxy object or Undici's dispatcher rather than assuming every Node package reads environment variables identically.
Measured acceptance: Track valid records, challenge pages, transport failures, and latency separately so route quality is observable.
Axios vs Fetch Proxy Decision Table
Axios and fetch can both use an authenticated HTTP proxy, but their configuration surfaces and response defaults differ.
Decision
Axios
Node fetch + Undici
Proxy attachment
Request proxy object
ProxyAgent dispatcher
Non-2xx default
Axios rejects by default
fetch resolves; inspect ok or status
Timeout
Axios timeout
AbortSignal.timeout()
Socket lifecycle
Managed by Axios/agents
Reuse and close dispatcher deliberately
Credentials
proxy.auth object
Encoded agent URI or documented token option
Choose Axios if the project already uses its interceptors and error model. Choose native fetch with Undici when reducing wrapper dependencies and explicit dispatcher control matter. Do not mix Axios's built-in proxy field with a custom tunneling agent unless the client is configured with proxy: false; double configuration causes confusing routes.
Troubleshooting Node Proxy Errors
Node proxy failures need separate transport, authentication, timeout, and content diagnostics.
Symptom
Check
ECONNREFUSED
Proxy host, port, listener, firewall
ETIMEDOUT or abort
Connect/read budget, target latency, proxy health
HTTP 407
Proxy credentials or allowlist; do not retry blindly
HTTP 403 or challenge HTML
Authorization and target policy; stop rather than rotate aggressively
Axios ignores agent
Ensure proxy: false when a custom agent owns routing
Process does not exit
Close Undici dispatchers and other open handles
When pages require browser rendering, an HTTP client is the wrong execution model. Review extracting content from JavaScript-heavy sites and move to an authorized browser workflow instead of fabricating DOM content from incomplete HTML.
Conclusion
A dependable Node.js web scraping proxy setup begins by choosing the correct client boundary: an Axios proxy object or an Undici dispatcher. Add secret-safe configuration, timeouts, semantic response checks, bounded rotation, and resource cleanup before increasing throughput. Nstdata can supply managed sessions, while your application retains responsibility for authorization and accepted data.
Node fetch can use an Undici ProxyAgent passed as the request dispatcher. Environment-proxy behavior varies by Node release and opt-in settings, so explicit library configuration is clearer across mixed deployments.
Q: How do I authenticate an Axios proxy?
Set proxy.auth.username and proxy.auth.password in the Axios request configuration. Read both values from a secret store or environment variables and never log the resulting configuration object.
Q: Is JavaScript Proxy related to HTTP proxies?
No, JavaScript Proxy intercepts object operations inside the language runtime. An HTTP proxy is a network intermediary used by Axios, fetch, a browser, or another client.
Q: Should I create a new ProxyAgent for every request?
Reuse one ProxyAgent for requests that share a stable identity, then close it when that job or process ends. Create a new agent when the route or session identity intentionally changes.
Q: Why does Axios work but fetch fails through the same proxy?
Axios and fetch use different proxy configuration surfaces. Confirm that Axios uses proxy, fetch uses an Undici dispatcher, credentials are encoded correctly, and both clients apply the same timeout and acceptance checks.
Q: Can a proxy scrape every website?
No, a proxy only changes the network route. It does not grant access, execute JavaScript in an HTTP-only client, solve CAPTCHAs, or replace compliance with site terms and applicable law.
Lena Zhou
Sep. 15th 2026
110M+ real IPs with 99.9% access success
Blazing-fast average response ~0.5s for high-concurrency tasks
From only $0.1/GB
Get immediate access to premium residential, datacenter, IPv6 and ISP proxy pools.
