What Is HTTP/2? Multiplexing, Performance, and Its Impact on Crawling

HTTP/2 is the 2015 revision of the HTTP protocol that replaces HTTP/1.1's plain-text, one-request-per-connection model with a binary, multiplexed one — allowing many requests and responses to travel over a single TCP connection at once. For crawling and scraping specifically, it changes both performance and, less obviously, how a request looks to anti-bot detection systems.

What Is HTTP/2?

HTTP/2 is the second major version of the Hypertext Transfer Protocol, standardized in 2015 as the first significant update since HTTP/1.1 in 1997. It preserves the same core semantics as HTTP/1.1 — the same methods, status codes, and general request-response model — but changes how that data actually travels over the wire, moving from a human-readable plain-text format to a binary one.

The headline difference is multiplexing: a single HTTP/2 connection organizes requests and responses into independent streams, identified by a stream ID, that can be sent and received interleaved over that one connection. HTTP/1.1 instead handles one request per connection at a time, which is why browsers historically opened multiple parallel connections — typically six to eight per domain — just to approximate concurrency.

How Multiplexing and Header Compression Work

In HTTP/1.1, a slow response blocks every subsequent request queued behind it on the same connection — a problem known as head-of-line blocking. Browsers worked around this by opening several connections in parallel, which just relocates the problem: those connections now compete with each other for bandwidth and are each still individually vulnerable to blocking. HTTP/2 solves this at the application layer by splitting messages into binary frames, each tagged with a stream identifier, so the receiving side can reconstruct many concurrent messages from a single connection regardless of the order frames actually arrive in.

HTTP/2 also introduces HPACK header compression, which reduces the size of the often-repetitive headers sent with every request, and stream prioritization, which lets a client signal which responses matter more so a server can allocate frame-sending priority accordingly. Server push, a feature that let a server proactively send resources it expected a client would need, was part of the original 2015 specification but has fallen out of favor and is being deprecated in major browsers due to limited real-world benefit relative to its complexity.

Protocol Adoption in 2026

Protocol-version measurements from mid-2026 put HTTP/2 at roughly half of measured web requests, HTTP/1.x at around a quarter, and HTTP/3 — the QUIC-based successor — holding a bit over a fifth, with the three-way split holding roughly steady for several months. The gap between browser-only measurements (where HTTP/1.1 use is much lower) and all-request measurements (where it's meaningfully higher) is largely explained by bot and server-to-server traffic, which still leans more heavily on the older protocol than typical browser traffic does.

What This Means for Crawling and Scraping

For most modern scraping workloads, HTTP/2 is the preferred choice when both the scraping tool and the target server support it, particularly when fetching multiple pages from the same domain in one session. The choice should still account for the target site's own capabilities and the library support available in whatever scraping stack is in use.

HTTP/2 vs. Adjacent Concepts

HTTP/1.1 is the predecessor HTTP/2 improves on while keeping the same core semantics — same methods, same status codes, different wire format. HTTP/3 goes a step further, replacing TCP itself with QUIC to solve head-of-line blocking at the transport layer rather than just the application layer, which matters on lossy or high-latency connections where a single dropped TCP packet can still stall an entire HTTP/2 connection despite its application-layer multiplexing. For scraping specifically, protocol choice is a performance and detection-avoidance question, separate from whether a target page's content is static or dynamic and requires JavaScript rendering — the two considerations are independent and both matter for a well-built collection pipeline.

Limits

HTTP/2's multiplexing advantage shrinks for small sites with few assets, where the connection-overhead savings barely register compared to a simple single-page fetch under HTTP/1.1. It also doesn't fully eliminate head-of-line blocking — a lost packet at the TCP layer can still stall every multiplexed stream sharing that connection, which is precisely the problem HTTP/3's QUIC-based transport was designed to address. And because HTTP/2's specific implementation details are more distinctive than HTTP/1.1's simpler connection pattern, using it carelessly in a scraping tool can make automated traffic easier, not harder, to fingerprint if the rest of the client's behavior doesn't match what a real browser's HTTP/2 stack would produce.

Conclusion

HTTP/2 improves on HTTP/1.1 primarily through multiplexing and header compression, cutting connection overhead for anyone fetching many resources from the same domain — which includes both real browsers loading a page and scrapers collecting multiple pages from one target. For crawling and scraping work specifically, the protocol choice carries a second, less obvious consideration: matching a real browser's HTTP/2 fingerprint matters as much as getting the performance benefit.

For teams that want HTTP/2 performance without manually managing protocol-level fingerprinting, evaluate Nstdata Crawl against your own target sites.

FAQ