How to Scrape Job Boards with OpenAI's Agent Tools
TL;DR
OpenAI's consumer "Operator" agent shut down on August 31, 2025, and its successor, ChatGPT agent, was itself retired in favor of ChatGPT Work in July 2026 โ neither is a scraping tool, and neither exposes a public API you can call from your own code.
The actual buildable path for "scrape job boards with an OpenAI agent" is the computer_use_preview tool on the computer-use-preview model, called through the Responses API โ it drives a real browser via screenshots and click/type/scroll actions rather than reading page source.
Before reaching for browser automation at all, check whether the target board runs on an applicant tracking system (Greenhouse, Lever, Ashby) with a free, public, unauthenticated JSON job feed โ it returns structured data directly and skips the agent loop entirely.
A computer-use agent is the right tool only when a board has no public feed and no accessible JobPosting structured data in the page's HTML, since it's slower, more expensive per page, and more brittle than a direct HTTP request.
Scraping publicly reachable job listings has held up under the Computer Fraud and Abuse Act in U.S. courts, but a board's own Terms of Service can still create separate breach-of-contract exposure, and circumventing bot-detection or rate limits raises distinct legal risk under anti-circumvention law.
Route the agent's outbound browser traffic through a rotating residential proxy pool to avoid datacenter-IP blocks, and normalize every board's output into one shared schema (title, location, URL, posted date, description) before storing it.
Introduction
"OpenAI Operator" no longer refers to a running product. OpenAI shipped Operator as a research-preview browser agent on January 23, 2025, restricted at launch to ChatGPT Pro subscribers, and shut it down on August 31, 2025 after folding its capabilities into a broader "ChatGPT agent" mode. ChatGPT agent itself didn't last either: OpenAI's help center now states plainly that "ChatGPT agent is no longer available. Use ChatGPT Work for longer, multi-step tasks and finished deliverables," per , which launched in July 2026 as the current consumer agent surface.
None of those three products โ Operator, ChatGPT agent, or ChatGPT Work โ ship a public API. They're chat-interface features you drive by typing a request, not something you can wire into a scheduled job that pulls job postings into a database every morning. If you want to build a repeatable pipeline, the relevant piece of OpenAI's stack is the computer use tool in the Responses API โ a developer-facing capability that plays the same role Operator did (looking at a screenshot, deciding where to click or type, and repeating), but as an API you call from your own code. This tutorial builds a job board scraper on top of that tool, and is explicit throughout about when a full browser agent is the wrong tool for the job.
Why scrape job boards at all
Recruiting and market-research teams pull structured job data for a handful of recurring reasons: tracking a competitor's hiring velocity by department, benchmarking compensation and title inflation across a sector, feeding a talent-sourcing tool with fresh requisitions, or building a vertical job-search product that aggregates listings a general search engine doesn't surface well. All of these need the same three fields at minimum โ title, location, and a stable link back to the source โ plus whatever else the use case calls for (posted date, seniority, remote eligibility, salary band when disclosed).
An AI browser agent earns its cost specifically on boards that render listings behind heavy client-side JavaScript, gate pagination behind an infinite-scroll interaction, or use CSS class names that rotate on every deploy, since the agent reasons over what it sees on screen rather than a fixed selector. It's the wrong tool for a board that already answers with structured JSON on request โ that case is covered in the next section, and it's both faster and cheaper than driving a browser at all.
Choosing the right extraction approach
Before writing an agent loop, spend five minutes checking whether the target board makes this unnecessary. Most careers pages sit on top of one of a handful of applicant tracking systems (ATS), and several of the largest ones โ Greenhouse, Lever, Ashby โ expose a free, unauthenticated JSON endpoint that returns every open listing for a given company. Greenhouse's public Job Board API, for example, serves GET https://boards-api.greenhouse.io/v1/boards/{board_token}/jobs with no API key at all, returning id, title, location, absolute_url, and updated_at for every posting, with an optional content=true parameter that adds the full job description. If a company's careers page is Greenhouse-hosted, that endpoint is both more reliable and dramatically cheaper than any agent.
A second fallback sits one layer down: many job posting pages โ including ones that don't run a known ATS โ embed a schema.org/JobPosting block as JSON-LD directly in the page's HTML, specifically because job boards want their listings picked up correctly by Google's job search feature. That JSON-LD is machine-readable without any browser rendering; a plain HTTP GET and an HTML parser recover it. Reach for the computer-use agent only after confirming neither a public ATS feed nor embedded structured data exists โ that's the scenario where the page is genuinely interactive (client-rendered results, scroll-triggered pagination, or a search form that has to be filled in before results appear) and a script that just requests the URL gets nothing useful back.
Take a Quick Look
When a board has no public feed but also doesn't need full interactive automation, Nstdata Crawl can render the page's JavaScript and hand back clean Markdown or JSON from a single API call instead of scripting a browser agent.
An OpenAI account with access to the Responses API and the computer-use-preview model โ this is a separate developer capability from ChatGPT's consumer plans and is billed per token ($3.00 per 1M input tokens, $12.00 per 1M output tokens as of this writing) plus a per-tool-call fee; verify current pricing on OpenAI's own pricing page before budgeting a production run.
Python 3.9+ with requests installed, or an equivalent HTTP client in your language of choice.
A way to render and screenshot a browser page and translate the model's actions back into real input events โ OpenAI's guide points to Playwright as the reference implementation; this tutorial's code samples assume a Playwright-driven browser.
A rotating proxy pool for any run that touches more than a handful of pages, since a single scraping IP making repeated automated requests is exactly the pattern job-board bot detection is built to catch.
Five minutes with the target board's Terms of Service and robots.txt โ see Observations and limits below before pointing this at a specific site.
Step 1: Confirm there's no public feed to use instead
Start by checking the target company's careers page for a known ATS pattern. If the careers page URL contains greenhouse.io, lever.co, or redirects through one of those domains, try the board's public jobs endpoint directly:
If that returns a jobs array, you're done with extraction โ skip to Step 4 and normalize that response instead of building an agent. The board_token is usually the company's slug as it appears in the careers URL itself. If the endpoint 404s, the company either isn't on Greenhouse or uses a different token than its public brand name, and it's worth a quick manual check of the page's outgoing network requests (via browser devtools) for a JSON API call before concluding you need an agent at all.
Step 2: Set up the computer-use agent loop
If no feed exists, initialize a Responses API call with the computer_use_preview tool. The tool needs a fixed display_width and display_height matching whatever viewport your Playwright browser actually renders, and an environment of "browser":
from openai import OpenAI
client = OpenAI()# reads OPENAI_API_KEY from the environmentresponse = client.responses.create( model="computer-use-preview", tools=[{"type":"computer_use_preview","display_width":1024,"display_height":768,"environment":"browser",}],input=[{"role":"user","content":[{"type":"input_text","text":("Open the careers page and list every job title, ""location, and posting URL visible without scrolling."),}],}], truncation="auto",)
The model doesn't execute anything itself โ it returns a computer_call describing one action (a click at a coordinate, a type with literal text, a scroll, and so on). Your code is responsible for actually performing that action in a real browser via Playwright, capturing a fresh screenshot, and sending it back as a computer_call_output so the model can decide the next step:
call = response.output[0]# the computer_call objectaction = call.action # e.g. {"type": "click", "x": 512, "y": 384}run_action_in_playwright(action)# execute the action, then capture a fresh screenshotnext_response = client.responses.create( model="computer-use-preview", previous_response_id=response.id, tools=[{"type":"computer_use_preview","display_width":1024,"display_height":768,"environment":"browser",}],input=[{"call_id": call.call_id,"type":"computer_call_output","output":{"type":"input_image","image_url":f"data:image/png;base64,{screenshot_base64}",},}], truncation="auto",)
This exchange repeats โ act, screenshot, send, read the next action โ until a response comes back without a computer_call, which signals the model believes the task is finished. This request/response shape is illustrative of the documented schema rather than a run captured against a live OpenAI account in this environment; treat variable names and exact field ordering as subject to the current API reference before shipping it, since computer-use-preview is still a preview-labeled model.
Step 3: Paginate and maintain state across the run
Job boards paginate in three common ways, and each needs a different loop condition: numbered pages with a "next" control (click it, screenshot, repeat, stop when the control disappears), infinite scroll (send a scroll action, screenshot, and stop once two consecutive screenshots produce no new listings), or a "load more" button (click, wait for the new DOM to render, screenshot). Because the agent only sees pixels, track de-duplication yourself outside the model: hash each extracted listing's title-plus-URL pair and skip anything you've already recorded, since an agent re-reading a partially-scrolled page will otherwise re-report postings it already listed.
Cap every run with a hard page limit and a wall-clock timeout. A computer-use loop has no built-in sense of "this site has 40 pages and that's too many" โ that boundary is your code's responsibility, not the model's.
Step 4: Normalize into a stable output schema
Whether the data came from an ATS JSON feed, a JSON-LD block, or an agent transcript, land it in the same shape before storing it:
{"id":"8077887","title":"Data Platform Engineer","location":"Remote - US","url":"https://boards.example.com/jobs/8077887","updated_at":"2026-08-06T12:10:17-04:00","summary":"Build and maintain data pipelines...","source":"greenhouse-api",# or "jsonld" / "computer-use-agent"}
Here's that normalization running against a live JSON endpoint returning this exact field layout โ using a local fixture standing in for a real ATS host, since this environment's outbound network access doesn't reach arbitrary external domains directly (the schema itself was confirmed live against Greenhouse's public API):
This ran successfully against the local fixture with two sample listings, confirming the parsing logic against Greenhouse's real field names (id, title, location.name, absolute_url, updated_at, content) before you point it at a live board.
Step 5: Route requests through a rotating proxy pool
A single IP issuing dozens of automated requests to the same board in a short window is the exact signature bot-detection systems are tuned to flag, whether those requests come from a plain requests.get() loop or from the browser a computer-use agent is driving. For an ATS-feed approach, this means routing your HTTP client through a rotating pool; for a computer-use agent, it means launching Playwright with a proxy configured on the browser context itself so every page load โ not just the API calls โ comes from a residential IP that changes between sessions.
Nstdata Residential Prime Proxies fit that role directly: they route traffic through real residential IPs across a large country pool, which reads to a job board's risk-control system as ordinary browsing traffic rather than concentrated datacenter requests. For teams whose real bottleneck is the extraction step itself rather than proxy IPs โ pages that need JavaScript rendering, retries, and clean structured output without maintaining a Playwright/computer-use loop at all โ Nstdata Crawl is worth evaluating as a replacement for Steps 2 and 3 entirely on boards that don't strictly require interactive automation. Crawl:
Renders JavaScript and returns clean output โ a single API call gets back Markdown, cleaned HTML, or a screenshot, without you standing up a browser process yourself.
Runs proxy-backed by default โ every fetch goes through Nstdata's own proxy infrastructure with browser-fingerprint handling, covering the IP-diversity problem this section is about.
Bills per successful fetch, not per attempt โ a request that gets a real response (including a 404 or 403) is billable once; only a system-side failure to retrieve anything goes unbilled, which makes cost predictable across a batch of career pages.
Crawl doesn't currently do natural-language field extraction the way some competitors do โ you still write the Step 4 normalization yourself against its returned Markdown or HTML โ but it removes the browser-orchestration burden for any board where a computer-use agent would otherwise be overkill. The Crawl API reference covers the exact request and response shape for both single-page and site-level crawls, and Crawl's current subscription pricing bills per successful fetch rather than per attempt, which is worth checking against your expected page volume before committing to it over a self-hosted browser loop. Nstdata's Crawl launch announcement covers the fuller feature set, including site-level crawling and multi-format output, if this specific use case grows beyond single career pages.
Observations and limits
A computer-use agent is slow and expensive relative to a direct request: every action costs a model round trip plus a full-resolution screenshot upload, so a 40-listing page with several scroll actions can run to dozens of API calls before you have all its data. Budget accordingly, and prefer the ATS-feed or JSON-LD path whenever either is available โ reserve the agent for boards that genuinely require interactive navigation.
Legally, the ground here is more settled than it might feel. In hiQ Labs v. LinkedIn, the Ninth Circuit held โ twice, on appeal and on remand โ that scraping data a site has made publicly accessible does not violate the Computer Fraud and Abuse Act, because the CFAA targets unauthorized access to non-public systems, not automated collection of what anyone can already see in a browser. That does not make scraping risk-free: a board's own Terms of Service can independently prohibit automated collection, and violating that agreement creates separate breach-of-contract exposure even where no CFAA claim exists. A newer and arguably sharper risk sits in anti-circumvention law โ Reddit's 2025 suit against Perplexity centers on DMCA ยง 1201 claims over circumventing rate limits, CAPTCHAs, and bot-detection systems specifically, which is a different legal theory than "was the data public." Read a target board's ToS before running any scraper against it, and treat "the site put up a CAPTCHA" as a signal to stop rather than a puzzle to solve.
Job listings can also carry personal data โ a named recruiter's direct email or phone number embedded in a posting, for instance โ which triggers ordinary data-protection obligations (GDPR among them) regardless of whether the page itself is public. Storing hiring-manager contact details at scale without a legal basis for that specific use is a materially different activity than storing the job title and location, and this workflow should not casually cross that line by capturing more of a posting's content than a use case actually needs.
Markup and page structure on any given board will change without notice, and an agent-based approach tolerates that better than a brittle CSS-selector script โ but both approaches degrade if a board switches ATS providers or redesigns its careers page, so budget for periodic re-verification rather than a "set it and forget it" pipeline.
Conclusion
"Scraping with OpenAI Operator" describes a product that no longer exists; the real, buildable equivalent today is the computer_use_preview tool on OpenAI's computer-use-preview model, called through the Responses API and paired with your own browser automation. Reach for it only after ruling out a public ATS feed or embedded JobPosting structured data, since both are faster, cheaper, and more stable than driving a browser. Whichever extraction method you land on, normalize output into one schema, cap pagination explicitly, route traffic through a residential proxy pool or a rendering API like Nstdata Crawl to avoid IP-based blocks, and read the target board's Terms of Service before pointing anything automated at it.
FAQ
Q: Is OpenAI Operator still available for scraping job boards?
No. Operator shut down on August 31, 2025. Its successor, ChatGPT agent, was also retired โ OpenAI's help center now directs users to ChatGPT Work instead. Neither Operator, ChatGPT agent, nor ChatGPT Work exposes a public API; the buildable equivalent for a custom pipeline is the computer_use_preview tool on the computer-use-preview model in the Responses API.
Q: Do I need a computer-use agent for every job board?
No. Check first whether the board runs on an ATS with a public JSON feed (Greenhouse, Lever, and Ashby all offer one) or embeds schema.org/JobPosting structured data directly in the page HTML. Both are faster, cheaper, and more stable than driving a browser, and a computer-use agent is worth the added cost and latency only when neither exists.
Q: Is it legal to scrape publicly visible job listings?
U.S. courts have held, in hiQ Labs v. LinkedIn, that scraping data a site makes publicly accessible does not by itself violate the Computer Fraud and Abuse Act. That doesn't clear every legal risk: a board's Terms of Service can separately prohibit automated collection as a matter of contract, and circumventing rate limits or bot detection raises distinct anti-circumvention exposure. Read the specific board's ToS before scraping it, and treat that as a per-site decision rather than a blanket legal fact.
Q: What happens if a job board changes its page layout?
A computer-use agent tolerates layout and markup changes better than a fixed CSS-selector script, since it reasons over what's visually on screen rather than a specific DOM path. It doesn't tolerate an ATS migration or a full site redesign gracefully โ budget periodic re-verification of your extraction logic regardless of which method you use.
Q: How many pages or listings can this handle at once?
Set an explicit page-count and wall-clock timeout in your own code; neither the ATS-feed approach nor the computer-use loop enforces a sane stopping point on its own. For an agent loop specifically, also de-duplicate extracted listings by title-and-URL, since re-scrolling a partially-read page can cause the model to re-report entries it already found.
Q: Does this workflow require a proxy to function at all?
No โ the code in this tutorial runs without one. A proxy pool becomes necessary once you're making repeated automated requests to the same board in a short window, which is the pattern most job-board bot detection is tuned to flag regardless of whether those requests come from a plain HTTP client or a browser a computer-use agent is driving.
Q: Can this same approach extract salary and contact information from listings?
Technically, yes, if that data appears on the page. Treat it carefully: a posting's recruiter email or phone number is personal data subject to ordinary data-protection rules (including GDPR) even though the page itself is public, so only capture and store fields your actual use case needs.
Ivy Lin
Sep. 3rd 2026
110M+ real IPs with 99.9% access success
Blazing-fast average response ~0.5s for high-concurrency tasks
From only $0.1/GB
Get immediate access to premium residential, datacenter, IPv6 and ISP proxy pools.
