Begin a Working Nomads jobs scraper with the API linked from the platform's own jobs page.
Treat the returned array as an observed feed, without assuming historical completeness or undocumented pagination.
Preserve url, title, company_name, location, and pub_date before deriving additional labels.
Normalize salary, skill, and hiring-region evidence separately from raw source fields.
Use browser rendering or a managed crawl only for authorized content that the published endpoint does not provide.
Nstdata Crawl supplies a Web Crawling API for LLM-ready data; it does not replace your job-validation and freshness logic.
What Is the Best Starting Point for a Working Nomads Jobs Scraper?
The best starting point is the public API linked from Working Nomads itself, because a publisher-provided structured response can eliminate unnecessary browser automation. The Working Nomads jobs page links to its API alongside other product resources. During preparation of this guide, the linked endpoint returned a JSON array containing job URLs, titles, descriptions, company names, categories, tags, locations, and publication dates.
That observation does not establish an exhaustive catalog, a service-level agreement, or permission for every commercial reuse. Check the current platform terms and contact the publisher when your intended distribution or collection volume requires clarification. A visible API response is a useful engineering input, not a substitute for an agreement.
For a broader permitted job-data pipeline, Nstdata Crawl can collect approved detail pages and return LLM-ready content. Start with the simplest verified source before adding that infrastructure.
Experience Nstproxy Crawl - Start Your Free Trial Today
Why Does Source Selection Matter More Than Selector Syntax?
Source selection matters because a structured publisher feed, rendered listing page, and private application endpoint have different stability and permission boundaries. A tutorial that demonstrates an internal search request may work at one moment while depending on an undocumented implementation detail. The official page's public API link gives you a more direct first option to evaluate.
Keep a source registry containing the entry URL, purpose, owner, observed fields, permitted use, refresh policy, and last successful validation. Recheck the source when its schema changes rather than silently adding browser fallbacks. This makes a failure explainable and prevents the pipeline from drifting into a different access method without review.
If you are building a production web-data pipeline, source selection should be a configuration decision. Avoid embedding a feed URL, a hidden endpoint, and a browser selector in one function that makes its own undocumented choice.
What Do You Need Before Running the Example?
You need Python, access to the publisher-linked endpoint, an approved use for the returned data, and a directory for output files. The standard-library example needs no separate package installation. It deliberately reads a single response rather than inventing pagination parameters.
Input or field
Purpose
Required handling
API URL
Publisher-linked discovery source
Verify the link on the current jobs page
url
Source record identity
Keep original URL and validate its host
title
Original job title
Require a nonempty string
company_name
Publisher-supplied company label
Preserve spelling; do not assume global uniqueness
location
Stated location information
Keep raw text alongside derived regions
pub_date
Source publication value
Preserve original value before parsing
observed_at
Collection time
Generate a UTC timestamp locally
Decide whether you need a research snapshot or a live job recommendation service. A snapshot can be useful with a clear date label. A recommendation product needs an expiration policy and a way to avoid presenting stale listings as current opportunities.
How Does the Collection and Normalization Flow Work?
The collection flow fetches a source response, validates its shape, normalizes supported fields, and upserts accepted jobs by source identity. It should retain enough raw evidence to explain a transformation without exposing unnecessary data.
Keep bounded retry rules separate from job acceptance so a temporary network fault cannot become a source-field value. Do not substitute a derived field for the raw source value. For example, preserve location_raw even if you add hiring_region_normalized. A future parser can then repair classification errors without re-fetching every page. Keep the extractor version with the record when changes affect downstream eligibility or matching.
How Do You Collect Working Nomads Jobs Step by Step?
Collect jobs by validating the official response first, then add only the enrichment stages your field contract requires. These methods are intended for different needs rather than as interchangeable techniques for unrestricted bulk collection.
Method 1: Read the publisher-linked JSON feed
Step 1: Save the single-response collector
Save this code as working_nomads.py. The field names were inspected in the live response during article preparation. The response count is intentionally not part of the contract.
import json
import urllib.request
from datetime import datetime, timezone
from urllib.parse import urlsplit
url ="https://www.workingnomads.com/api/exposed_jobs/"with urllib.request.urlopen(url, timeout=30)as response: payload = response.read(5_000_001)iflen(payload)>5_000_000:raise RuntimeError("Response exceeds local size limit")items = json.loads(payload)ifnotisinstance(items,list):raise TypeError("Expected a list; inspect the current API response")observed = datetime.now(timezone.utc).isoformat()records, seen =[],set()for item in items:ifnotisinstance(item,dict):continue link =str(item.get("url")or"").strip() title =str(item.get("title")or"").strip() parts = urlsplit(link)if parts.scheme !="https"or parts.hostname notin{"workingnomads.com","www.workingnomads.com"}:continueifnot title or link in seen:continue seen.add(link) records.append({"source_url": link,"title": title,"company_name": item.get("company_name"),"location_raw": item.get("location"),"category": item.get("category_name"),"tags_raw": item.get("tags"),"published_at": item.get("pub_date"),"description_html": item.get("description"),"observed_at": observed
})ifnot records:raise RuntimeError("No accepted jobs; retain the last good snapshot")withopen("working-nomads-jobs.json","w", encoding="utf-8")as output: json.dump(records, output, ensure_ascii=False, indent=2)print("accepted jobs:",len(records))
Step 2: Run the collector
Run python working_nomads.py. Confirm that the output file contains source-linked records with original publication values. The parser was exercised against the current publisher-linked response, but your run can return a different number of jobs or a different schema.
Step 3: Check the current source against the output
Compare a small sample with the visible listing pages. Check company labels and location restrictions explicitly. If a field is absent, store null and record the absence; do not ask a model to infer a fact simply to make the record look complete.
Method 2: Enrich an approved detail page
Step 1: Identify a justified enrichment field
Choose a field whose absence changes the product's usefulness, such as an employment-type label. The JobPosting vocabulary can help define the internal schema, but it does not prove the target page publishes the same structure.
Step 2: Inspect the authorized page's loading behavior
Use Developer Tools โ Network โ Fetch/XHR to see whether the field is loaded separately, and Elements to inspect its rendered evidence. Keep the inspection observational unless your agreement permits the specific collection method. Avoid copying an internal search endpoint from a third-party tutorial as a supported publisher interface.
[Screenshot: The official Working Nomads API link and an approved detail page with the enrichment field visible.]
Step 3: Record the enrichment outcome
Store a source URL, evidence text, extraction time, and status for the additional field. Keep an absent field distinct from a failed request. This prevents a temporary browser problem from deleting information obtained from a successful feed read.
Method 3: Add managed collection with Nstdata Crawl
Step 1: Configure a permitted sample
Open the Crawl Playground, choose an approved page URL, and compare Markdown with HTML output. Add JavaScript rendering when the needed content is missing from the initial document. Start with a single page before enabling recursive collection.
Step 2: Connect the documented client
Install nstdata-ai-crawl, then set NSTDATA_API_TOKEN and APPROVED_PAGE_URL in your local environment. The example requires your own token and access rights; it is a documented configuration, not evidence of an authenticated Working Nomads extraction during this article's preparation.
import os
from pathlib import Path
from nstdata_ai_crawl import NstDataClient, ScrapeRequestDto
token = os.environ["NSTDATA_API_TOKEN"]url = os.environ["APPROVED_PAGE_URL"]with NstDataClient(token)as client: result = client.submit_scrape_task_sync(ScrapeRequestDto( url=url, formats=["markdown"], onlyMainContent=True, timeout=60000))if result.success isnotTrueor result.status !="completed":raise RuntimeError("Page retrieval was not completed successfully")if result.data isNoneor result.data.truncated isTrue:raise RuntimeError("Missing or partial content requires review") text = result.data.get_markdown()ifnot text ornot text.strip():raise RuntimeError("No usable Markdown returned") Path("page.md").write_text(text, encoding="utf-8")print("Saved page.md; validate its content before downstream use")
Step 3: Apply the same job acceptance rules
Require a completed page result and nonempty content, then validate the job identity and required fields. A rendered page can still be a login message or an expired listing. Keep its retrieval status separate from whether a job record was accepted.
Where Does Nstdata Crawl Fit in a Multi-Source Job Pipeline?
Nstdata Crawl fits where an approved pipeline needs page retrieval and cleaning across sources with different presentation layers. It is a Web Crawling API for LLM-ready data that supplies the collection stage before your job schema. Discovery and recursive crawling can follow permitted links within page and depth limits. JavaScript rendering and output conversion reduce the need to maintain those components separately. The product does not resolve duplicate employers, determine work authorization, or grant redistribution rights.
Formats for different consumers: Use Markdown for grounded job summaries, HTML for parser inspection, and links for discovery. JSON, PDF, and visual artifacts can support other review and archival requirements.
Built-in proxy option: Standard managed requests can inherit Nstdata's proxy infrastructure without separate proxy host, port, username, and password configuration. Account for proxy usage under the current billing model.
Controlled collection: Keep a small page cap and approved path list, preserve task IDs, and diagnose errors before resubmitting. A larger crawl should follow a measured acceptance rate rather than an arbitrary concurrency target.
Why Do Job Counts or Locations Change Unexpectedly?
Job counts and location labels change when the publisher updates its feed, the source window moves, or a normalization rule changes. Track these possibilities separately before concluding that the labor market changed.
Symptom
Likely cause
Response
Job count drops sharply
Feed window or access behavior changed
Compare response shape and keep the last accepted snapshot
Duplicate jobs appear
URL variants or repeated imports
Use source-specific identity and an upsert operation
Remote roles are classified as global
Location text was discarded
Preserve raw restrictions and revise the classifier
Tags become one long string
Input type assumption changed
Validate the source type and normalize deliberately
Description is blank after rendering
Missing field or unexpected page
Inspect HTML and page state before retrying
Use the Python JSON interface to parse the response, and add explicit application checks for expected object types. Successful JSON decoding alone says nothing about whether the document contains jobs.
How Should You Maintain the Dataset Over Time?
Maintain the dataset with stable source keys, timestamped observations, and explicit freshness states. Avoid marking a role closed merely because it disappears from one feed response. If your product requires confirmation, define a permitted recheck policy and keep uncertain records out of โcurrently hiringโ recommendations.
Start with the public endpoint and a small manually checked sample, then add only necessary enrichment. Use pagination and discovery controls when your approved scope grows. Nstdata's alternate HTML output can help diagnose content mismatches before they affect job recommendations.
Q: Does Working Nomads expose a public data endpoint?
Its jobs page currently links an API endpoint that returned a JSON array during research. Recheck the source link, terms, and response shape before relying on it.
Q: Does the response prove complete job coverage?
No. The observed response does not establish historical completeness or a documented pagination contract. Describe coverage according to what you actually collect.
Q: Do I need a browser for the demonstrated collector?
No. The single-response example uses Python's standard library. Browser rendering becomes relevant only for additional approved page content.
Q: Can an AI model fill in missing hiring regions?
An AI model should not invent missing eligibility facts. Keep the field unknown unless you have source evidence and a reviewable derivation.
Q: What should I monitor after deployment?
Monitor accepted jobs, missing fields, duplicate IDs, response-schema changes, collection errors, and record age. These reveal different failure modes that one request-success metric hides.
Lena Zhou
Nov. 9th 2026
Crawl entire websites with a single API request
Turn any website into Markdown, HTML, JSON, links, PDFs and more โ without managing crawling infrastructure.
