How jobs work
Scraping API · on-demand jobsAuth, the job lifecycle, approval, statuses, results and billing.
Every request needs a bearer token in the Authorization header:
Authorization: Bearer YOUR_API_TOKEN
A missing or invalid token gets 401. A valid token that isn't scoped for the endpoint you're calling gets 403 — contact support to enable access. Create a key in Scraping jobs and fund your wallet before submitting jobs; the same key also works for the Data API.
Every endpoint works the same way: submit → job ID → poll → download. Nothing returns the full result in the submit response.
- Submit. POST the endpoint's /submit path (or its single POST route). You get back a job_id or task_id.
- Poll. GET the matching retrieve/{id} path until the status is terminal (see the table below). Keep the ID and keep polling — don't resubmit while a job is still running.
- Download. Depending on the endpoint, results come back inline in the retrieve response, via a signed download_url, or per item from a separate data path. See Results below.
Single-item endpoints (Shopee Product Page, Product Ratings, the bulk CSV uploads, and every other marketplace's PDP/search/merchant endpoints) have no approval step and typically finish in seconds to tens of minutes. The four endpoints that go through Shopee's manual review — Variant Sold, Listing product (Search Items), Merchant Items, and Shopee App Search — queue first and can take hours; see Approval flow below.
Jobs for Variant Sold, Merchant Items, Search Items (Listing product), and Shopee App Search go through manual approval:
- You submit → status AWAITING_APPROVAL. No charge yet.
- Our team reviews and approves, typically within 24 hours.
- Approved work only runs inside Shopee's processing window — Monday–Friday, 08:00–17:00 Asia/Jakarta (WIB). Outside that window an approved job sits queued with a runs_after timestamp for the next window; that's expected, not an error — keep polling, don't resubmit.
- Job starts → PROCESSING, then COMPLETED or PARTIAL_COMPLETE. If approval is declined or the 24-hour window passes first, the job becomes REJECTED and nothing is charged.
Shopee Product Page, PDP Bundling, Product Ratings, and every bulk CSV endpoint (across Shopee, Tokopedia, Blibli, TikTok Shop, Amazon, and Mercado Libre) do not require approval — they start processing immediately, any hour.
Recurring (cron/run_at) schedules for Listing product and Merchant Items follow the same window — a submission isn't even created outside it.
| Status | Meaning | What to do next |
|---|---|---|
| AWAITING_APPROVAL | Submitted; waiting on manual review (approval-gated endpoints only). No charge yet. | Wait. Don't resubmit to speed it up. |
| SCHEDULED | Waiting for its scheduled run_at or next cron time. | Nothing to do — it submits itself when due. |
| QUEUED | Accepted but not yet running (also seen as SUBMITTED, PENDING, or INITIALIZING depending on the endpoint). | Keep polling retrieve. |
| PROCESSING | Approved (if gated) and actively scraping. | Keep polling; an empty worker queue doesn't mean results are ready. |
| COMPLETED | Every item succeeded (also reported as SUCCESS by some endpoints). | Download the results. |
| PARTIAL_COMPLETE | Some items or pages failed; the rest succeeded. | Download what succeeded. Credits for the failed items are refunded. |
| TERMINATED | A bulk task was stopped (by you) before finishing. | Retrieve again for the completed-so-far export; unused credits are refunded. |
| FAILED | All items failed. | Credits are refunded. Check the error message before retrying. |
| REJECTED | Approval was declined, or the 24-hour approval timeout passed. | Nothing was charged. Resubmit if you still need the data. |
| CANCELED | Stopped before completing (also CANCELLED). | Unprocessed credits are refunded. |
Per-item status inside a job's results (not the job status itself) is typically PENDING / SUCCESS / FAILED.
HTTP status codes
- 200 / 202 — success / job accepted
- 401 — invalid or missing token
- 402 — insufficient credits
- 403 — token lacks scope for this endpoint
- 404 — job not found
- 422 — validation error
- 503 — endpoint unavailable; contact support rather than retrying repeatedly
Troubleshooting
- Save every job_id — if a network error hides the submit response, check your jobs before resubmitting.
- awaiting_approval / queued: no charge yet; resubmitting creates a second, separately billed job.
- partial_complete: inspect individual item or page statuses — unsuccessful ones aren't billed.
- Inline: single-item endpoints (Shopee Product Page, Product Ratings) return the result directly in the retrieve response.
- Signed download_url: most bulk endpoints compile results into a single gzip-compressed JSONL (or CSV) file and hand back a link to it. Tokopedia PDP Bulk and Blibli PDP Bulk links are plain signed GCS URLs valid for 7 days — no bearer token needed. Other endpoints' download_url values require the same Bearer token you submitted with; the guide doesn't state a fixed expiry for those, so download promptly.
- Per item: Search Items and Merchant Items instead give each successful item its own download_url path under /v1/shopee/pgnt/jobs/{job_id}/items/{token}/data, which still needs your Bearer token.
- Most records carry collected_at as an integer Unix timestamp (UTC seconds) — when the data was actually captured upstream, not when your job ran.
Every "Result data" snippet in the API reference is trimmed to one item. These are the same real, anonymized payloads in full — up to 25 records each, downloadable without a token.
| Endpoint | Records | Size | Captured | Download |
|---|---|---|---|---|
| Variant Sold | 1 | 4 KB | 2025-09-16 | variant-sold.json |
| Shopee Product Page | 1 | 178 KB | 2025-09-16 | shopee-product-page.json |
| Shopee PDP Bundling | 6 | 234 KB | 2026-10-08 | shopee-pdp-bundling.json |
| Shopee Listing product (search) | 1 | 8 KB | 2025-09-16 | shopee-search-items.json |
| Shopee Listing product (merchant) | 1 | 111 KB | 2026-08-01 | shopee-merchant-items.json |
| Shopee App Search | 1 | 3 KB | 2026-10-13 | shopee-app-search.json |
| Tokopedia PDP Bulk | 25 | 175 KB | 2026-10-26 | tokopedia-pdp-bulk.json |
| Tokopedia Search Bulk | 1 | 240 KB | 2026-10-23 | tokopedia-search-bulk.json |
| Tokopedia Merchant Bulk | 25 | 31 KB | 2026-10-26 | tokopedia-merchant-bulk.json |
| Blibli PDP Bulk | 20 | 264 KB | 2026-10-26 | blibli-pdp-bulk.json |
| Blibli Merchant Bulk | 25 | 152 KB | 2026-10-26 | blibli-merchant-bulk.json |
| Shopify PDP Bulk | 7 | 39 KB | 2026-07-30 | shopify-pdp-bulk.json |
| Shopify Search Bulk | 1 | 254 KB | 2026-10-26 | shopify-search-bulk.json |
| TikTok PDP Bulk | 6 | 274 KB | 2026-10-26 | tiktok-pdp-bulk.json |
| Amazon PDP Bulk | 20 | 107 KB | 2026-10-26 | amazon-pdp-bulk.json |
| Amazon Search Bulk | 1 | 8 KB | 2026-07-23 | amazon-search-bulk.json |
| Mercado Libre PDP Bulk | 1 | 343 KB | 2026-10-08 | mercadolibre-pdp-bulk.json |
| Mercado Libre Search Bulk | 1 | 48 KB | 2026-10-05 | mercadolibre-search-bulk.json |
| Lazada — Category Tree | 1 | 25 KB | 2026-10-14 | lazada-category-tree.json |
| Lazada — Seller Directory | 1 | 657 B | 2026-10-14 | lazada-seller-directory.json |
| Lazada — Seller Details | 1 | 49 KB | 2026-10-14 | lazada-seller-details.json |
| Lazada — Seller Promotions | 1 | 516 B | 2026-10-14 | lazada-seller-promotions.json |
| Lazada — Keyword Tracking | 1 | 679 B | 2026-10-14 | lazada-keyword-tracking.json |
- No-approval endpoints (bulk CSV uploads, single-item fetches, and the rest of the non-gated endpoints) reserve credits upfront at submit, for every row, URL, or item in the request.
- Approval-gated endpoints (Variant Sold, Search/Merchant Items, Shopee App Search) are not charged at submit. Credits are deducted only once a job is approved and actually starts processing — a job that's rejected or times out before that point costs nothing.
- Refunds cover whatever doesn't come back successfully: failed and canceled items are all non-billable. Terminating a bulk task early refunds its unfinished items too.
- Prices vary per endpoint and per unit (request, URL, item, or page) and change over time — see the live rate card rather than relying on a number in these docs.
Instead of polling, bulk endpoints can notify a signed webhook when results are ready — see Scraping webhooks.