Market share → daily price monitoring
Rank a category's SKUs by GMV with the Data API, then scrape exactly those product pages every day. The cheapest way to run daily price monitoring in Southeast Asia.
The problem
Which products should I even monitor? A category has tens of thousands of SKUs, but the long tail contributes very little GMV.
How it works
- Query the Data API for a category's SKUs ranked by monthly GMV.
- Keep the head — the top N SKUs that carry most of the category's revenue.
- Submit those product pages to the Scraping API daily and diff price, stock and rating.
A taste of the code
from magpie_data import MagpieClient, Metrics, Scraping
client = MagpieClient() # reads MAGPIE_API_KEY
rows = list(Metrics(client).iter_all("skus", country="ID",
category_3="Facial Serum",
date_from="2026-06", date_to="2026-06"))
rows.sort(key=lambda r: float(r.get("gmv_monthly") or 0), reverse=True)
top = rows[:200] # the head carries most of the category's GMV
items = [{"product_id": r["product_id"], "country": "id"} for r in top]
job = Scraping(client).submit("tiktok_pdp", items)
for row in job.wait().rows():
print(row["sku_name"], row["price"])The full, runnable pipeline — argument parsing, retries, partial-result handling, delivery to S3/GCS — is in the repo: open on GitHub →
What it costs
The Data API ranking runs monthly; the daily scrape bills per product page. Monitoring the top 200 SKUs instead of a whole category typically cuts daily cost by >90%.
Estimates are always free — see how credits work.