Python client

pip install magpie-data

The client reads MAGPIE_API_KEY from the environment — get a key from your dashboard. Source, issues and runnable pipeline recipes live on GitHub; releases on PyPI.

from magpie_data import MagpieClient, Exports

exports = Exports(MagpieClient())  # reads MAGPIE_API_KEY

# 1. Estimates are always FREE — exact rows and credits, nothing charged.
estimate = exports.estimate(
    country="ID",
    category_3=["Facial Serum"],
    date_from="2026-05",
    date_to="2026-05",
)
print(estimate)  # → 289,621 rows · credits and USD price

# 2. Only then commit — with a hard spend ceiling.
job = exports.submit(
    country="ID",
    category_3=["Facial Serum"],
    date_from="2026-05",
    date_to="2026-05",
    format="parquet",
    max_credits=5_000,  # refuses to run if it would cost more
)
job.download_to("s3://my-bucket/magpie/")  # or gs:// or ./local-dir

from magpie_data import MagpieClient, Scraping

s = Scraping(MagpieClient())
job = s.submit("amazon_pdp", [
    {"asin": "B0C1234567", "country": "us"},
    {"asin": "B0C7654321", "country": "de"},
])
for row in job.wait().rows():   # polls with backoff, streams gzipped JSONL
    print(row["title"], row["price"])

Items are validated client-side before anything is charged; lists longer than a product's per-job cap are split automatically with submit_all(). Long jobs can be resumed later by id with Scraping.resume() — no re-paying.

The repo ships production-shaped pipelines you can copy: monthly exports to S3, daily price monitoring, an Airflow DAG, warehouse loads, share of search and Amazon cross-market ranking.