Data dictionary
Data API · monthly dataset32 fields per row — what every export and metrics response is made of.
One row per product × platform × country × month. Monthly metrics (gmv_monthly*, sold_monthly, review_monthly) are that month's figures; stock and rating are the month-end snapshot, while price is the lowest price recorded during the month.
Money fields are in the market's local currency unless the name says _usd. Exports deliver every field below as Parquet or CSV — estimate a pull →
What the row is about — stable identifiers plus display fields.
| Field | Type | Description |
|---|---|---|
| product_id | STRING | The listing's own ID on its platform, carried through unchanged from source. The primary key for joining every other field back to one specific listing. |
| sku_name | STRING | The listing's title exactly as scraped, with no cleanup — good for a human eyeballing the row, but wording and casing vary listing to listing, so use sku_type_complete instead when matching or grouping products. |
| sku_type_complete | STRING | Standardized product type, matched against a maintained reference dictionary rather than derived from the raw title — better for grouping identical products than sku_name. Shows a placeholder when no dictionary match is found. |
| brand | STRING | Brand name, resolved by matching the listing's product/merchant identity against Magpie's brand reference data rather than read directly off the title. Shows the placeholder when no brand has been confirmed yet. |
| image | STRING | Direct URL to the listing's main product photo, as hosted by the platform — carried straight through from the raw scrape. |
| url | STRING | Canonical listing URL on the platform. |
| created_date | DATE | Date the listing first appeared on the platform. (Shopee & Tokopedia only — not currently captured for Lazada, TikTok, or Blibli; empty for listings on those marketplaces.) |
| flag_GWP | BOOL | True when the listing is a gift-with-purchase / bundle freebie. |
Where the row sits — market, platform, category tree, month.
| Field | Type | Description |
|---|---|---|
| country | STRING | Market code: ID, TH, VN, SG, PH, MY. Sets the local currency of non-USD money fields. |
| ecommerce_platform | STRING | Marketplace: Shopee, Lazada, Tiktok, Tokopedia, 'Tokopedia | Shop', Blibli. |
| category_1 | STRING | Top-level category (e.g. Beauty & Personal Care). |
| category_2 | STRING | Second-level category (e.g. Skincare). |
| category_3 | STRING | Leaf category (e.g. Facial Serum) — the export's atom grain. |
| month | DATE | First day of the month the row aggregates. |
Listed prices and realized (sold) averages. Local currency unless the field says USD.
| Field | Type | Description |
|---|---|---|
| price | FLOAT64 | The lowest price recorded for the listing during the month, local currency — the cheapest price point observed, not a price taken specifically at month's end. |
| price_usd | FLOAT64 | Price converted to USD using a fixed exchange rate per country, not a live daily rate — treat cross-market USD comparisons as directionally useful rather than accounting-precise. |
| before_discount_price | FLOAT64 | Pre-discount reference price, calculated from the price and discount columns rather than scraped from the listing's strike-through label. Equals price when no discount applies. |
| discount | FLOAT64 | Listed discount as a fraction (0.70 = 70% off). |
| avg_price_sold | FLOAT64 | Average realized selling price across the month's sales, local currency. |
The analysis metrics — what actually moved this month.
| Field | Type | Description |
|---|---|---|
| sold_monthly | INT64 | Units sold in this month. |
| gmv_monthly | FLOAT64 | GMV for this month, local currency (≈ units × realized price). |
| gmv_monthly_usd | FLOAT64 | GMV for this month in USD — the cross-market analysis metric. |
| review_monthly | INT64 | Reviews received in this month. |
| rating | FLOAT64 | Listing's rating (0–5) at month end. |
| stock | INT64 | Listing-level stock at month end. |
Who sells it and from where — official-store detection and regional cuts.
| Field | Type | Description |
|---|---|---|
| merchant_id | STRING | Platform's shop identifier. |
| merchant_name | STRING | Shop display name. |
| merchant_url | STRING | Shop URL on the platform. |
| merchant_badge | STRING | Platform badge, e.g. 'Shopee Mall', 'Star Seller' — official signal. |
| merchant_cities | STRING | Shop's city (as declared on the platform). |
| merchant_province | STRING | Shop's province/state. |
| merchant_region | STRING | Coarse region rollup (e.g. Jawa, Sumatera). |
Top-N export files additionally carry a rank column (1 = highest GMV in the atom). Schema questions? The live source of truth is the API reference.