How to Extract Product Specifications from Manufacturer and Retailer Pages

Product comparison and procurement systems rarely fail because a page has no data. They fail because the same fact arrives in several forms: a manufacturer calls it “net weight,” a retailer calls it “item weight,” one page says 1.5 kg, another says 3.3 lb, and a third attaches the value to a different color or capacity variant.

To reliably extract product specifications from websites, treat every value as an observation with evidence—not as an unquestioned fact. Your pipeline should preserve what the page said, where it said it, which variant it applied to, and how your system interpreted it.

This approach produces a catalog that is easier to audit, safer to merge, and more useful when pages inevitably change.

Start with a canonical schema, not site-specific fields

Before writing selectors, define the internal representation that downstream search, comparison, and review workflows require. The canonical schema should separate identity, specifications, and evidence.

A practical product record might contain:

type ProductIdentity = {
  brand?: string;
  name?: string;
  model?: string;
  mpn?: string;
  gtin?: string;
  variant?: Record<string, string>;
};

type SpecificationObservation = {
  canonicalField: string;
  rawLabel: string;
  rawValue: string;
  parsedValue?: number | string;
  unit?: string;
  normalizedValue?: number;
  normalizedUnit?: string;
  status: "observed" | "missing" | "ambiguous" | "conflicting" | "needs_review";
  sourceUrl: string;
  retrievedAt: string;
  extractionMethod: "json_ld" | "table" | "dom" | "text";
  sourceLocation?: string;
  variantContext?: Record<string, string>;
};

The important design choice is that rawLabel and rawValue are first-class fields. Do not reduce a source statement such as Dimensions: 10.2 × 4.8 × 2.1 in to a few numbers without retaining the original statement.

This mirrors the distinction available in Schema.org PropertyValue: a product characteristic can retain a name, value, unit, identifier, and description. For measurements, QuantitativeValue supports point values, ranges, and units. Your internal model does not need to mirror Schema.org exactly, but it should preserve the same useful separation between a displayed claim and a parsed interpretation.

Map labels into fields deliberately

A label dictionary is useful, but it must be scoped by category and sometimes by brand:

const aliases: Record<string, string> = {
  "product weight": "weight.net",
  "item weight": "weight.net",
  "net weight": "weight.net",
  "package weight": "weight.shipping",
  "power consumption": "power.rated",
  "rated power": "power.rated",
  "wattage": "power.rated"
};

Avoid universal mappings that hide semantic differences. “Weight” could describe the product, its shipping package, a filled container, or a maximum load. If the label and nearby context do not establish which meaning applies, store the observation as ambiguous or needs_review rather than forcing it into weight.net.

Extract in a layered order

A resilient extractor uses several acquisition and parsing paths. Each later layer is a fallback, not a replacement for earlier evidence.

1. Read structured product data first

Inspect application/ld+json blocks before parsing the visible page. Google recommends JSON-LD for product structured data, and product markup can expose identifiers, brand, model, offers, and product characteristics in a machine-readable form. See Google’s Product structured-data documentation and Schema.org Product.

Your JSON-LD parser should handle these realities:

• a script may contain an array or an @graph rather than one object;
• a page can contain multiple Product entities;
• additionalProperty may carry important specifications;
• values can be strings, numbers, PropertyValue objects, or QuantitativeValue objects;
• the structured record may describe a parent product while the page presents a selected variant.

For example, preserve both the source-shaped value and your interpretation:

function parseProperty(p: any): SpecificationObservation {
  const rawValue = String(p.value ?? p.description ?? "");

  return {
    canonicalField: resolveCanonicalField(p.name),
    rawLabel: p.name ?? "unnamed property",
    rawValue,
    parsedValue: typeof p.value === "number" ? p.value : undefined,
    unit: p.unitCode ?? p.unitText,
    status: p.value == null ? "missing" : "observed",
    sourceUrl: currentUrl,
    retrievedAt: new Date().toISOString(),
    extractionMethod: "json_ld",
    sourceLocation: "script[type='application/ld+json']"
  };
}

Prefer dedicated structured properties where they exist, but expect additionalProperty for specifications without a dedicated vocabulary field. That is the intended role of PropertyValue.

2. Parse specification tables by relationships

When structured data is incomplete, product pages often present a technical-details table. Do not assume every row is simply td[0] = label and td[1] = value.

Tables can include captions, section headers, row groups, multi-column comparisons, and cells associated through scope, headers, or id. The W3C guidance on associating table headers and data cells is a useful parsing model: resolve a data cell’s applicable row and column headers before deciding what the value means.

For a straightforward two-column table, extraction is simple:

for (const row of tableRows) {
  const label = normalizedText(row.querySelector("th")?.textContent);
  const value = normalizedText(row.querySelector("td")?.textContent);
  emitObservation(label, value, "table");
}

For a complex table, build a grid and collect header context. A value beneath a Capacity column and within a Model B row group should not be emitted as a page-wide capacity without that model context.

3. Use targeted rendered-page extraction when needed

Some retailer pages populate specifications after client-side rendering. In that case, use a browser fallback and wait for a specific specifications container, tab panel, or labeled section.

Playwright’s locator model is designed around targeted locators, auto-waiting, and retryability; its documentation also warns that reading a dynamically changing list immediately can be flaky. See Playwright locators.

A focused pattern is better than an arbitrary delay:

await page.goto(url, { waitUntil: "domcontentloaded" });

const specs = page.getByRole("region", { name: /specifications|details/i });
await specs.waitFor({ state: "visible" });

const rows = await specs.locator("tr").allTextContents();

In production, you may need site-specific selectors. Keep them as a final layer after structured data, semantic labels, and stable attributes. Record the selector or extraction rule in sourceLocation so a failed field can be traced to the rule that produced it.

Build an auditable extraction workflow: If you want to test URL retrieval and inspect the extracted content before committing to parsing rules, create a PagePith account.

Normalize values only when the transformation is unambiguous

Normalization helps comparison, but it is not a license to erase nuance.

For an observed 1.5 kg, you can safely retain the original and add a normalized value:

{
  "canonicalField": "weight.net",
  "rawLabel": "Net weight",
  "rawValue": "1.5 kg",
  "parsedValue": 1.5,
  "unit": "kg",
  "normalizedValue": 1500,
  "normalizedUnit": "g",
  "status": "observed"
}

For 10–12 hours, preserve an interval rather than selecting a midpoint. For up to 12 hours, preserve the qualifier; it is not equivalent to a guaranteed duration. For 120 V / 60 Hz, model voltage and frequency as separate fields only when the label and format clearly establish that interpretation.

Use a deterministic unit registry with explicit conversions. If a unit is unfamiliar, locale-dependent, or missing, retain the raw value and mark normalization as unavailable. Schema.org’s QuantitativeValue supports both standard unit codes and unit text, which is a useful reminder that not every page will provide a clean standardized unit.

Match manufacturer and retailer records before merging

A manufacturer page may be authoritative for technical dimensions, while a retailer page may be more current for a sellable SKU. Neither should automatically overwrite the other.

First establish identity using the strongest available identifiers:

1. GTIN, when present and applicable.
2. MPN plus brand.
3. Brand, model, and explicitly matched variant attributes.
4. A review queue when identity remains uncertain.

Do not merge solely on a similar product title. A family name can cover several sizes, finishes, regions, and revisions. Keep each page’s claims as independent observations until the exact product and selected variant are known.

Then reconcile by field:

• If two compatible sources agree after normalization, retain both provenance records and select a canonical display value.
• If they differ, set status: "conflicting"; do not silently choose one.
• If one source lacks a field, record it as missing for that source rather than borrowing a value from a related product.
• If a retailer’s value is tied to the selected color or size, attach that variant context before comparison.

The goal is not to make every record look complete. It is to distinguish verified agreement from absence, ambiguity, and conflict.

Make provenance part of the data contract

Field-level provenance is what turns extraction output into a system people can trust. At minimum, store:

• requested and final source URL;
• retrieval timestamp;
• extraction method and parser version;
• raw label and raw text;
• source location, such as JSON-LD path, table caption and row, or locator;
• selected variant context;
• normalization rule and version, if applied.

This is consistent with the purpose of W3C PROV-O, which models the entities, activities, and agents involved in producing information so users can evaluate origin and reliability.

Provenance also makes maintenance practical. When an upstream site redesign changes a specification component, you can find the observations produced by the affected rule and reprocess only those records.

Crawl efficiently and report uncertainty

Recurring product crawls should avoid downloading unchanged pages unnecessarily. Cache response metadata such as ETag and Last-Modified, then use conditional requests where supported. HTTP validators and If-None-Match or If-Modified-Since can let a server indicate that a resource has not changed, as described in MDN’s conditional request guide.

At the same time, make uncertainty visible in your output. A useful extraction response includes a field-status summary:

{
  "productKey": "brand:model:variant",
  "fieldSummary": {
    "weight.net": "observed",
    "dimensions": "observed",
    "battery.capacity": "missing",
    "power.rated": "conflicting"
  },
  "reviewRequired": true
}

This is safer than populating missing fields from category defaults, nearby products, or a different variant. Accurate partial data is more valuable than apparently complete data with unsupported assumptions.

A limited PagePith demonstration

The supplied PagePith proof shows a successful fetch-tier retrieval of Google’s Product structured-data documentation at https://developers.google.com/search/docs/appearance/structured-data/product. It returned the page title, reported a content length of 8,522, and produced Markdown beginning with the page’s introduction to Product structured data.

That is useful for an extraction workflow because it demonstrates retrieval plus content available for inspection from this specific documentation URL. It does not establish that PagePith can render every JavaScript-driven retailer page, extract every table format, normalize units, match variants, or resolve source conflicts. Those remain responsibilities for the pipeline described above and should be tested against the sites and page types you intend to support.

Build for evidence, not just values

A durable product-specification system has a simple discipline: structured data first, semantic tables second, targeted rendering only when necessary, and raw evidence retained throughout.

When every value retains its source and variant context, you can safely normalize comparable measurements, surface genuine disagreements, and send uncertain records to review instead of manufacturing certainty.

Ready to inspect source content and prototype your retrieval layer? Sign up for PagePith.