> ## Documentation Index
> Fetch the complete documentation index at: https://docs.entityml.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Quality

> Coverage windows, known gaps, and validation steps for historical Polymarket and Kalshi data.

## Coverage model

Coverage is market-specific. A market can have stored objects for a UTC date while still having intra-day gaps, null fields, or source-specific quality issues.

<CardGroup cols={2}>
  <Card title="Check stored dates" icon="calendar-days" href="/api-reference/endpoint/get-polymarket-market-date-range">
    Use date-range endpoints to find which UTC dates have S3 objects for a condition ID or ticker.
  </Card>

  <Card title="Inspect raw events" icon="list-tree" href="/api-reference/endpoint/get-market-data">
    Raw endpoints expose snapshots, deltas, and Polymarket gap markers when they are present.
  </Card>

  <Card title="Use summaries for quotes" icon="chart-candlestick" href="/api-reference/endpoint/get-polymarket-orderbook-summary">
    Summary endpoints reconstruct top-of-book quotes and OHLC candles from raw orderbook events.
  </Card>

  <Card title="Prefer ranges for audits" icon="scan-search" href="/api-reference/endpoint/get-polymarket-market-data-range">
    Range endpoints let you audit a partial day or a multi-day window with cursor pagination.
  </Card>
</CardGroup>

## Known windows

<AccordionGroup>
  <Accordion title="Kalshi parser issue: March 31-April 24, 2026">
    Affected Kalshi orderbook records from `2026-03-31` through `2026-04-24` can have `price: null`, `delta: null`, and empty snapshot sides even when the event count and order are present.

    The visible symptom in `GET /api/v1/kalshi/market/orderbook-summary` is `quote_count > 0` while `best_bid`, `best_ask`, `mid_price`, `spread`, and size fields remain `null`.

    The confirmed customer reproducer was a KXBTC15M ticker on `2026-04-04`, for example `KXBTC15M-26APR040030-30`.

    A forward fix was deployed on `2026-05-03` so new Kalshi raw records preserve the original websocket payload under `msg`. The March 31-April 24 affected payloads were not recoverable from the available Entity storage because the old worker discarded the original payload before durable storage.
  </Accordion>

  <Accordion title="Kalshi storage ingestion gap: May 4-May 11, 2026">
    Kalshi raw storage is missing for `2026-05-04` through `2026-05-11`. Production capture resumed on `2026-05-12`, and current API-facing Kalshi partitions are available again from that date forward.

    The missing May 4-May 11 window is a known coverage gap in EntityML storage. Date-range endpoints can show `2026-05-03` and `2026-05-12` for long-lived tickers, but they will not fill the days in between unless a separate external historical replay source is added.

    For audits that cross this window, explicitly test each UTC date with the Kalshi date-range and raw-data endpoints instead of assuming continuous coverage between the first and last stored dates.
  </Accordion>

  <Accordion title="Polymarket April 2026 collector gaps">
    Some Polymarket markets have gap-marked or missing data during an April 2026 collector instability window. The most documented production incident was on `2026-04-08` and `2026-04-09`; market-level impact can span adjacent UTC dates depending on shard assignment, collector state, and reconciliation status.

    For raw Polymarket data, look for `event_type` values such as `gap_start` and `gap_end`. These markers indicate that a collector opened or closed a recovery gap for one or more asset books.

    The date-range endpoint reports stored dates, not second-by-second completeness. For April 2026 work, confirm the exact market and timestamp window with raw data or range queries.
  </Accordion>

  <Accordion title="Early history before broad capture">
    Polymarket has selected historical data from `2025-09-01`. Broad Polymarket crypto-related coverage begins on `2026-02-04`, and the API rejects Polymarket data requests before `2026-02-08`.

    Kalshi has selected historical data from `2026-02-17`. Broad Kalshi crypto-related coverage begins on `2026-02-25`.

    Broad all-market capture begins on `2026-04-02` for Polymarket and `2026-03-31` for Kalshi, subject to the April quality notes above.
  </Accordion>
</AccordionGroup>

## Validation workflow

<Steps>
  <Step title="Resolve the market identifier">
    For Polymarket, use [Look Up Polymarket Slug](/api-reference/endpoint/get-polymarket-slug-lookup) if you only have a slug or URL. For Kalshi, use the exact ticker from Kalshi.
  </Step>

  <Step title="Check stored date range">
    Call the matching date-range endpoint to confirm whether the market has stored objects for the UTC dates you need.
  </Step>

  <Step title="Sample raw records">
    Pull a small raw page around the target time. For Polymarket, inspect `event_type` and gap markers. For Kalshi, inspect whether snapshots have `yes` or `no` levels and whether deltas have non-null price fields.
  </Step>

  <Step title="Use summaries for top-of-book analytics">
    Use orderbook summaries for best bid, best ask, midpoint, spread, and OHLC. Confirm `quote_count`, `is_forward_filled`, and null fields before treating a bucket as tradable quote data.
  </Step>
</Steps>

## What nulls mean

| Field pattern                                    | Interpretation                                                                                                                                                    |
| ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `data_count = 0`                                 | No stored records matched the requested market and date or range.                                                                                                 |
| `quote_count = 0` and `is_forward_filled = true` | No quote was observed in that bucket; values were carried forward from the previous bucket.                                                                       |
| `quote_count > 0` and quote fields are `null`    | Events were observed, but a complete top-of-book could not be reconstructed. This can happen before both sides are seen or during the Kalshi parser issue window. |
| Polymarket `gap_start` / `gap_end` records       | The collector marked a recovery gap. Treat the covered interval as lower confidence unless reconciled data confirms coverage.                                     |

<Tip>
  For high-stakes backtests, store the raw response payloads and the date you pulled them. Historical availability can improve after backfills, but unrecoverable parser-loss windows remain documented here.
</Tip>