UK Commercial Property API Pagination: Checkpoint Rent and Sale Exports

Published October 30, 2026

currentPage is an integer, but it is not a durable export checkpoint. Saving only currentPage: 4 loses which commercial search produced that page, whether its rows reached disk, and whether the next response still describes the same result set. Build the checkpoint around that uncertainty before scheduling another request.

currentPage belongs to one commercial search, not your whole export

Both commercial-property-to-rent and commercial-property-to-sale returned an object with status=true, a string message, a data array of objects, and integer currentPage, resultsPerPage and totalResultCount. These are the verified commercial search envelopes; they do not verify individual listing attributes or every filter combination.

Treat a run as one operation, one fixed filter set, sequential page requests. A rental search and a sale search need separate run directories even when their area matches. The request input is page; the returned acknowledgement is currentPage. Start at page 1 and require that acknowledgement to match before advancing.

Freeze rent or sale filters before writing a checkpoint

Record the operation and every non-page query input, including whether optional inputs were omitted. Resolve a different area through auto-complete with location text, then select its data[].locationIdentifier. Preserve the full value rather than sending a display name. The rental quick-start uses identifier=REGION^274, search_radius=0.0 and page=1; it does not guarantee today's inventory.

Commercial floor-area bounds are min_size and max_size. Bind them to an explicit size_unit: SquareFeet, SquareMetres, Acres or Hectares. For rent, keep min_price and max_price with the selected price_type, either PricePerUnit or PricePerCalendarMonth. The selector alone does not establish comparable normalized rents. Sale search has no rental-price-basis input.

Keep comma-separated property types such as Office,BusinessPark unchanged. Commercial sort_by values and radius distance units are not documented; do not invent a sorting token or label a radius code as miles. A changed filter starts a new run, not a resumed checkpoint.

Validate data and all three integer pagination fields

Diagnose a saved response before adding retries. HTTP success and the gateway's code/message/data business status are separate: require gateway code=200, then inspect its data payload's status and message. HTTP 200 alone is insufficient. Use only an authorized platform key; query URLs contain it, so redact URLs from logs and exceptions.

Inside the property payload, require an array of objects and all three integer pagination fields. Reject booleans masquerading as integers, an unexpected page, negative totals, or a nonpositive page size for this traversal. A missing field is a stop, not permission to substitute zero. An empty array can be a legitimate no-match response; it still cannot justify silently advancing an export.

Commit each raw page before advancing the saved page

Use two durable commits: first the raw response, then the checkpoint acknowledging it. The Python core below makes one explicit request per invocation. It reuses an already-saved candidate after interruption instead of fetching that page again. It deliberately treats rows as opaque objects, not an invented listing schema.

import hashlib
import json
import os
from pathlib import Path
from urllib.parse import urlencode
from urllib.request import urlopen

OP = "commercial-property-to-rent"
FILTERS = {"identifier": "REGION^274", "search_radius": "0.0"}
RUN = Path("commercial-rent-run")
CONTEXT = {"operation": OP, "filters": FILTERS}


def durable(path, raw):
    temp = path.with_name(path.name + ".tmp")
    with temp.open("wb") as stream:
        stream.write(raw)
        stream.flush()
        os.fsync(stream.fileno())
    os.replace(temp, path)
    descriptor = os.open(path.parent, os.O_RDONLY)
    try:
        os.fsync(descriptor)
    finally:
        os.close(descriptor)


def save_state(path, state):
    durable(path, json.dumps(state, sort_keys=True).encode())


def advance():
    RUN.mkdir(exist_ok=True)
    checkpoint = RUN / "state.json"
    if not checkpoint.exists():
        if any(RUN.iterdir()):
            raise RuntimeError("Nonempty run without checkpoint; review it")
        save_state(checkpoint, dict(context=CONTEXT, next_page=1,
                   total=None, size=None, hashes=[], rows=0, stopped=False))
    state = json.loads(checkpoint.read_bytes())
    if state["context"] != CONTEXT:
        raise RuntimeError("Search changed; use a new run directory")
    if state["stopped"]:
        return
    page = state["next_page"]
    target = RUN / f"page-{page:06d}.json"
    if not target.exists():
        query = dict(FILTERS, page=page, key=os.environ["PLATFORM_API_KEY"])
        url = f"https://api.opendata-api.com/api/project/rightmove/{OP}?"
        try:
            with urlopen(url + urlencode(query), timeout=30) as response:
                raw = response.read()
        except Exception:
            raise RuntimeError("Transport stopped; checkpoint unchanged") from None
        durable(target, raw)
    gateway = json.loads(target.read_bytes())
    if not isinstance(gateway, dict) or gateway.get("code") != 200:
        raise RuntimeError("Review gateway business status")
    payload = gateway.get("data")
    if (not isinstance(payload, dict) or payload.get("status") is not True
            or not isinstance(payload.get("message"), str)):
        raise RuntimeError("Review property payload status")
    rows = payload.get("data")
    fields = ("currentPage", "resultsPerPage", "totalResultCount")
    if any(type(payload.get(field)) is not int for field in fields):
        raise RuntimeError("Integer pagination fields required")
    current, size, total = (payload[field] for field in fields)
    if current != page or size <= 0 or total < 0:
        raise RuntimeError("Invalid page acknowledgement")
    if not isinstance(rows, list) or any(not isinstance(row, dict) for row in rows):
        raise RuntimeError("Expected an array of objects")
    if not rows:
        raise RuntimeError("Empty batch; inspect without advancing")
    digest = hashlib.sha256(json.dumps(rows, sort_keys=True).encode()).hexdigest()
    if digest in state["hashes"]:
        raise RuntimeError("Repeated batch; inspect without advancing")
    if state["total"] is not None and (total, size) != (state["total"], state["size"]):
        raise RuntimeError("Pagination changed; review saved candidate")
    state.update(next_page=page + 1, total=total, size=size,
                 hashes=state["hashes"] + [digest], rows=state["rows"] + len(rows),
                 stopped=current * size >= total)
    save_state(checkpoint, state)


if __name__ == "__main__":
    advance()

Run this with a secret-injected PLATFORM_API_KEY, never a committed key. Use a single writer and a local filesystem supporting atomic rename and directory sync. Review the saved candidate on any failure; do not delete evidence to force continuation. Add an explicit request ceiling before scheduling repeated invocations. A transport timeout is ambiguous: reconcile it before another invocation, rather than blindly retrying a potentially charged request.

For sale, choose commercial-property-to-sale, a separate directory and sale-specific filters. Its checked public example used identifier=REGION^1036, page=1 and search_radius=0.0. Copying or editing this example sends nothing; running it explicitly sends one request.

Halt on repeated pages, empty batches or moving totals

An identical row-batch hash, repeated currentPage, changed total or changed page size pauses the run. The saved raw candidate remains available while next_page stays unchanged. Investigate whether the request parameters, result ordering or inventory changed before deciding on a separately identified restart.

A batch hash catches exact repetition, not reordered duplicates. Only add row-level deduplication after validating an actual stable identifier in your returned records. Keep raw pages regardless. The code's stopped flag means the reported page-size boundary was reached under unchanged observed totals, not that every unique listing was collected. Do not request an extra page merely to manufacture proof of completeness.

APIs for similar apps can be requested via the contact page, and private source-code deployment is available.

Keep the sold-house-prices meta envelope out of this loop

house-prices-sold-house-prices returned meta while root-level currentPage, resultsPerPage and totalResultCount were absent. That successful response is not a broken commercial page. Give it a separate parser; no particular keys inside meta were established by the check. Resolve its required identifier and type together through house-prices-auto-complete, whose behavior still needs validation.

Other housing tasks also stay separate: use new-homes-for-sale for dedicated new-build discovery, and property-to-rent for documented budget, bedroom, furnishing and radius filters. Student search has its own detail route and search-derived listing ID, not a general-rental detail assumption. Those routes were not live-verified. Historical completed-sale prices must remain separate from listing asking prices; this catalog does not establish an exact property join.

Report observed coverage, not a complete market snapshot

Publish the operation, frozen filters, collection times, committed page range, raw row count, observed pagination values and stop reason with the export. Identify deduplicated counts separately if you have validated their identity rule.

Even unchanged totals cannot make sequential requests a stable market snapshot. A successful response does not certify current availability, permitted property use, comprehensive UK coverage, redistribution rights or a valuation. Describe what the run observed and retained, including any pause or unverified field assumptions, rather than calling the file a complete market dataset.