Core concepts

Pagination

Cursors, not offsets. On a feed where records arrive while you are reading it, an offset quietly skips records and a cursor does not.

Why not page numbers

Offset pagination assumes the result set holds still. Emergency data does not. If three new incidents arrive between your request for page 1 and your request for page 2, everything shifts down by three and you never see the three that moved across the boundary. Nothing errors. You simply have a gap.

A cursor points at a position in the data rather than a count from the start, so new arrivals cannot shift it.

How to page

Follow links.next until it is null or absent. It is the next page as a path on the same host: your request with only the cursor changed, so every filter and the limit carry over.

Stop on links.next, not on a short or empty page. A full page can be the last one, and a /declarations page can hold fewer records than limit, or none, while more pages follow.

Each list names its cursor in its own field:

Endpoint	Next page	Cursor field	Last page
`/incidents`	`links.next` (its cursor rides as `after`)	`meta.next_cursor`	Both absent
`/incidents/history`	`links.next`	`meta.next_cursor`	`links.next` absent, `meta.next_cursor` null
`/declarations`	`links.next`	None: follow `links.next`	`links.next` null
`/publications`	`links.next`	None: follow `links.next`	`links.next` null
`/gauges`, `/gauges/{id}/readings`	No `links.next`: send `meta.nextCursor` as `cursor` with the same filters and limit	`meta.nextCursor`	`meta.nextCursor` absent, `meta.hasMore` false
`/mobile-sites`	`offset` and `limit`: the ACCC's yearly release, not a live feed	None	`hasMore` false

Request

# first page
curl "https://dataquoll.io/api/v1/incidents?state=nsw&limit=100" \
  -H "Authorization: Bearer YOUR_API_KEY"

# next page: the path in links.next, on the same host
curl "https://dataquoll.io/api/v1/incidents?state=nsw&limit=100&after=eyJ1cGRhdGVkX2F0IjoiMjAyNi0wNy0yNlQwMDowMDowMFoiLCJpZCI6Im5zdy1yZnMtMTIzIn0" \
  -H "Authorization: Bearer YOUR_API_KEY"

limit: Records per page. Defaults to 100, maximum 500. A value above the maximum is a 400, not a silent clamp, so you never think you asked for more than you received.
cursor: An opaque token encoding a position. Treat it as opaque: its contents are an implementation detail and a malformed cursor returns 400. A cursor marks a position, not a query: if you send one yourself, send the same filters and limit as the request that returned it.
links.next: The next page as a path: this request with only the cursor changed, so every filter and the limit carry over. Absent or null on the last page.

One correct loop

Stop when links.next is null or absent. The same loop pages /incidents/history, /declarations and /publications: read their records from features, declarations and publications.

Python

from urllib.parse import urljoin

import requests

url = "https://dataquoll.io/api/v1/incidents?state=nsw&limit=100"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
incidents = []

while url:
    r = requests.get(url, headers=headers, timeout=30)
    r.raise_for_status()
    body = r.json()
    incidents.extend(body["features"])

    # links.next is a path on the same host, absent or null on the last page
    next_path = (body.get("links") or {}).get("next")
    url = urljoin(url, next_path) if next_path else None

print(f"{len(incidents)} incidents")