Use Entity Tags to Check for Schema Updates

The Open Data API operates a rate limiting system that is designed to prevent any one user or application from sending too many requests in a short period of time. By default, a single API key can submit up to 100 requests an hour.

To reduce the number of requests you need to make that count against your rate limit, all of the endpoints (except /table) support entity tags. These are designed to allow your application to maintain a local copy of certain information and then periodically check for updates without impacting your rate limit.

They are most useful for maintaining a local cache of the dataset catalogue/schema.

ETag Example - Schema Endpoint

When you send a query to the /schema endpoint, the response headers will contain an ETag value:

Your application should save a copy of the ETag value along with the response body data.

When you want to check whether the resource has changed, submit the same query but this time include the ETag value in an If-None-Match header:

If the Resource is Unchanged...

The API will return a 304 Not Modified status (with no body content in the response). This response does not count against your rate limit:

If the Resource has Changed...

The API returns a 200 OK response instead with the new version of the resource in the body. The response will also contain a new ETag value that you can use in subsequent requests to check whether this resource has changed.