fix(rate-limiter): return Iceberg ErrorResponse JSON body on HTTP 429

[travis-bowen]

Motivation

RateLimiterFilter previously returned HTTP 429 with an empty body and no Content-Type header:

ctx.abortWith(Response.status(Response.Status.TOO_MANY_REQUESTS).build());

Iceberg REST clients (pyiceberg, Spark, and any third-party caller) cannot meaningfully parse this. The Iceberg Java client's DefaultErrorHandler.parseResponse() attempts ErrorResponseParser.fromJson("") on the empty body, catches the JSON parse failure, and falls back to building an ErrorResponse with message = "". Since there is no case 429: in any ErrorHandlers switch, the result is always:

RESTException("Unable to process: ")

— an opaque error with no actionable context for the caller.

Changes

Add an Iceberg-compatible ErrorResponse JSON body to the 429 response.

ctx.abortWith(
    Response.status(Response.Status.TOO_MANY_REQUESTS)
        .type(MediaType.APPLICATION_JSON_TYPE)
        .entity(
            ErrorResponse.builder()
                .responseCode(Response.Status.TOO_MANY_REQUESTS.getStatusCode())
                .withType("TooManyRequestsException")
                .withMessage("Rate exceeded")
                .build())
        .build());

Ecosystem Context

The Iceberg REST OpenAPI spec does not define a 429 response schema — implementations are on their own. Surveying the major catalog implementations:

Implementation	Body on 429?	Follows Iceberg ErrorModel?
AWS Glue	Yes	No (AWS {"__type": "ThrottlingException", ...} format)
Project Nessie	Yes	No (Nessie {"status": 429, "errorCode": "TOO_MANY_REQUESTS", ...} format)
Databricks Unity Catalog	Yes	No (Databricks {"error_code": "RESOURCE_EXHAUSTED", ...} format)
Apache Polaris (this fix)	Yes	Yes

All other catalogs return some JSON, but none follow the Iceberg ErrorModel schema. With this fix, Polaris becomes the most spec-consistent implementation — clients using ErrorResponseParser will be able to round-trip the 429 body correctly.

Compatibility

• Existing clients that ignore the 429 body are unaffected.
• Clients that previously hit the empty-body parse failure will now receive a well-formed ErrorResponse.
• No change to status code or any other behavior.

Tests

Added testRateLimitedResponseHasIcebergErrorBody — a full integration test that performs a live HTTP request against a rate-limited endpoint, reads the response body, and round-trips it through ErrorResponseParser.fromJson(...) to prove Iceberg client compatibility. Asserts Content-Type: application/json, code == 429, type == "RateLimitExceededException", and message contains "Rate exceeded".

Fixes #4402

---

This fix was developed with AI assistance (Claude). The author has reviewed the change end-to-end and takes full responsibility for correctness and ASF licensing compliance.

[adnanhemani]

Seems like a reasonable change to me.

[dimas-b]

Thanks for your contribution, @travis-bowen !

Technically, changing from a no-body response to a JSON response is a non-breaking change, I think.

[dimas-b]

This response form is specific to the Iceberg REST API, but the filter covers all Polaris REST API, IIRC... I'm not sure using this response form is relevant to all of them.

... especially considering that changing the response payload after we release with some payload is going to be very difficult.

Is there a way to restrict this to IRC endpoints?

[travis-bowen]

So, it should be possible with changes in the PreMatching filter.
However, it seems that we already have expectations in polaris of ErrorResponse being built even for management APIs.

Here's what I found.
https://github.com/apache/polaris/blob/main/runtime/service/src/main/java/org/apache/polaris/service/exception/PolarisExceptionMapper.java#L44-L83

What do you think about leaving this to be universal? If not I can try the filtering to Iceberg?

[dimas-b]

🤦 TBH I'm not sure that applying PolarisExceptionMapper to non-IRC APIs was intentional 😅

[dimas-b]

I did a quick wire capture of a duplicate "create catalog" response (Management API) and got:

    {
        "error": {
            "message": "Cannot create Catalog polaris. Catalog already exists or resolution failed",
            "type": "AlreadyExistsException",
            "code": 409
        }
    }

[dimas-b]

So, that does look like an IRC error response payload... while I do not agree with this approach for Polaris' own API, this is the state of the project ATM.

Therefore, it makes sense to use the same payload structure for 429 responses too.

Thanks for the pointers, @travis-bowen 👍

[dimas-b]

I'll add this to the Community sync agenda on May 14... will merge after the call unless other reviewers raise concerns.

[snazy]

Slight correction: It's Project Nessie not Apache Nessie (PR description)