Skip to content

Commit e9f525a

Browse files
sungwyCopilot
authored and
gabeiglio
committed
Add Support for Custom AuthManager implementation (apache#2055)
<!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${1960} --> # Rationale for this change Expose a way for users to use a custom AuthManager when they define their REST Catalog. - [x] Add property parsing for custom AuthManager - [x] Add docs The properties should follow the pattern in: apache/iceberg#13209 # Are these changes tested? Unit and integration tests will be added # Are there any user-facing changes? New properties will be exposed to support adding a custom AuthManager to RestCatalog --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 38107a6 commit e9f525a

4 files changed

Lines changed: 230 additions & 2 deletions

File tree

‎mkdocs/docs/configuration.md‎

Lines changed: 74 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -359,7 +359,9 @@ catalog:
359359

360360
#### Authentication Options
361361

362-
##### OAuth2
362+
##### Legacy OAuth2
363+
364+
Legacy OAuth2 Properties will be removed in PyIceberg 1.0 in place of pluggable AuthManager properties below
363365

364366
| Key | Example | Description |
365367
| ------------------- | -------------------------------- | -------------------------------------------------------------------------------------------------- |
@@ -378,6 +380,77 @@ catalog:
378380
| rest.signing-region | us-east-1 | The region to use when SigV4 signing a request |
379381
| rest.signing-name | execute-api | The service signing name to use when SigV4 signing a request |
380382

383+
##### Pluggable Authentication via AuthManager
384+
385+
The RESTCatalog supports pluggable authentication via the `auth` configuration block. This allows you to specify which how the access token will be fetched and managed for use with the HTTP requests to the RESTCatalog server. The authentication method is selected by setting the `auth.type` property, and additional configuration can be provided as needed for each method.
386+
387+
###### Supported Authentication Types
388+
389+
- `noop`: No authentication (no Authorization header sent).
390+
- `basic`: HTTP Basic authentication.
391+
- `custom`: Custom authentication manager (requires `auth.impl`).
392+
393+
###### Configuration Properties
394+
395+
The `auth` block is structured as follows:
396+
397+
```yaml
398+
catalog:
399+
default:
400+
type: rest
401+
uri: http://rest-catalog/ws/
402+
auth:
403+
type: <auth_type>
404+
<auth_type>:
405+
# Type-specific configuration
406+
impl: <custom_class_path> # Only for custom auth
407+
```
408+
409+
###### Property Reference
410+
411+
| Property | Required | Description |
412+
|------------------|----------|-------------------------------------------------------------------------------------------------|
413+
| `auth.type` | Yes | The authentication type to use (`noop`, `basic`, or `custom`). |
414+
| `auth.impl` | Conditionally | The fully qualified class path for a custom AuthManager. Required if `auth.type` is `custom`. |
415+
| `auth.basic` | If type is `basic` | Block containing `username` and `password` for HTTP Basic authentication. |
416+
| `auth.custom` | If type is `custom` | Block containing configuration for the custom AuthManager. |
417+
418+
###### Examples
419+
420+
No Authentication:
421+
422+
```yaml
423+
auth:
424+
type: noop
425+
```
426+
427+
Basic Authentication:
428+
429+
```yaml
430+
auth:
431+
type: basic
432+
basic:
433+
username: myuser
434+
password: mypass
435+
```
436+
437+
Custom Authentication:
438+
439+
```yaml
440+
auth:
441+
type: custom
442+
impl: mypackage.module.MyAuthManager
443+
custom:
444+
property1: value1
445+
property2: value2
446+
```
447+
448+
###### Notes
449+
450+
- If `auth.type` is `custom`, you **must** specify `auth.impl` with the full class path to your custom AuthManager.
451+
- If `auth.type` is not `custom`, specifying `auth.impl` is not allowed.
452+
- The configuration block under each type (e.g., `basic`, `custom`) is passed as keyword arguments to the corresponding AuthManager.
453+
381454
<!-- markdown-link-check-enable-->
382455

383456
#### Common Integrations & Examples

‎pyiceberg/catalog/rest/__init__.py‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,8 @@ class IdentifierKind(Enum):
134134
SIGV4_SERVICE = "rest.signing-name"
135135
OAUTH2_SERVER_URI = "oauth2-server-uri"
136136
SNAPSHOT_LOADING_MODE = "snapshot-loading-mode"
137+
AUTH = "auth"
138+
CUSTOM = "custom"
137139

138140
NAMESPACE_SEPARATOR = b"\x1f".decode(UTF8)
139141

@@ -247,7 +249,23 @@ def _create_session(self) -> Session:
247249
elif ssl_client_cert := ssl_client.get(CERT):
248250
session.cert = ssl_client_cert
249251

250-
session.auth = AuthManagerAdapter(self._create_legacy_oauth2_auth_manager(session))
252+
if auth_config := self.properties.get(AUTH):
253+
auth_type = auth_config.get("type")
254+
if auth_type is None:
255+
raise ValueError("auth.type must be defined")
256+
auth_type_config = auth_config.get(auth_type, {})
257+
auth_impl = auth_config.get("impl")
258+
259+
if auth_type == CUSTOM and not auth_impl:
260+
raise ValueError("auth.impl must be specified when using custom auth.type")
261+
262+
if auth_type != CUSTOM and auth_impl:
263+
raise ValueError("auth.impl can only be specified when using custom auth.type")
264+
265+
session.auth = AuthManagerAdapter(AuthManagerFactory.create(auth_impl or auth_type, auth_type_config))
266+
else:
267+
session.auth = AuthManagerAdapter(self._create_legacy_oauth2_auth_manager(session))
268+
251269
# Set HTTP headers
252270
self._config_headers(session)
253271

‎pyiceberg/catalog/rest/auth.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,11 +42,15 @@ def auth_header(self) -> Optional[str]:
4242

4343

4444
class NoopAuthManager(AuthManager):
45+
"""Auth Manager implementation with no auth."""
46+
4547
def auth_header(self) -> Optional[str]:
4648
return None
4749

4850

4951
class BasicAuthManager(AuthManager):
52+
"""AuthManager implementation that supports basic password auth."""
53+
5054
def __init__(self, username: str, password: str):
5155
credentials = f"{username}:{password}"
5256
self._token = base64.b64encode(credentials.encode()).decode()
@@ -56,6 +60,12 @@ def auth_header(self) -> str:
5660

5761

5862
class LegacyOAuth2AuthManager(AuthManager):
63+
"""Legacy OAuth2 AuthManager implementation.
64+
65+
This class exists for backward compatibility, and will be removed in
66+
PyIceberg 1.0.0 in favor of OAuth2AuthManager.
67+
"""
68+
5969
_session: Session
6070
_auth_url: Optional[str]
6171
_token: Optional[str]

‎tests/catalog/test_rest.py‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
# specific language governing permissions and limitations
1616
# under the License.
1717
# pylint: disable=redefined-outer-name,unused-argument
18+
import base64
1819
import os
1920
from typing import Any, Callable, Dict, cast
2021
from unittest import mock
@@ -1519,6 +1520,132 @@ def test_request_session_with_ssl_client_cert() -> None:
15191520
assert "Could not find the TLS certificate file, invalid path: path_to_client_cert" in str(e.value)
15201521

15211522

1523+
def test_rest_catalog_with_basic_auth_type(rest_mock: Mocker) -> None:
1524+
# Given
1525+
rest_mock.get(
1526+
f"{TEST_URI}v1/config",
1527+
json={"defaults": {}, "overrides": {}},
1528+
status_code=200,
1529+
)
1530+
# Given
1531+
catalog_properties = {
1532+
"uri": TEST_URI,
1533+
"auth": {
1534+
"type": "basic",
1535+
"basic": {
1536+
"username": "one",
1537+
"password": "two",
1538+
},
1539+
},
1540+
}
1541+
catalog = RestCatalog("rest", **catalog_properties) # type: ignore
1542+
assert catalog.uri == TEST_URI
1543+
1544+
encoded_user_pass = base64.b64encode(b"one:two").decode()
1545+
expected_auth_header = f"Basic {encoded_user_pass}"
1546+
assert rest_mock.last_request.headers["Authorization"] == expected_auth_header
1547+
1548+
1549+
def test_rest_catalog_with_custom_auth_type() -> None:
1550+
# Given
1551+
catalog_properties = {
1552+
"uri": TEST_URI,
1553+
"auth": {
1554+
"type": "custom",
1555+
"impl": "dummy.nonexistent.package",
1556+
"custom": {
1557+
"property1": "one",
1558+
"property2": "two",
1559+
},
1560+
},
1561+
}
1562+
with pytest.raises(ValueError) as e:
1563+
# Missing namespace
1564+
RestCatalog("rest", **catalog_properties) # type: ignore
1565+
assert "Could not load AuthManager class for 'dummy.nonexistent.package'" in str(e.value)
1566+
1567+
1568+
def test_rest_catalog_with_custom_basic_auth_type(rest_mock: Mocker) -> None:
1569+
# Given
1570+
catalog_properties = {
1571+
"uri": TEST_URI,
1572+
"auth": {
1573+
"type": "custom",
1574+
"impl": "pyiceberg.catalog.rest.auth.BasicAuthManager",
1575+
"custom": {
1576+
"username": "one",
1577+
"password": "two",
1578+
},
1579+
},
1580+
}
1581+
rest_mock.get(
1582+
f"{TEST_URI}v1/config",
1583+
json={"defaults": {}, "overrides": {}},
1584+
status_code=200,
1585+
)
1586+
catalog = RestCatalog("rest", **catalog_properties) # type: ignore
1587+
assert catalog.uri == TEST_URI
1588+
1589+
encoded_user_pass = base64.b64encode(b"one:two").decode()
1590+
expected_auth_header = f"Basic {encoded_user_pass}"
1591+
assert rest_mock.last_request.headers["Authorization"] == expected_auth_header
1592+
1593+
1594+
def test_rest_catalog_with_custom_auth_type_no_impl() -> None:
1595+
# Given
1596+
catalog_properties = {
1597+
"uri": TEST_URI,
1598+
"auth": {
1599+
"type": "custom",
1600+
"custom": {
1601+
"property1": "one",
1602+
"property2": "two",
1603+
},
1604+
},
1605+
}
1606+
with pytest.raises(ValueError) as e:
1607+
# Missing namespace
1608+
RestCatalog("rest", **catalog_properties) # type: ignore
1609+
assert "auth.impl must be specified when using custom auth.type" in str(e.value)
1610+
1611+
1612+
def test_rest_catalog_with_non_custom_auth_type_impl() -> None:
1613+
# Given
1614+
catalog_properties = {
1615+
"uri": TEST_URI,
1616+
"auth": {
1617+
"type": "basic",
1618+
"impl": "basic.package",
1619+
"basic": {
1620+
"username": "one",
1621+
"password": "two",
1622+
},
1623+
},
1624+
}
1625+
with pytest.raises(ValueError) as e:
1626+
# Missing namespace
1627+
RestCatalog("rest", **catalog_properties) # type: ignore
1628+
assert "auth.impl can only be specified when using custom auth.type" in str(e.value)
1629+
1630+
1631+
def test_rest_catalog_with_unsupported_auth_type() -> None:
1632+
# Given
1633+
catalog_properties = {
1634+
"uri": TEST_URI,
1635+
"auth": {
1636+
"type": "unsupported",
1637+
"unsupported": {
1638+
"property1": "one",
1639+
"property2": "two",
1640+
},
1641+
},
1642+
}
1643+
with pytest.raises(ValueError) as e:
1644+
# Missing namespace
1645+
RestCatalog("rest", **catalog_properties) # type: ignore
1646+
assert "Could not load AuthManager class for 'unsupported'" in str(e.value)
1647+
1648+
15221649
EXAMPLE_ENV = {"PYICEBERG_CATALOG__PRODUCTION__URI": TEST_URI}
15231650

15241651

0 commit comments

Comments
 (0)