Skip to content

[#11279] docs: Add built-in IDP operator how-to and align design doc#11281

Open
lasdf1234 wants to merge 43 commits into
apache:mainfrom
lasdf1234:idp-doc
Open

[#11279] docs: Add built-in IDP operator how-to and align design doc#11281
lasdf1234 wants to merge 43 commits into
apache:mainfrom
lasdf1234:idp-doc

Conversation

@lasdf1234
Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

  • Add docs/how-to-use-built-in-idp.md, an operator guide for the built-in IDP (idp-basic plugin): prerequisites (REST extension package, schema upgrade, service admin password init), configuration, management REST APIs with curl examples, password rules, and access-control integration.
  • Update design-docs/gravitino-local-authentication.md to align with the how-to (service admin initialization, configuration, administrative APIs; remove outdated or inaccurate prose).
  • Link the guide from docs/index.md and docs/security/security.md.

Why are the changes needed?

Operators need a single how-to and an aligned design document as built-in IdP capabilities land across multiple PRs under epic #10959.

Fix: #11279

Does this PR introduce any user-facing change?

Documentation only.

How was this patch tested?

  • Manual review of markdown content and cross-links.
  • N/A (documentation-only change; no OpenAPI spec updates in this PR).

Made with Cursor

lasdf1234 and others added 30 commits May 28, 2026 17:11
Document /api/idp management APIs from idp-basic implementation,
clarify unimplemented Basic auth and startup init, and fix group
membership paths in the local authentication design doc.

Co-authored-by: Cursor <cursoragent@cursor.com>
The default entity store is already relational and IdP schema setup
is covered by the following prerequisite item.

Co-authored-by: Cursor <cursoragent@cursor.com>
Avoid suggesting a combined authenticator list that is not the
intended production configuration.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Reorganize the guide to match other Gravitino how-to docs:
Introduction, Prerequisites, Configuration, Operations, access
control integration, and Further reading.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Explain enabling authorization, configuring serviceAdmins, and
calling /api/idp as an authenticated service admin before API examples.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop Plugin JAR copy steps from prerequisites and document service
admin configuration, authentication, and optional IDP password setup.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Document GRAVITINO_INITIAL_ADMIN_PASSWORD as mandatory startup setup
and remove simple-mode workarounds for calling /api/idp management APIs.

Co-authored-by: Cursor <cursoragent@cursor.com>
Describe service admin initialization and basic authentication as
normative behavior aligned with the IdP how-to guide.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Usernames come from serviceAdmins; the env var holds only the password,
not a JSON username:password array.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Update slug and doc site links to match the new filename.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Drop gravitino.authenticators=basic and "when basic is enabled"
wording; document built-in IDP via rest.extensionPackages instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
Remove duplicate sections, incorrect AuthenticatorFactory claims, and awkward wording in the how-to and design doc.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Copilot AI review requested due to automatic review settings May 28, 2026 09:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds documentation for the new built-in IDP (local authentication) feature and updates the existing local authentication design doc to reflect the implemented configuration model (extension packages + service admin initialization) rather than the earlier basic authenticator approach.

Changes:

  • New user-facing guide docs/how-to-use-built-in-idp.md describing prerequisites, configuration, and /api/idp/* operations.
  • Updates design-docs/gravitino-local-authentication.md to remove the gravitino.authenticators=basic model and use gravitino.server.rest.extensionPackages plus GRAVITINO_INITIAL_ADMIN_PASSWORD as a single password string.
  • Links the new guide from docs/index.md and docs/security/security.md.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
docs/security/security.md Adds a link to the new Built-in IDP guide in the security index.
docs/index.md Adds the Built-in IDP entry under the Security section.
docs/how-to-use-built-in-idp.md New how-to guide covering setup and /api/idp/* operations.
design-docs/gravitino-local-authentication.md Rewrites design to match the implemented IDP configuration and admin init flow.

Comment thread docs/security/how-to-use-built-in-idp.md Outdated
Comment thread docs/security/how-to-use-built-in-idp.md Outdated
Comment thread design-docs/gravitino-local-authentication.md
Comment thread docs/security/how-to-use-built-in-idp.md
Comment thread docs/security/how-to-use-built-in-idp.md Outdated
Comment thread design-docs/gravitino-local-authentication.md
Relocate how-to-use-built-in-idp.md next to other security guides, update
slug and cross-links in index.md and security.md.

Co-authored-by: Cursor <cursoragent@cursor.com>

This guide describes how to enable and operate the management APIs in `plugins:idp-basic`. For
design background, see
[Design of local authentication support](../design-docs/gravitino-local-authentication.md). For
Copy link
Copy Markdown
Contributor

@roryqi roryqi May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could u avoid telling the implementation and design details?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got remove design doc.

gravitino.server.rest.extensionPackages = org.apache.gravitino.idp.web.rest.feature
```

2. **IDP database tables** — Run the appropriate upgrade script under `${GRAVITINO_HOME}/scripts/`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this step?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it has been deleted.

Comment thread docs/security/how-to-use-built-in-idp.md Outdated
Comment thread docs/security/how-to-use-built-in-idp.md Outdated

---

## Use with Gravitino access control
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got removed.

Comment thread docs/security/security.md Outdated
Comment thread docs/security/how-to-use-built-in-idp.md Outdated
Comment thread docs/how-to-use-built-in-idp.md Outdated

| Configuration item | Description | Example |
|-----------------------------------------|------------------------------------------|---------|
| `gravitino.authorization.enable` | Enable Gravitino authorization | `true` |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not required.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got resolved.

@roryqi roryqi changed the title [#11279] [MINOR] docs: Add built-in IDP operator how-to and align design doc [#11279] docs: Add built-in IDP operator how-to and align design doc May 28, 2026
lasdf1234 and others added 4 commits May 28, 2026 17:33
Remove the relational upgrade-script prerequisite and all
gravitino.authorization.enable settings from the operator guide.

Co-authored-by: Cursor <cursoragent@cursor.com>
Remove the setting from the initialization example and built-in IDP
configuration table to match the operator how-to guide.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread docs/security/how-to-use-built-in-idp.md Outdated
Comment thread docs/security/how-to-use-built-in-idp.md
Comment thread docs/security/how-to-use-built-in-idp.md Outdated
@roryqi
Copy link
Copy Markdown
Contributor

roryqi commented May 28, 2026

Could u check all the links?


For full request and response definitions, see the [Built-in IDP OpenAPI](../open-api/idp/openapi.yaml).

---
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should add complete user process?

  1. Add a metalake
  2. Add the user
  3. Add a Iceberg catalog
  4. How IRC clients to access

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will mention an improvement later.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 28, 2026

Code Coverage Report

Overall Project 66.55% 🟢
Files changed No Java source files changed -

Module Coverage
aliyun 1.72% 🔴
api 46.82% 🟢
authorization-common 85.96% 🟢
aws 3.66% 🔴
azure 2.47% 🔴
catalog-common 10.04% 🔴
catalog-fileset 80.33% 🟢
catalog-glue 66.08% 🟢
catalog-hive 79.55% 🟢
catalog-jdbc-clickhouse 80.02% 🟢
catalog-jdbc-common 45.31% 🟢
catalog-jdbc-doris 80.28% 🟢
catalog-jdbc-hologres 54.03% 🟢
catalog-jdbc-mysql 79.23% 🟢
catalog-jdbc-oceanbase 78.38% 🟢
catalog-jdbc-postgresql 82.29% 🟢
catalog-jdbc-starrocks 78.51% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 44.89% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 85.65% 🟢
catalog-lakehouse-paimon 79.29% 🟢
catalog-model 77.72% 🟢
cli 44.51% 🟢
client-java 77.91% 🟢
common 49.99% 🟢
core 82.37% 🟢
filesystem-hadoop3 76.97% 🟢
flink 0.0% 🔴
flink-common 41.37% 🟢
flink-runtime 0.0% 🔴
gcp 14.12% 🔴
hadoop-common 10.39% 🔴
hive-metastore-common 53.26% 🟢
iceberg-common 54.98% 🟢
iceberg-rest-server 69.99% 🟢
idp-basic 85.99% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 20.83% 🔴
lance-rest-server 60.27% 🟢
lineage 53.02% 🟢
optimizer 82.95% 🟢
optimizer-api 21.95% 🔴
server 85.1% 🟢
server-common 72.85% 🟢
spark 32.79% 🔴
spark-common 39.61% 🔴
trino-connector 39.44% 🔴

Add navigation entries so operators can discover the built-in IDP
operator guide from the docs home page and security capabilities list.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread docs/security/security.md Outdated

### [Authentication](how-to-authenticate.md)

### [Built-in IDP (local authentication)](how-to-use-built-in-idp.md)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove local authentication.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got resolved.

Use "built-in IDP" consistently in the how-to title and navigation
links on the docs index and security overview pages.

Co-authored-by: Cursor <cursoragent@cursor.com>
roryqi
roryqi previously approved these changes May 28, 2026
Copy link
Copy Markdown
Contributor

@roryqi roryqi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Wait for CI.

@mchades mchades requested a review from jerryshao May 28, 2026 15:22
Resolve docs/index.md conflict: keep main's authentication list (simple,
basic, OAuth, Kerberos) and retain the built-in IDP how-to link.

Co-authored-by: Cursor <cursoragent@cursor.com>
lasdf1234 and others added 3 commits May 28, 2026 23:54
Clarify serviceAdmins table wording, use "Basic authentication" without
"HTTP" in prose, and say "Start Gravitino" in prerequisites.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Drop "usernames" from serviceAdmins descriptions and pad markdown table columns in the built-in IDP docs.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Subtask] Document built-in IdP operator guide and align design doc

3 participants