Proxy Admin gRPC Interface Surface (M1: Online Client Query Module)

[zhaohai666]

Issue Title

[RIP-2] Proxy Admin gRPC Interface Surface (M1: Online Client Query Module)

---

Issue Description

This tracking issue covers the delivery of RIP-2 (Proxy Admin Interface Surface) in the apache/rocketmq main repository.

The initiative establishes an independent, standardized admin gRPC interface layer on the Proxy side, fully decoupled from the data-plane messaging pipeline. The first milestone (M1) delivers online client query capabilities as the foundational module, resolving the long-standing blind spot of gRPC client runtime observability. This interface surface will serve as the unified carrier for all subsequent Proxy-side admin tools, including configuration management, quota control, session management and diagnostic capabilities, which will be rolled out progressively based on community demand.

---

1. Background & Motivation

As RocketMQ 5.0 elevates Proxy to a first-class access layer with protocol parsing, authentication, rate limiting, routing and telemetry reporting capabilities, the lack of a standardized admin interface surface creates structural operational pain points:

1. Invisible runtime state: gRPC client connection status, subscription configuration, heartbeat liveness and consumption progress are only maintained internally within Proxy, with no standard query entry for control planes. This directly causes known issues 在一个正在运行的rocket集群中增加一台nameserver有什么需要注意的吗？会影响正常的读取和消费吗？ #381 (gRPC consumer latency displays -1) and [ISSUE #379] FIX admin subcommand consumeMessage can pull message with timestamp greater than now #380 (Pop client status false positive).
2. Fragmented and unstandardized admin capabilities: Operations such as configuration query, rate limit adjustment and session management either reuse data-plane RPC interfaces or rely on log troubleshooting, lacking unified service definitions, authentication systems and version contracts.
3. High ecosystem integration cost: Upper-layer ecosystems including control panels, AI agents and CLI tools cannot connect to Proxy-side O&M capabilities through standard interfaces, and can only infer runtime status via bypass metrics and logs, with insufficient accuracy and timeliness.

---

2. Core Goals

1. Establish an independent Proxy Admin gRPC interface surface, fully decoupled from data-plane interfaces in terms of port, authentication and service definition.
2. Deliver online client query capabilities in the first release, covering gRPC client list, details and group/topic dimension filtering, to support the RIP-1 CLIENT-01 final view.
3. Unify the ACL 2.0 authentication model with the independent proxy.admin.* resource hierarchy for fine-grained permission control.
4. Establish standardized interface contracts and version evolution rules to ensure backward compatibility, serving as the unified underlying entry for upper-layer control panels, AI tools and CLIs.
5. Guarantee high performance: paginated query P99 latency < 1s at million-connection scale, without blocking normal data-plane traffic.

---

3. Non-Goals

• Does not modify Broker or NameServer core logic; only targets Proxy-side admin capability construction.
• The initial release does not enforce cluster-level aggregated views; only single-Proxy local views are provided, and aggregation can be implemented by upper-layer control platforms.
• Does not replace existing mqadmin O&M commands; serves as a supplementary gRPC standardized admin channel.
• Planned modules (configuration, quota, session, diagnostics) are not mandatory for the initial release; they will be iterated version by version based on community needs.

---

4. Core Design Principles

1. Interface isolation: Admin interfaces are separated from data-plane message interfaces in terms of service, port and authentication to prevent operational actions from affecting data-plane stability.
2. Progressive iteration: Delivered module by module. Client query is delivered first, and capabilities such as configuration, quota and diagnosis are expanded on demand.
3. Least privilege: Independent proxy.admin.* resource system with separated read/write permissions, aligned with the ACL 2.0 authentication baseline.
4. Performance first: Query interfaces support filter pushdown, pagination and sampling. Full memory dump of data is prohibited to ensure stability at million-connection scale.
5. Stable contract: Proto definitions follow Apache RocketMQ interface compatibility specifications; new fields are backward compatible, and deprecated fields reserve a transition period.
6. Built-in observability: Admin interfaces natively expose OpenTelemetry metrics including call count, latency and error rate for monitoring the health of the admin channel.

---

5. Scope of Work

M1: Initial Delivery — Online Client Query

The first delivered module is the core unit of the Proxy Admin interface surface, which directly solves the industry pain point of invisible gRPC clients and supports the final view of RIP-1 CLIENT-01.

Core RPC Interfaces (4 in total)

All client data is sourced from the internal ClientManager module of Proxy, maintained based on information reported by the gRPC Telemetry stream.

RPC Method	Description	Input Parameters
ListClients	Paginated filtered client list with server-side filter pushdown	group, topic, clientIdPrefix, language, connectTimeStart, connectTimeEnd, pageNum, pageSize
DescribeClient	Full telemetry details for a single client instance	clientId
ListClientsByGroup	O&M-friendly shortcut query by consumer group	group, pageNum, pageSize
ListClientsByTopic	O&M-friendly shortcut query by topic	topic, pageNum, pageSize

ACL 2.0 Resource Model

Independent admin resource hierarchy completely separated from data-plane resources:

proxy.admin              (root resource)
└── proxy.admin.client   (client query module, read-only permission)

Performance & Implementation Requirements

• All filter conditions are pushed down to the ClientManager layer; full in-memory traversal after query is prohibited.
• Mandatory pagination with maximum pageSize of 100 to prevent large queries from causing memory pressure.
• Independent thread pool for admin interfaces, completely isolated from the data-plane thread pool.
• Weak consistency snapshot: client list is a near-real-time snapshot, allowing second-level delay to avoid locking that affects data-plane performance.
• Built-in OpenTelemetry metrics for all admin interfaces: QPS, P95/P99 latency, error rate.

M2+: Planned Iteration Modules

The following modules are the long-term evolution direction of the Proxy Admin interface surface. They are not mandatory for the initial release and will be rolled out version by version based on community demand and priority:

1. Runtime configuration query & hot update: DescribeProxyConfig / UpdateProxyConfig to query and modify runtime configurations without restart.
2. Rate limit quota visualization & adjustment: Query current rate limiting policies and trigger counts by namespace/topic/group dimensions; support controlled quota adjustment.
3. Cross-node session & connection control: KickClient / DisconnectChannel to proactively remove abnormal or lost clients, covering session synchronization semantics in multi-Proxy deployment.
4. Advanced semantic O&M diagnosis: Expose Pop lock holding view, ACK/NACK count, batch pull window status and NOT_CONSUME_YET trigger reasons for troubleshooting.
5. Routing & load balancing observation: View Proxy → Broker connection topology, current load distribution, fault removal status and geo-affinity routing effectiveness.

---

6. Acceptance Criteria

M1 Initial Release Acceptance

• Proxy Admin overall architecture and proto definitions pass community review, with clear interface positioning, naming conventions, permission system and evolution strategies.
• All 4 core client query RPCs are implemented, with complete functions, accurate filtering and normal pagination.
• Paginated query P99 latency < 1s on a single Proxy with million connections, no memory overflow risk.
• Fully integrated with ACL 2.0 authentication, with independent proxy.admin.client resource type and minimum permission policy documented.
• OpenTelemetry metrics for admin interfaces are enabled by default, meeting observability requirements.
• End-to-end joint debugging with RIP-1 Dashboard CLIENT-01 passes, and the gRPC client view is displayed normally.
• Complete bilingual interface documentation, authentication configuration examples and best practice manuals are provided.

Overall Architecture Acceptance

• Independent admin service framework is established, supporting subsequent module expansion without structural changes.
• Admin interfaces are completely isolated from the data plane in terms of port, thread pool and authentication.
• Interface contracts follow Apache RocketMQ compatibility specifications and support smooth version iteration.

Quality & Documentation

• Unit test coverage of core modules ≥ 85%.
• Integration tests cover all interfaces, authentication scenarios and exception handling.
• Performance benchmark report for million-connection scale is available.
• Bilingual (EN/CN) official documents are synchronized with the code release.

---

7. Related RIPs & Dependencies

• RIP-1 (Control Plane 5.0): RIP-1 CLIENT-01 final view depends on the M1 module of this RIP. Before the M1 module is released, CLIENT-01 uses telemetry metrics to build a transition view, and switches to the final view seamlessly after release. The two RIPs can be developed in parallel.
• RIP-3 (LLM Native Integration): RIP-3's rmq.client.* series of tools directly benefit from the M1 module of this RIP. Subsequent planned modules will also be synchronously exposed to LLM toolchains through RIP-3.
• Dependency rules:
  • RIP-2 itself does not depend on other RIPs and can be developed and released independently.
  • RIP-1 has a soft dependency on RIP-2 M1; the transition view is available without RIP-2.
  • RIP-3 indirectly depends on RIP-2 through RIP-1's abstraction layer.

---

8. Contribution Notes

• All code is delivered in the proxy module of the apache/rocketmq main repository.
• All submissions must follow Apache RocketMQ community coding standards, proto specification and contribution guidelines.
• Sub-tasks can be claimed by commenting on this issue. Please raise design questions and technical discussions in this thread or the corresponding dev mailing list.
• Planned modules can be proposed and promoted by interested contributors, and will be included in the RIP iteration roadmap after community review.

[zhaohai666]

#10597

[RockteMQ-AI]

Issue Evaluation

Category: type/enhancement | Status: Duplicate

This issue appears to be a duplicate of #10599, which contains the same content (RIP-2 Proxy Admin gRPC Interface Surface). Please follow the discussion there.

---

Automated evaluation by github-manager-bot

[RongtongJin]

Duplicate close