Skip to content

Commit 7cb9a2e

Browse files
kbatuigaspgellert
andauthored
[25.3] Client monitoring (#1389)
Co-authored-by: Gellért Peresztegi-Nagy <gellert.nagy@redpanda.com>
1 parent 2521c4a commit 7cb9a2e

6 files changed

Lines changed: 448 additions & 8 deletions

File tree

modules/ROOT/nav.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@
161161
*** xref:manage:cluster-maintenance/disk-utilization.adoc[]
162162
*** xref:manage:cluster-maintenance/manage-throughput.adoc[Manage Throughput]
163163
*** xref:manage:cluster-maintenance/compaction-settings.adoc[Compaction Settings]
164-
*** xref:manage:cluster-maintenance/configure-availability.adoc[]
164+
*** xref:manage:cluster-maintenance/configure-client-connections.adoc[]
165165
*** xref:manage:cluster-maintenance/partition-recovery.adoc[Forced Partition Recovery]
166166
*** xref:manage:cluster-maintenance/nodewise-partition-recovery.adoc[Node-wise Partition Recovery]
167167
** xref:manage:security/index.adoc[Security]

modules/get-started/pages/release-notes/redpanda.adoc

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ This topic includes new content added in version {page-component-version}. For a
77
* xref:redpanda-cloud:get-started:whats-new-cloud.adoc[]
88
* xref:redpanda-cloud:get-started:cloud-overview.adoc#redpanda-cloud-vs-self-managed-feature-compatibility[Redpanda Cloud vs Self-Managed feature compatibility]
99
10+
== Connected client monitoring
11+
12+
You can view details about Kafka client connections using the Admin API ListKafkaConnections endpoint. This allows you to view detailed information about about active client connections on a cluster, and identify and troubleshoot problematic clients. For more information, see xref:manage:monitoring.adoc#identify-high-throughput-clients[Monitor Redpanda].
13+
1014
== New Admin API endpoints
1115

1216
Redpanda v25.3 introduces a new API style for the Admin API, powered by https://connectrpc.com/docs/introduction[ConnectRPC]. New Redpanda features and operations in v25.3 are available as ConnectRPC services, allowing you to use autogenerated Protobuf clients in addition to using HTTP clients such as `curl`.
@@ -23,4 +27,4 @@ Existing Admin API endpoints from versions earlier than 25.3 remain supported, a
2327
Redpanda Schema Registry now supports an import mode that allows you to import existing schemas and retain their current IDs and version numbers. Import mode is useful when migrating from another schema registry.
2428

2529
Starting with this release, import mode must be used when importing schemas. Read-write mode no longer allows specifying a schema ID and version when registering a schema.
26-
See xref:manage:schema-reg/schema-reg-api.adoc#set-schema-registry-mode[Use the Schema Registry API] for more information.
30+
See xref:manage:schema-reg/schema-reg-api.adoc#set-schema-registry-mode[Use the Schema Registry API] for more information.

modules/manage/pages/cluster-maintenance/configure-availability.adoc renamed to modules/manage/pages/cluster-maintenance/configure-client-connections.adoc

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,36 @@
11
= Configure Client Connections
22
:description: Learn about guidelines for configuring client connections in Redpanda clusters for optimal availability.
33
:page-categories: Management, Networking
4+
:page-aliases: manage:cluster-maintenance:configure-availability.adoc
45
// tag::single-source[]
56

67
Optimize the availability of your clusters by configuring and tuning properties.
78

9+
// Don't display ListKafkaConnections in Cloud docs until support is added
10+
ifdef::env-cloud[]
11+
TIP: Before you configure connection limits or reconnection settings, start by gathering detailed data about your client connections. Use the xref:reference:public-metrics-reference.adoc#redpanda_rpc_active_connections[`redpanda_rpc_active_connections` metric] to view current Kafka client connections.
12+
endif::[]
13+
ifndef::env-cloud[]
14+
[TIP]
15+
====
16+
Before you configure connection limits or reconnection settings, start by gathering detailed data about your client connections.
17+
18+
* Internal metrics that follow the `vectorized_kafka_rpc_.*connect.*` naming pattern provide details on Kafka client connection activity. For example, xref:reference:internal-metrics-reference.adoc#vectorized_kafka_rpc_active_connections[`vectorized_kafka_rpc_active_connections`] reports the current number of active connections.
19+
* For Redpanda v25.3 and later, use the Admin API ListKafkaConnections endpoint to identify:
20+
+
21+
--
22+
** Which clients and applications are connected
23+
** Long-lived connections and long-running requests
24+
** Connections with no activity
25+
** Whether any clients are causing excessive load
26+
--
27+
+
28+
By reviewing connection details, you can make informed decisions about tuning connection limits and troubleshooting issues.
29+
30+
See also: link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections[ListKafkaConnections reference], xref:manage:monitoring.adoc#throughput[Monitor Redpanda]
31+
====
32+
endif::[]
33+
834
== Limit client connections
935

1036
To mitigate the risk of a client creating too many connections and using too many system resources, you can configure a Redpanda cluster to impose limits on the number of client connections that can be created.
@@ -70,7 +96,7 @@ NOTE: If you find a high load of unexpected connections from multiple IP address
7096
* Redpanda may modify this property during internal operations.
7197
* Availability incidents caused by misconfiguring this feature are excluded from the Redpanda Cloud SLA.
7298

73-
endif::env-cloud[]
99+
endif::[]
74100

75101
== Configure client reconnections
76102

@@ -87,6 +113,9 @@ Additionally, you can use Kafka properties to control message retry behavior. De
87113
* `retries`: Number of times a producer can retry sending a message before marking it as failed. The default value is 2147483647 for Kafka >= 2.1, or 0 for Kafka \<= 2.0.
88114
* `retry.backoff.ms`: Amount of time to wait before attempting to retry a failed request to a given topic partition. The default is 100 milliseconds.
89115

90-
See also: xref:develop:produce-data/configure-producers.adoc[Configure Producers]
116+
== See also
117+
118+
- xref:develop:produce-data/configure-producers.adoc[Configure Producers]
119+
- xref:manage:cluster-maintenance/manage-throughput.adoc[Manage Throughput]
91120

92121
// end::single-source[]

0 commit comments

Comments
 (0)