Skip to content

Commit 32be47d

Browse files
micheleRPFeediver1
authored andcommitted
DOC-1720 Customer-Incident-Delete Records (#1388)
Co-authored-by: Joyce Fee <102751339+Feediver1@users.noreply.github.com>
1 parent 02dbb28 commit 32be47d

3 files changed

Lines changed: 18 additions & 7 deletions

File tree

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,14 @@
11
== Delete records from a topic
22

3-
Redpanda lets you delete data from the beginning of a partition up to a specific event, also known as offset. The offset represents the true creation time of the event, not the time when it was stored by Redpanda. Deleting records frees up disk space, which is especially helpful if your producers are pushing more data than you anticipated in your retention plan. Do this when you know that all consumers have read up to that given offset, and the data is no longer needed.
3+
Redpanda allows you to delete data from the beginning of a partition up to a specific offset (a monotonically increasing sequence number for records in a partition). Deleting records frees up disk space, which is especially helpful if your producers are pushing more data than anticipated in your retention plan. Delete records when you know that all consumers have read up to that given offset, and the data is no longer needed.
44

5-
There are different ways to delete records from a topic, including using the xref:reference:rpk/rpk-topic/rpk-topic-trim-prefix.adoc[`rpk topic trim-prefix`] command or using the `DeleteRecords` Kafka API with Kafka clients.
5+
There are different ways to delete records from a topic, including using the xref:reference:rpk/rpk-topic/rpk-topic-trim-prefix.adoc[`rpk topic trim-prefix`] command, using the `DeleteRecords` Kafka API with Kafka clients, or using {ui}.
66

77
[NOTE]
88
====
99
- To delete records, `cleanup.policy` must be set to `delete` or `compact,delete`.
1010
- Object storage is deleted asynchronously. After messages are deleted, the partition's start offset will have advanced, but garbage collection of deleted segments may not be complete.
1111
- Similar to Kafka, after deleting records, local storage and object storage may still contain data for deleted offsets. (Redpanda does not truncate segments. Instead, it bumps the start offset, then it attempts to delete as many whole segments as possible.) Data before the new start offset is not visible to clients but could be read by someone with access to the local disk of a Redpanda node.
12-
====
12+
====
13+
14+
include::shared:partial$warning-delete-records.adoc[]

modules/reference/pages/rpk/rpk-topic/rpk-topic-trim-prefix.adoc

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,12 @@ Trim records from topics
55

66
This command allows you to trim records from topics, where Redpanda
77
sets the LogStartOffset for partitions to the requested offset. All segments
8-
whose base offset is less then the requested offset are deleted, and any records
8+
whose base offset is less than the requested offset are deleted, and any records
99
within the segment before the requested offset can no longer be read.
1010

1111
The `--offset/-o` flag allows you to indicate which index you want to set the
1212
partition's low watermark (start offset) to. It can be a single integer value
13-
denoting the offset, or a timestamp if you prefix the offset with an '@'. You may
14-
select which partition you want to trim the offset from with the `--partitions/-p`
15-
flag.
13+
denoting the offset, or it can be a timestamp if you prefix the offset with an '@'. You can select which partition to trim the offset from using the `--partitions/-p` flag.
1614

1715
The `--from-file` option allows to trim the offsets specified in a text file with
1816
the following format:
@@ -25,6 +23,8 @@ the following format:
2523

2624
or the equivalent keyed JSON/YAML file.
2725

26+
include::shared:partial$warning-delete-records.adoc[]
27+
2828
== Examples
2929

3030
* Trim records in 'foo' topic to offset 120 in partition 1:
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
[WARNING]
2+
====
3+
When you delete records from a topic with a timestamp, Redpanda advances the partition start offset to the first record whose timestamp is after the threshold. If record timestamps are not in order with respect to offsets, this may result in unintended deletion of data. Before using a timestamp, verify that timestamps increase in the same order as offsets in the topic to avoid accidental data loss. For example:
4+
5+
[,bash]
6+
----
7+
rpk topic consume <topic> -n 50 --format '%o %d{go[2006-01-02T15:04:05Z07:00]} %k %v'
8+
----
9+
====

0 commit comments

Comments
 (0)