You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: doc/modules/cassandra/pages/managing/operating/compaction/overview.adoc
+29-42Lines changed: 29 additions & 42 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,18 +2,18 @@
2
2
3
3
== What is compaction?
4
4
5
-
Data in {cassandra} is created in xref:cassandra:architecture/storage-engine.adoc#memtables[memtables].
5
+
Data in {cassandra} is created in xref:cassandra:architecture/storage-engine.adoc#memtables[memtables].
6
6
Once a memory threshold is reached, to free up memory again, the data is written to an xref:cassandra:architecture/storage-engine.adoc#SSTables[SSTable], an https://cassandra.apache.org/_/glossary.html#immutable[immutable] file residing on disk.
7
7
8
-
Because SSTables are immutable, when data is updated or deleted, the old data is not overwritten with inserts or updates, or removed from the SSTable.
9
-
Instead, a new SSTable is created with the updated data with a new timestamp, and the old SSTable is marked for deletion.
8
+
Because SSTables are immutable, when data is updated or deleted, the old data is not overwritten with inserts or updates, or removed from the SSTable.
9
+
Instead, a new SSTable is created with the updated data with a new timestamp, and the old SSTable is marked for deletion.
10
10
The piece of deleted data is known as a https://cassandra.apache.org/_/glossary.html#tombstone[tombstone].
11
11
12
-
Over time, Cassandra may write many versions of a row in different SSTables.
13
-
Each version may have a unique set of columns stored with a different timestamp.
12
+
Over time, Cassandra may write many versions of a row in different SSTables.
13
+
Each version may have a unique set of columns stored with a different timestamp.
14
14
As SSTables accumulate, the distribution of data can require accessing more and more SSTables to retrieve a complete row.
15
15
16
-
To keep the database healthy, Cassandra periodically merges SSTables and discards old data.
16
+
To keep the database healthy, Cassandra periodically merges SSTables and discards old data.
17
17
This process is called https://cassandra.apache.org/_/glossary.html#compaction[compaction].
18
18
19
19
== Why must compaction be run?
@@ -26,22 +26,22 @@ Deleting, updating, or expiring data are all valid triggers for compaction.
26
26
== What does compaction accomplish?
27
27
28
28
Two important factors accomplished by compaction are performance improvement and disk space reclamation.
29
-
If SSTables have duplicate data that must be read, read operations are slower.
29
+
If SSTables have duplicate data that must be read, read operations are slower.
30
30
Once tombstones and duplicates are removed, read operations are faster.
31
31
SSTables use disk space, and reducing the size of SSTables through compaction frees up disk space.
32
32
33
33
== How does compaction work?
34
34
35
-
Compaction works on a collection of SSTables.
36
-
From these SSTables, compaction collects all versions of each unique row and assembles one complete row, using the most up-to-date version (by timestamp) of each of the row's columns.
37
-
The merge process is performant, because rows are sorted by partition key within each SSTable, and the merge process does not use random I/O.
38
-
The new versions of each row is written to a new SSTable.
35
+
Compaction works on a collection of SSTables.
36
+
From these SSTables, compaction collects all versions of each unique row and assembles one complete row, using the most up-to-date version (by timestamp) of each of the row's columns.
37
+
The merge process is performant, because rows are sorted by partition key within each SSTable, and the merge process does not use random I/O.
38
+
The new versions of each row is written to a new SSTable.
39
39
The old versions, along with any rows that are ready for deletion, are left in the old SSTables, and are deleted as soon as pending reads are completed.
40
40
41
41
== Types of compaction
42
42
43
43
The concept of compaction is used for different kinds of operations in
44
-
{cassandra}, the common thing about these operations is that it takes one
44
+
{cassandra}, the common thing about these operations is that they take one
45
45
or more SSTables, merges, and outputs new SSTables. The types of compactions are:
46
46
47
47
Minor compaction::
@@ -56,11 +56,11 @@ A major compaction is triggered when a user executes a compaction over all SSTab
56
56
User defined compaction::
57
57
Similar to a major compaction, a user-defined compaction executes when a user triggers a compaction on a given set of SSTables.
58
58
Scrub::
59
-
A scrub triggers a compaction to try to fix any broken SSTables.
59
+
A scrub triggers a compaction to try to fix any broken SSTables.
60
60
This can actually remove valid data if that data is corrupted.
61
61
If that happens you will need to run a full repair on the node.
62
62
UpgradeSSTables::
63
-
A compaction occurs when you upgrade SSTables to the latest version.
63
+
A compaction occurs when you upgrade SSTables to the latest version.
64
64
Run this after upgrading to a new major version.
65
65
Cleanup::
66
66
Compaction executes to remove any ranges that a node no longer owns.
@@ -71,8 +71,8 @@ Anticompaction::
71
71
After repair, the ranges that were actually repaired are split out of the SSTables that existed when repair started. This type of compaction rewrites SSTables to accomplish this task.
72
72
Sub range compaction::
73
73
It is possible to only compact a given sub range - this action is useful if you know a token that has been misbehaving - either gathering many updates or many deletes.
74
-
The command `nodetool compact -st x -et y` will pick all SSTables containing the range between x and y and issue a compaction for those SSTables.
75
-
For Size Tiered Compaction Strategy, this will most likely include all SSTables, but with Leveled Compaction Strategy, it can issue the compaction for a subset of the SSTables.
74
+
The command `nodetool compact -st x -et y` will pick all SSTables containing the range between x and y and issue a compaction for those SSTables.
75
+
For Size Tiered Compaction Strategy, this will most likely include all SSTables, but with Leveled Compaction Strategy, it can issue the compaction for a subset of the SSTables.
76
76
With LCS the resulting SSTable will end up in L0.
77
77
78
78
== Strategies
@@ -82,14 +82,14 @@ Picking the right compaction strategy for your workload will ensure the best per
Since it might not be possible to drop any tombstones when doing a single SSTable compaction we need to make sure that one SSTable is not constantly getting recompacted - this option states how often we should try for a given SSTable.
180
167
`log_all` (default: false)::
181
-
New detailed compaction logging, see `below <detailed-compaction-logging>`.
168
+
New detailed compaction logging, see <<detailed-compaction-logging, below>>.
The single SSTable compaction has quite strict checks for whether it should be started, this option disables those checks and for some use cases this might be needed.
170
+
The single SSTable compaction has quite strict checks for whether it should be started, this option disables those checks and for some use cases this might be needed.
184
171
Note that this does not change anything for the actual compaction, tombstones are only dropped if it is safe to do so - it might just rewrite an SSTable without being able to drop any tombstones.
Option to enable the extra safety of making sure that tombstones are only dropped if the data has been repaired.
187
174
`min_threshold` (default: 4)::
188
175
Lower limit of number of SSTables before a compaction is triggered.
@@ -195,7 +182,7 @@ Further, see the section on each strategy for specific additional options.
195
182
196
183
== Compaction nodetool commands
197
184
198
-
The `nodetool <nodetool>` utility provides a number of commands related to compaction:
185
+
The `nodetool` utility provides a number of commands related to compaction:
199
186
200
187
`enableautocompaction`::
201
188
Enable compaction.
@@ -212,7 +199,7 @@ Set the min/max SSTable count for when to trigger compaction, defaults to 4/32.
212
199
213
200
== Switching the compaction strategy and options using JMX
214
201
215
-
It is possible to switch compaction strategies and its options on just a single node using JMX, this is a great way to experiment with settings without affecting the whole cluster.
202
+
It is possible to switch compaction strategies and its options on just a single node using JMX, this is a great way to experiment with settings without affecting the whole cluster.
0 commit comments