Skip to content

Fix index compatibility table header #7186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yetanothertw merged 10 commits into from
Jul 8, 2026
+57 −9
Merged

Fix index compatibility table header #7186

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
b7b24cc
Fix index compatibility table header
yetanothertw Jul 2, 2026
ffccebe
Trying another option + table intro paragraph tweaks
yetanothertw Jul 2, 2026
d2496c6
Adding a draft trying to explain Kibana compatibility
yetanothertw Jul 3, 2026
128d035
Some tweaks based on some SME suggestions/comments
yetanothertw Jul 6, 2026
5d3fdc5
Linter suggestions
yetanothertw Jul 6, 2026
fab1343
Update table labels for clarity
yetanothertw Jul 7, 2026
ab1cde1
Apply feedback from Edu's review
yetanothertw Jul 8, 2026
bcc56cc
Add description of what we mean by index compatibility
yetanothertw Jul 8, 2026
fd39feb
Apply suggestions from code review
yetanothertw Jul 8, 2026
5fdd1ac
Merge branch 'main' into index-compatibility-table
yetanothertw Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 57 additions & 9 deletions deploy-manage/tools/snapshot-and-restore.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,22 +158,38 @@

Any index you restore from a snapshot must also be compatible with the current cluster’s version. If you try to restore an index created in an incompatible version, the restore attempt fails.

@eedugon eedugon Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this line, in the "INDEX COMPATIBILITY" section, I'd suggest to add some extra scope of what we mean exactly with compatibility, and that compatibility doesn't mean restored data will be valid, as that depends on the data owner expectations (such as Kibana data, but this could happen with other stack features data if they write in their own indices).

Let me explain:

When it comes to snapshot compatibility or index compatibility we talk about the capacity of Elasticsearch of restoring indices from snapshots created in older versions of Elasticsearch. Compatibility means that Elasticsearch is capable of restoring the original index and data (maybe as read-only in case of archive mode), but that does not mean that the data will be VALID for the application using it in all cases <-- that's the important thing to clarify.

For user data, compatibility will almost always imply that the restored data will be valid to the user, but for Kibana or stack features data this validity that cannot be guaranteed, as the application could expect different data format in different stack versions (such as Kibana). Because of that, users shouldn't use snapshot and restore as a way to "upgrade" except if they are focused only on the data they own.

I love the sentence you included:

Restoring an old snapshot directly to a 9.x cluster is not a shortcut around the normal upgrade paths. It is recommended you upgrade to the latest compatible minor release before a major upgrade, even when you use snapshots to move data between clusters.

Maybe that sentence could be included as an important note in this index compatibility intro, more than just in the Kibana use case that you are documenting.

Anyway feel free to ignore this if you or the rest of the team believes it doesn't add value, the current PR is really good and adds value!

@yetanothertw yetanothertw Jul 8, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is such a good comment and observation, you're totally right! We're missing crucial context, so I've tried to add some. Can you please review this again and let me know if this is what you had in mind, @eedugon ? 🙏


| Index creation version | {{version.stack.base}}–{{version.stack}} | 8.3–8.19 | 8.0–8.2 | 7.2–7.17 | 7.0–7.1 | 6.8 |
|------------------------------------------| ----------------------------------------- |----------|---------|----------|---------|-----|
| 5.0–5.6 | ✅ ^1^ | ✅ ^1^ | ❌ | ❌ | ❌ | ✅ |
| 6.0–6.7 | ✅ ^1^ | ✅ ^1^ | ❌ | ✅ | ✅ | ✅ |
| 6.8 | ✅ ^1^ | ✅ ^1^ | ❌ | ✅ | ❌ | ✅ |
| 7.0–7.1 | ✅ ^1, 2^ | ✅ | ✅ | ✅ | ✅ | ❌ |
| 7.2–7.17 | ✅ ^1, 2^ | ✅ | ✅ | ✅ | ❌ | ❌ |
| 8.0–8.19 | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
| {{version.stack.base}}–{{version.stack}} | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
Index compatibility in the context of the snapshot and restore process, indicates that {{es}} can restore the index and its data from the snapshot as a regular index or in a read-only form through [archive indices](/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md). It does not mean that every application reading that data will treat it as valid on the target cluster version.

For indices and data streams you own and control, a compatible restore usually means the data remains usable. For {{stack}} feature data, including {{kib}} saved objects and data that other features write to their own indices, {{es}} compatibility does not guarantee validity across versions. Those applications can expect different data formats across {{stack}} versions, so a restore that {{es}} accepts can still fail or leave data in an unusable state.

Use snapshot restore to recover or move data you own. Do not use it as a substitute for upgrading {{stack}} features. Follow the normal upgrade path for {{kib}} and other feature states.

The following table shows whether an index can be restored to a given cluster version. Find your index’s creation version in the left column and your cluster version (which is the restore target version) across the top. For example, an index created in 6.8 can be restored to a {{version.stack.base}}–{{version.stack}} cluster (✅) but not to a 7.0–7.1 cluster (❌).

Check warning on line 167 in deploy-manage/tools/snapshot-and-restore.md

View workflow job for this annotation

GitHub Actions / build / vale 

Elastic.DirectionalLanguage: Don't use directional language. Use 'the label of the element' instead of 'in the left'.

:::{table}
:widths: 3-2-1-2-2-1-1

| | | | | | | |
|-----------------------|-----|--------|-------|----------|---------|-----|
| | Restore to <br> {{version.stack.base}}–{{version.stack}} | 8.3–8.19 | 8.0–8.2 | 7.2–7.17 | 7.0–7.1 | 6.8 |
| Index created in version | | | | | | |
| 5.0–5.6 | ✅ ^1^ | ✅ ^1^ | ❌ | ❌ | ❌ | ✅ |
| 6.0–6.7 | ✅ ^1^ | ✅ ^1^ | ❌ | ✅ | ✅ | ✅ |
| 6.8 | ✅ ^1^ | ✅ ^1^ | ❌ | ✅ | ❌ | ✅ |
| 7.0–7.1 | ✅ ^1, 2^ | ✅ | ✅ | ✅ | ✅ | ❌ |
| 7.2–7.17 | ✅ ^1, 2^ | ✅ | ✅ | ✅ | ❌ | ❌ |
| 8.0–8.19 | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
| {{version.stack.base}}–{{version.stack}} | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
:::


^1^ $$$footnote-1$$$ Supported with [archive indices](/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md).

^2^ $$$footnote-2$$$ Supported with [searchable snapshots](/deploy-manage/tools/snapshot-and-restore/searchable-snapshots.md).

You can’t restore an index to an earlier version of {{es}}. For example, you can’t restore an index created in 8.18.0 to a cluster running 8.15.0.


#### Restoring incompatible indices

A compatible snapshot can contain indices created in an older incompatible version. To restore these incompatible indices, you must take additional steps. For example, a snapshot of a 7.17 cluster might contain an index created in 6.8. Restoring the 6.8 index to an 8.18 cluster fails unless you use the [archive functionality](/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md). To restore a 7.17 index to a 9.0 cluster, you can use the [archive functionality](/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md) or [searchable snapshots](/deploy-manage/tools/snapshot-and-restore/searchable-snapshots.md). Keep this in mind if you take a snapshot before upgrading a cluster.
Expand All @@ -182,6 +198,38 @@

Reindexing from remote can take significantly longer than restoring a snapshot. Before you start, test the reindex from remote process with a subset of the data to estimate your time requirements.

### {{kib}} compatibility

{{es}} and {{kib}} apply different compatibility rules to restored snapshots. A restore that {{es}} accepts can still fail when {{kib}} starts.

{{es}} can typically read data from indices created in a previous major version. For example, restoring a snapshot from an 8.1.0 cluster to a 9.2.4 cluster can succeed at the {{es}} layer, but can fail at the {{kib}} layer. This might happen due to an additional check {{kib}} applies on startup: it inspects version aliases on the `.kibana` index and runs saved object migrations.

If the restored {{kib}} system indices, such as `.kibana` and `.kibana_task_manager`, were last written by a version older than 8.18.0, {{kib}} 9.x doesn't start the migrations and reports an error similar to:

```
FATAL Error: Kibana 8.1.0 deployment detected. Please upgrade to Kibana 8.18.0 or newer before upgrading to 9.x series.
```

Restoring an old snapshot directly to a 9.x cluster is not a shortcut around the normal [upgrade paths](/deploy-manage/upgrade.md#upgrade-paths). It is recommended you upgrade to the latest compatible minor release before a major upgrade, even when you use snapshots to move data between clusters.

#### Move {{kib}} configuration and saved objects across major versions [move-kibana-config-saved-objects]

To make the {{kib}} state from an older 8.x snapshot compatible with a 9.x cluster, restore and migrate to an intermediate 8.19.x cluster first, then create a snapshot on the intermediate cluster, and then restore again on the target version:

1. [Restore the snapshot](/deploy-manage/tools/snapshot-and-restore/restore-snapshot.md) to a cluster running {{kib}} 8.19.x, or, if your target is 9.0.x, to a cluster running 8.18.x, as described in [Prepare to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md).
2. Start {{kib}} on the intermediate cluster and wait for startup to complete. During the startup, {{kib}} detects data from a previous compatible version and runs saved object migrations, similar to what would happen during a {{kib}} upgrade. This process rewrites the `.kibana` system indices from their restored version (for example, 8.1.0) to the 8.19.x format, which is fully compatible with 9.x. {{es}} does not run these migrations during a snapshot restore.
3. [Take a new snapshot](/deploy-manage/tools/snapshot-and-restore/create-snapshots.md) of the migrated cluster, then restore that snapshot to your 9.x cluster. Start {{kib}} and wait for the startup to complete. During startup, {{kib}} runs saved object migrations, as it would when upgrading from 8.19.x to 9.x.

For example, to restore the {{kib}} state from an 8.1.0 snapshot to a 9.2.4 cluster, restore the 8.1.0 snapshot to an 8.19.x cluster, let {{kib}} complete its startup migrations, snapshot the 8.19.x cluster, restore that snapshot to 9.2.4, and let {{kib}} perform the latest data migrations.

Alternatively, if you only need to recover data and can accept a fresh {{kib}} setup, restore the snapshot without the `kibana` feature state.

:::{admonition} Roll back after a failed upgrade
The steps described in [moving the {{kib}} configuration and saved objects across major versions](#move-kibana-config-saved-objects) move {{kib}} state forward to a newer major version. Don't use these steps to roll back after a failed upgrade.

To roll back {{kib}}, restore the `kibana` feature state from a snapshot taken immediately before the failed upgrade, then start {{kib}} on the version that was running before that upgrade attempt. For more information, refer to [Roll back {{kib}}](/deploy-manage/upgrade/deployment-or-cluster/kibana-roll-back.md).
:::

## Warnings

### Other backup methods
Expand Down
Loading