Merge pull request #9804 from ThomasWaldmann/interals-docs-update

docs: update / fix "internals" section

Author: ThomasWaldmann

docs/internals/data-structures.rst

@@ -51,9 +51,14 @@ data/
       0000... .. ffff...
 
 keys/
-    When using encryption in repokey mode, the encrypted, passphrase protected
-    key is stored here as a base64 encoded text. The sha256 content hash is
-    used for the name.
+    When using repokey mode, the encrypted, passphrase protected borg keys are
+    stored here as a base64 encoded text. The sha256 content hash of the
+    stored borg key is used for the name.
+
+    A repository may contain *multiple* such borg keys (one per passphrase) to
+    support the :ref:`multiple borg keys <borgcrypto_multiple_keys>` feature.
+    keyfile and repokey borg keys use the same format and naming (only the
+    storage location differs).
 
 locks/
   used by the locking system to manage shared and exclusive locks.
@@ -67,7 +72,10 @@ byte strings of fixed length (256-bit, 32 bytes), computed like this::
 
   key = id = id_hash(plaintext_data)  # plain = not encrypted, not compressed, not obfuscated
 
-The id_hash function depends on the :ref:`encryption mode <borg_repo-create>`.
+The id_hash function is selected via ``borg repo-create --id-hash`` (independently
+of ``--encryption``). For encrypted repositories it is a keyed MAC over the
+plaintext (keyed by ``id_key``): ``sha256`` selects HMAC-SHA256, ``blake3``
+selects a keyed BLAKE3. The unencrypted ``none`` mode uses a plain ``sha256``.
 
 As the id / key is used for deduplication, id_hash must be a cryptographically
 strong hash or MAC.
@@ -718,11 +726,15 @@ Both modes
 
 Encryption keys (and other secrets) are kept either in the keys directory on
 the client ('keyfile' mode) or under the keys/ namespace in the repository
-('repokey' mode) using the sha256 of the file content as the name.
+('repokey' mode) using the sha256 of the borg key content as the name.
 
 In both cases, the secrets are generated from random and then encrypted by a
 key derived from your passphrase (this happens on the client before the key
-is stored into the keyfile or as repokey).
+is stored as keyfile or repokey).
+
+keyfile and repokey borg keys use the **same** format; only the storage location
+differs. Borg finds the correct key by trying each key against the supplied
+passphrase. See :ref:`borgcrypto_multiple_keys`.
 
 The passphrase is passed through the ``BORG_PASSPHRASE`` environment variable
 or prompted for interactive usage.

docs/internals/security.rst

@@ -116,22 +116,27 @@ Encryption
 AEAD modes
 ~~~~~~~~~~
 
-Modes: --encryption (repokey|keyfile)-[blake2-](aes-ocb|chacha20-poly1305)
+Modes: ``--encryption (aes256-ocb|chacha20-poly1305)`` plus
+``--id-hash (sha256|blake3)``
 
 Supported: borg 2.0+
 
+The cipher is selected by ``--encryption`` (see :ref:`borg_repo-create`), the
+key storage location (repokey or keyfile) by ``--key-location``, and the chunk
+ID hash function by ``--id-hash`` — these three are orthogonal.
+
 Encryption with these modes is based on AEAD ciphers (authenticated encryption
 with associated data) and session keys.
 
-Depending on the chosen mode (see :ref:`borg_repo-create`) different AEAD ciphers are used:
+Depending on the chosen mode different AEAD ciphers are used:
 
 - AES-256-OCB - super fast, single-pass algorithm IF you have hw accelerated AES.
 - chacha20-poly1305 - very fast, purely software based AEAD cipher.
 
 The chunk ID is derived via a MAC over the plaintext (mac key taken from borg key):
 
-- HMAC-SHA256 - super fast IF you have hw accelerated SHA256 (see section "Encryption" below).
-- Blake2b - very fast, purely software based algorithm.
+- HMAC-SHA256 (``--id-hash sha256``) - super fast IF you have hw accelerated SHA256 (see section "Encryption" below).
+- keyed BLAKE3 (``--id-hash blake3``) - very fast, purely software based algorithm.
 
 For each borg invocation, a new session id is generated by `os.urandom`_.
 
@@ -177,8 +182,8 @@ Decryption::
 Notable:
 
 - More modern and often faster AEAD ciphers instead of self-assembled stuff.
-- Due to the usage of session keys, IVs (nonces) do not need special care here as
-  they did for the legacy encryption modes.
+- Due to the usage of session keys, which just start at 0 per session, IVs (nonces)
+  do not need long-term special care here as they did for the legacy encryption modes.
 - The id is now also input into the authentication tag computation.
   This strongly associates the id with the written data (== associates the key with
   the value). When later reading the data for some id, authentication will only
@@ -188,11 +193,14 @@ Notable:
 Legacy modes
 ~~~~~~~~~~~~
 
-Modes: --encryption (repokey|keyfile)-[blake2]
+Modes: ``--encryption (repokey|keyfile)[-blake2]``
 
 Supported: borg < 2.0
 
-These were the AES-CTR based modes in previous borg versions.
+These were the AES-CTR based modes in previous borg versions, with the chunk ID
+derived via HMAC-SHA256 or (in the ``-blake2`` variants) Blake2b. ``blake2b`` is
+only used by these legacy modes; new repositories use ``sha256`` or ``blake3``
+(see above).
 
 borg 2.0 does not support creating new repos using these modes,
 but ``borg transfer`` can still read such existing repos.
@@ -215,13 +223,30 @@ to Encrypt-*then*-MAC a packed representation of the keys using the
 chacha20-poly1305 AEAD cipher and a constant IV == 0.
 The ciphertext is then converted to base64.
 
-This base64 blob (commonly referred to as *keyblob*) is then stored in
-the key file or in the repository config (keyfile and repokey modes
-respectively).
+This base64-encoded *borg key* is then stored in the key file or under the
+repository's ``keys/`` namespace (keyfile and repokey modes respectively), named
+by the sha256 of its content.
 
 The use of a constant IV is secure because an identical passphrase will
 result in a different derived KEK for every key encryption due to the salt.
 
+.. _borgcrypto_multiple_keys:
+
+Multiple borg keys
+~~~~~~~~~~~~~~~~~~
+
+A repository (or a client-side keyfile directory) may hold *multiple* borg keys,
+each encrypted with its own passphrase but all wrapping the **same** underlying
+key material. This lets several people access a shared repository with
+independent passphrases, without sharing one secret. Or you can add borg keys
+for redundant, more fault-tolerant storage.
+
+keyfile and repokey borg keys use the same format and the same sha256-content
+naming; borg locates a borg key independently of its key type byte and tries each
+available one against the supplied passphrase until one decrypts. A borg key may
+carry a label for management. The constant-IV argument above still holds, because
+each borg key has its own random argon2 salt and therefore a distinct derived KEK.
+
 
 .. seealso::
 
@@ -239,13 +264,16 @@ on widely used libraries providing them:
   We think this is not an additional risk, since we don't ever
   use OpenSSL's networking, TLS or X.509 code, but only their
   primitives implemented in libcrypto.
-- SHA-256, SHA-512 and BLAKE2b from Python's hashlib_ standard library module are used.
+- SHA-256 and SHA-512 from Python's hashlib_ standard library module are used.
+- BLAKE3 is used via the blake3_ package (new repos, ``--id-hash blake3``).
+- BLAKE2b from Python's hashlib_ is only used to read legacy (borg < 2.0) repos.
 - HMAC and a constant-time comparison from Python's hmac_ standard library module are used.
 - argon2 is used via argon2-cffi.
 
 .. _Horton principle: https://en.wikipedia.org/wiki/Horton_Principle
 .. _length extension: https://en.wikipedia.org/wiki/Length_extension_attack
 .. _hashlib: https://docs.python.org/3/library/hashlib.html
+.. _blake3: https://pypi.org/project/blake3/
 .. _hmac: https://docs.python.org/3/library/hmac.html
 .. _os.urandom: https://docs.python.org/3/library/os.html#os.urandom