Add support for compounding validators and the related withdrawal credentials from EIP 7251 (#228)

* Initial implementation for compounding validators and support for EIP 7251

* Adding amount option to new-mnemonic and existing-mnemonic commands

* Fix for passing decimal values for amount from CLI

* Fix for function renamed

* Refactor all the prompt_if arguments from captive_prompt_callback in a generic callable

* Improved help and message when using compounding validators

* Adding documentation about the new compounding and amount options

* Unit tests for compounding features and related fixes

Author: remyroy

docs/src/existing_mnemonic.md

@@ -23,6 +23,10 @@ Uses an existing BIP-39 mnemonic phrase for key generation.
 
 - **`--withdrawal_address`**: The Ethereum address that will be used in withdrawal. It typically starts with '0x' followed by 40 hexadecimal characters. Please make sure you have full control over the address you choose here. Once you set a withdrawal address on chain, it cannot be changed.
 
+- **`--compounding / --regular-withdrawal`**: Generates compounding validators with 0x02 withdrawal credentials for a 2048 ETH maximum effective balance or generate regular validators with 0x01 withdrawal credentials for a 32 ETH maximum effective balance. Use of this option requires a withdrawal address. This feature is only supported on networks that have undergone the Pectra fork. Defaults to regular withdrawal.
+
+- **`--amount`**: The amount to deposit to these validators in ether denomination. Must be at least 1 ether and can not have greater precision than 1 gwei. Use of this option requires compounding validators. Defaults to 32 ether.
+
 - **`--pbkdf2`**: Will use pbkdf2 key derivation instead of scrypt for generated keystore files as defined in [EIP-2335](https://eips.ethereum.org/EIPS/eip-2335#decryption-key). This can be a good alternative if you intend to work with a large number of keys, as it can improve performance however it is less secure. You should only use this option if you understand the associated risks and have familiarity with encryption.
 
 - **`--folder`**: The folder where keystore and deposit data files will be saved.

docs/src/new_mnemonic.md

@@ -17,6 +17,10 @@ Generates a new BIP-39 mnemonic along with validator keystore and deposit files
 
 - **`--withdrawal_address`**: The Ethereum address that will be used in withdrawal. It typically starts with '0x' followed by 40 hexadecimal characters. Please make sure you have full control over the address you choose here. Once you set a withdrawal address on chain, it cannot be changed.
 
+- **`--compounding / --regular-withdrawal`**: Generates compounding validators with 0x02 withdrawal credentials for a 2048 ETH maximum effective balance or generate regular validators with 0x01 withdrawal credentials for a 32 ETH maximum effective balance. Use of this option requires a withdrawal address. This feature is only supported on networks that have undergone the Pectra fork. Defaults to regular withdrawal.
+
+- **`--amount`**: The amount to deposit to these validators in ether denomination. Must be at least 1 ether and can not have greater precision than 1 gwei. Use of this option requires compounding validators. Defaults to 32 ether.
+
 - **`--pbkdf2`**: Will use pbkdf2 key encryption instead of scrypt for generated keystore files as defined in [EIP-2335](https://eips.ethereum.org/EIPS/eip-2335#decryption-key). This can be a good alternative if you intend to work with a large number of keys, as it can improve performance. pbkdf2 encryption is, however, less secure than scrypt. You should only use this option if you understand the associated risks and have familiarity with encryption.
 
 - **`--folder`**: The folder where keystore and deposit data files will be saved.

docs/src/partial_deposit.md

@@ -18,6 +18,8 @@ If you wish to create a validator with 0x00 credentials, you must use the **[new
 
 - **`--withdrawal_address`**: The withdrawal address of the existing validator or the desired withdrawal address.
 
+- **`--compounding / --regular-withdrawal`**: Generates compounding validators with 0x02 withdrawal credentials for a 2048 ETH maximum effective balance or generate regular validators with 0x01 withdrawal credentials for a 32 ETH maximum effective balance. Use of this option requires a withdrawal address. This feature is only supported on networks that have undergone the Pectra fork. Defaults to regular withdrawal.
+
 - **`--output_folder`**: The folder path for the `deposit-*` JSON file.
 
 - **`--devnet_chain_setting`**: The custom chain setting of a devnet or testnet. Note that it will override your `--chain` choice. This should be a JSON string containing an object with the following keys: network_name, genesis_fork_version, exit_fork_version and genesis_validator_root.

ethstaker_deposit/cli/existing_mnemonic.py

@@ -19,6 +19,7 @@
     captive_prompt_callback,
     choice_prompt_func,
     jit_option,
+    prompt_if_none,
 )
 from ethstaker_deposit.utils.intl import fuzzy_reverse_dict_lookup, get_first_options, load_text
 from ethstaker_deposit.utils.validation import validate_int_range
@@ -39,7 +40,7 @@ def load_mnemonic_arguments_decorator(function: Callable[..., Any]) -> Callable[
                 captive_prompt_callback(
                     lambda mnemonic: validate_mnemonic(mnemonic=mnemonic, language=c.params.get('mnemonic_language')),
                     prompt=lambda: load_text(['arg_mnemonic', 'prompt'], func='existing_mnemonic'),
-                    prompt_if_none=True,
+                    prompt_if=prompt_if_none,
                 )(c, _, mnemonic),
             help=lambda: load_text(['arg_mnemonic', 'help'], func='existing_mnemonic'),
             param_decls='--mnemonic',
@@ -105,7 +106,7 @@ def validate_mnemonic_language(ctx: click.Context, param: Any, language: str) ->
         lambda num: validate_int_range(num, 0, 2**32),
         lambda: load_text(['arg_validator_start_index', 'prompt'], func='existing_mnemonic'),
         lambda: load_text(['arg_validator_start_index', 'confirm'], func='existing_mnemonic'),
-        prompt_if_none=True,
+        prompt_if=prompt_if_none,
     ),
     default=0,
     help=lambda: load_text(['arg_validator_start_index', 'help'], func='existing_mnemonic'),

ethstaker_deposit/cli/exit_transaction_keystore.py

@@ -18,6 +18,8 @@
     captive_prompt_callback,
     choice_prompt_func,
     jit_option,
+    prompt_if_none,
+    prompt_if_other_is_none,
 )
 from ethstaker_deposit.utils.constants import DEFAULT_EXIT_TRANSACTION_FOLDER_NAME
 from ethstaker_deposit.utils.intl import (
@@ -45,7 +47,7 @@
             lambda: load_text(['arg_exit_transaction_keystore_chain', 'prompt'], func=FUNC_NAME),
             ALL_CHAIN_KEYS
         ),
-        prompt_if_other_is_none='devnet_chain_setting',
+        prompt_if=prompt_if_other_is_none('devnet_chain_setting'),
         default=MAINNET,
     ),
     default=MAINNET,
@@ -57,7 +59,7 @@
     callback=captive_prompt_callback(
         lambda file: validate_keystore_file(file),
         lambda: load_text(['arg_exit_transaction_keystore_keystore', 'prompt'], func=FUNC_NAME),
-        prompt_if_none=True,
+        prompt_if=prompt_if_none,
     ),
     help=lambda: load_text(['arg_exit_transaction_keystore_keystore', 'help'], func=FUNC_NAME),
     param_decls='--keystore',

ethstaker_deposit/cli/exit_transaction_mnemonic.py

@@ -18,6 +18,7 @@
     captive_prompt_callback,
     choice_prompt_func,
     jit_option,
+    prompt_if_other_is_none,
 )
 from ethstaker_deposit.utils.constants import DEFAULT_EXIT_TRANSACTION_FOLDER_NAME
 from ethstaker_deposit.utils.intl import (
@@ -61,7 +62,7 @@ def _exit_verifier(kwargs: Dict[str, Any]) -> bool:
             lambda: load_text(['arg_exit_transaction_mnemonic_chain', 'prompt'], func=FUNC_NAME),
             ALL_CHAIN_KEYS
         ),
-        prompt_if_other_is_none='devnet_chain_setting',
+        prompt_if=prompt_if_other_is_none('devnet_chain_setting'),
         default=MAINNET,
     ),
     default=MAINNET,
@@ -140,6 +141,7 @@ def exit_transaction_mnemonic(
             'amount': 0,
             'chain_setting': chain_setting,
             'hex_withdrawal_address': None,
+            'compounding': False,
         } for index in key_indices]
 
         with concurrent.futures.ProcessPoolExecutor() as executor:

ethstaker_deposit/cli/generate_bls_to_execution_change.py

@@ -32,6 +32,8 @@
     captive_prompt_callback,
     choice_prompt_func,
     jit_option,
+    prompt_if_none,
+    prompt_if_other_is_none,
 )
 from ethstaker_deposit.exceptions import ValidationError
 from ethstaker_deposit.utils.intl import (
@@ -81,7 +83,7 @@ def _validate_credentials_match(kwargs: Dict[str, Any]) -> Optional[ValidationEr
             lambda: load_text(['arg_chain', 'prompt'], func=FUNC_NAME),
             ALL_CHAIN_KEYS
         ),
-        prompt_if_other_is_none='devnet_chain_setting',
+        prompt_if=prompt_if_other_is_none('devnet_chain_setting'),
         default=MAINNET,
     ),
     default=MAINNET,
@@ -114,7 +116,7 @@ def _validate_credentials_match(kwargs: Dict[str, Any]) -> Optional[ValidationEr
         lambda bls_withdrawal_credentials_list:
             validate_bls_withdrawal_credentials_list(bls_withdrawal_credentials_list),
         lambda: load_text(['arg_bls_withdrawal_credentials_list', 'prompt'], func=FUNC_NAME),
-        prompt_if_none=True,
+        prompt_if=prompt_if_none,
     ),
     help=lambda: load_text(['arg_bls_withdrawal_credentials_list', 'help'], func=FUNC_NAME),
     param_decls='--bls_withdrawal_credentials_list',
@@ -126,7 +128,7 @@ def _validate_credentials_match(kwargs: Dict[str, Any]) -> Optional[ValidationEr
         lambda: load_text(['arg_withdrawal_address', 'prompt'], func=FUNC_NAME),
         lambda: load_text(['arg_withdrawal_address', 'confirm'], func=FUNC_NAME),
         lambda: load_text(['arg_withdrawal_address', 'mismatch'], func=FUNC_NAME),
-        prompt_if_none=True,
+        prompt_if=prompt_if_none,
     ),
     help=lambda: load_text(['arg_withdrawal_address', 'help'], func=FUNC_NAME),
     param_decls=['--withdrawal_address'],
@@ -180,6 +182,7 @@ def generate_bls_to_execution_change(
         chain_setting=chain_setting,
         start_index=validator_start_index,
         hex_withdrawal_address=withdrawal_address,
+        compounding=False,
     )
 
     # Check if the given old bls_withdrawal_credentials is as same as the mnemonic generated

ethstaker_deposit/cli/generate_bls_to_execution_change_keystore.py

@@ -27,6 +27,8 @@
     captive_prompt_callback,
     choice_prompt_func,
     jit_option,
+    prompt_if_none,
+    prompt_if_other_is_none,
 )
 from ethstaker_deposit.utils.intl import (
     closest_match,
@@ -53,7 +55,7 @@
             lambda: load_text(['arg_chain', 'prompt'], func=FUNC_NAME),
             ALL_CHAIN_KEYS
         ),
-        prompt_if_other_is_none='devnet_chain_setting',
+        prompt_if=prompt_if_other_is_none('devnet_chain_setting'),
         default=MAINNET,
     ),
     default=MAINNET,
@@ -65,7 +67,7 @@
     callback=captive_prompt_callback(
         lambda file: validate_keystore_file(file),
         lambda: load_text(['arg_bls_to_execution_changes_keystore_keystore', 'prompt'], func=FUNC_NAME),
-        prompt_if_none=True,
+        prompt_if=prompt_if_none,
     ),
     help=lambda: load_text(['arg_bls_to_execution_changes_keystore_keystore', 'help'], func=FUNC_NAME),
     param_decls='--keystore',
@@ -99,7 +101,7 @@
         lambda: load_text(['arg_withdrawal_address', 'prompt'], func=FUNC_NAME),
         lambda: load_text(['arg_withdrawal_address', 'confirm'], func=FUNC_NAME),
         lambda: load_text(['arg_withdrawal_address', 'mismatch'], func=FUNC_NAME),
-        prompt_if_none=True,
+        prompt_if=prompt_if_none,
     ),
     help=lambda: load_text(['arg_withdrawal_address', 'help'], func=FUNC_NAME),
     param_decls=['--withdrawal_address'],

ethstaker_deposit/cli/generate_keys.py

@@ -18,17 +18,24 @@
     validate_int_range,
     validate_password_strength,
     validate_withdrawal_address,
+    validate_yesno,
+    validate_deposit_amount,
     validate_devnet_chain_setting,
 )
 from ethstaker_deposit.utils.constants import (
     DEFAULT_VALIDATOR_KEYS_FOLDER_NAME,
     MIN_ACTIVATION_AMOUNT,
+    ETH2GWEI,
 )
 from ethstaker_deposit.utils.ascii_art import RHINO_0
 from ethstaker_deposit.utils.click import (
     captive_prompt_callback,
     choice_prompt_func,
     jit_option,
+    prompt_if_none,
+    prompt_if_other_is_none,
+    prompt_if_other_exists,
+    prompt_if_other_value,
 )
 from ethstaker_deposit.utils.intl import (
     closest_match,
@@ -43,6 +50,9 @@
 )
 
 
+min_activation_amount_eth = MIN_ACTIVATION_AMOUNT // ETH2GWEI
+
+
 def generate_keys_arguments_decorator(function: Callable[..., Any]) -> Callable[..., Any]:
     '''
     This is a decorator that, when applied to a parent-command, implements the
@@ -71,7 +81,7 @@ def generate_keys_arguments_decorator(function: Callable[..., Any]) -> Callable[
                     lambda: load_text(['chain', 'prompt'], func='generate_keys_arguments_decorator'),
                     ALL_CHAIN_KEYS
                 ),
-                prompt_if_other_is_none='devnet_chain_setting',
+                prompt_if=prompt_if_other_is_none('devnet_chain_setting'),
                 default=MAINNET,
             ),
             default=MAINNET,
@@ -86,7 +96,7 @@ def generate_keys_arguments_decorator(function: Callable[..., Any]) -> Callable[
                 lambda: load_text(['keystore_password', 'confirm'], func='generate_keys_arguments_decorator'),
                 lambda: load_text(['keystore_password', 'mismatch'], func='generate_keys_arguments_decorator'),
                 True,
-                prompt_if_none=True,
+                prompt_if=prompt_if_none,
             ),
             help=lambda: load_text(['keystore_password', 'help'], func='generate_keys_arguments_decorator'),
             hide_input=True,
@@ -100,13 +110,40 @@ def generate_keys_arguments_decorator(function: Callable[..., Any]) -> Callable[
                 lambda: load_text(['arg_withdrawal_address', 'confirm'], func='generate_keys_arguments_decorator'),
                 lambda: load_text(['arg_withdrawal_address', 'mismatch'], func='generate_keys_arguments_decorator'),
                 default="",
-                prompt_if_none=True,
+                prompt_if=prompt_if_none,
             ),
             default="",
             help=lambda: load_text(['arg_withdrawal_address', 'help'], func='generate_keys_arguments_decorator'),
             param_decls=['--withdrawal_address', '--execution_address', '--eth1_withdrawal_address'],
             prompt=False,  # the callback handles the prompt
         ),
+        jit_option(
+            callback=captive_prompt_callback(
+                lambda value: validate_yesno(None, None, value),
+                lambda: load_text(['arg_compounding', 'prompt'], func='generate_keys_arguments_decorator'),
+                default="False",
+                prompt_if=prompt_if_other_exists('withdrawal_address'),
+            ),
+            default=False,
+            help=lambda: load_text(['arg_compounding', 'help'], func='generate_keys_arguments_decorator'),
+            param_decls='--compounding/--regular-withdrawal',
+            prompt=False,  # the callback handles the prompt
+            type=bool,
+            show_default=True,
+        ),
+        jit_option(
+            callback=captive_prompt_callback(
+                lambda amount: validate_deposit_amount(amount),
+                lambda: load_text(['arg_amount', 'prompt'], func='generate_keys_arguments_decorator'),
+                default=str(min_activation_amount_eth),
+                prompt_if=prompt_if_other_value('compounding', True),
+            ),
+            default=str(min_activation_amount_eth),
+            help=lambda: load_text(['arg_amount', 'help'], func='generate_keys_arguments_decorator'),
+            param_decls='--amount',
+            prompt=False,  # the callback handles the prompt
+            show_default=True,
+        ),
         jit_option(
             default=False,
             is_flag=True,
@@ -130,11 +167,13 @@ def generate_keys_arguments_decorator(function: Callable[..., Any]) -> Callable[
 @click.pass_context
 def generate_keys(ctx: click.Context, validator_start_index: int,
                   num_validators: int, folder: str, chain: str, keystore_password: str,
-                  withdrawal_address: HexAddress, pbkdf2: bool,
+                  withdrawal_address: HexAddress, compounding: bool, amount: int, pbkdf2: bool,
                   devnet_chain_setting: Optional[BaseChainSetting], **kwargs: Any) -> None:
     mnemonic = ctx.obj['mnemonic']
     mnemonic_password = ctx.obj['mnemonic_password']
-    amounts = [MIN_ACTIVATION_AMOUNT] * num_validators
+    if withdrawal_address is None or not compounding:
+        amount = MIN_ACTIVATION_AMOUNT
+    amounts = [amount] * num_validators
     folder = os.path.join(folder, DEFAULT_VALIDATOR_KEYS_FOLDER_NAME)
 
     # Get chain setting
@@ -153,6 +192,7 @@ def generate_keys(ctx: click.Context, validator_start_index: int,
         chain_setting=chain_setting,
         start_index=validator_start_index,
         hex_withdrawal_address=withdrawal_address,
+        compounding=compounding,
         use_pbkdf2=pbkdf2
     )