feat: merge-train/fairies-v5 (#24470)

BEGIN_COMMIT_OVERRIDE
feat!: remove fallback and signing keys from pxe (#24451)
fix(aztec-nr): support empty notes and events (#24464)
docs: improve upgrades warning (#24462)
feat!: make contract classes dynamic (#24282)
fix(txe): align aztec_avm_returndataSize registry type with Noir u32
(#24480)
fix(wallet-sdk): use derived contract address in registerContract
(#24482)
docs: document how to make nested utility calls (#24478)
feat(aztec-nr): add interactive handshake to the handshake registry
(#24473)
END_COMMIT_OVERRIDE

Author: AztecBot

docs/docs-developers/docs/aztec-js/how_to_test.md

@@ -85,7 +85,7 @@ Use this to set up state preconditions, reproduce production bugs against pinned
 
 ### Fast-forwarding a contract update
 
-`fastForwardContractUpdate` returns a `SimulationOverrides` object that simulates a deployed instance as if it had already been upgraded to a new contract class. The new class must already be registered on chain. The cheat mirrors a real `pxe.updateContract` followed by waiting out the upgrade delay: the instance's `currentContractClassId` is bumped, and the `ContractInstanceRegistry`'s delayed-public-mutable storage is rewritten to look like the upgrade was scheduled in the past.
+`fastForwardContractUpdate` returns a `SimulationOverrides` object that simulates a deployed instance as if it had already been upgraded to a new contract class. The new class must already be registered on chain. The cheat mirrors a real onchain upgrade followed by waiting out the upgrade delay: the override instance's `currentContractClassId` is bumped, and the `ContractInstanceRegistry`'s delayed-public-mutable storage is rewritten to look like the upgrade was scheduled in the past.
 
 ```typescript
 import { fastForwardContractUpdate } from '@aztec/aztec.js';

docs/docs-developers/docs/aztec-nr/framework-description/calling_contracts.md

@@ -67,3 +67,46 @@ self.enqueue(Token::at(token_address).mint_to_public(recipient, amount));
 :::info
 Public functions execute after all private execution completes. Return values are not available in the private context. Learn more about [call types](../../foundational-topics/call_types.md).
 :::
+
+## Utility calls
+
+Utility functions can call other utility functions. These calls run entirely offchain in PXE and are never proven, so no guarantees are made on the correctness of their results.
+
+### Utility-to-utility calls
+
+From a utility function, use `self.call()` as with other call types:
+
+```rust
+let balance = self.call(Token::at(token_address).get_balance_of(owner));
+```
+
+To call a utility function of your own contract, use the `self.call_self` stubs instead:
+
+```rust
+self.call_self.my_utility_function(args)
+```
+
+### Private-to-utility calls
+
+Private functions can also call utility functions, through `self.utility`. The call executes unconstrained code, so it must be wrapped in `unsafe`, and its result is not proven: use it to inform logic, never as an input to a constrained assertion without validating it first.
+
+```rust
+// Safety: result is unconstrained
+let balance = unsafe { self.utility.call(Token::at(token_address).get_balance_of(owner)) };
+```
+
+The same-contract stubs are available here as `self.utility.call_self`:
+
+```rust
+unsafe { self.utility.call_self.my_utility_function(args) }
+```
+
+Public functions cannot call utility functions: they run on the sequencer, which has no access to private state or PXE.
+
+### Cross-contract authorization
+
+A utility call to the calling contract itself always succeeds. A call to a _different_ contract can expose that contract's private state to the caller, so PXE asks the wallet to authorize it before proceeding. If the wallet denies the call, or no `authorizeUtilityCall` hook is configured, the simulation fails with `Cross-contract utility call denied`. See [execution hooks](../../foundational-topics/pxe/execution_hooks.md#authorizeutilitycall) for how wallets handle these requests, and for authorizing targets in Noir tests with `with_authorized_utility_call_targets`.
+
+### `msg_sender` in nested utility calls
+
+A utility function called from another contract can read the calling contract's address via `self.msg_sender()`, mirroring private and public functions. A utility function invoked directly by an application has no caller: `self.msg_sender()` panics, and `self.context.maybe_msg_sender()` returns `Option::none()`. Within a simulation PXE takes this value from the actual call graph, but utility execution as a whole is unconstrained, so contracts must not rely on it as a security guarantee.

docs/docs-developers/docs/aztec-nr/framework-description/contract_upgrades.md

@@ -5,6 +5,17 @@ tags: [contracts]
 description: Understand contract upgrade patterns in Aztec and how to implement upgradeable contracts.
 ---
 
+:::warning[Upgrades are not yet well supported by the framework]
+Contract upgrades are currently a low-level protocol feature with minimal framework support. Before relying on them, be aware that:
+
+- Upgrades are hard to execute correctly, and need lots of careful testing.
+- State migrations are non-trivial and also require extensive testing. The new implementation must be compatible with all existing state, including public storage, private notes, and initialization status.
+- The `#[aztec]` macros are not at the moment built with upgrades in mind. Macro-generated code makes assumptions that upgrades can violate (for example, initialization checks in the new implementation can make all functions of already-deployed instances permanently uncallable), so users may need to drop the macros until support is added.
+- There is no additional tooling at the moment to help with upgrades, such as detection of storage layout compatibility between the old and new implementations.
+
+All in all, this feature is not well supported by the framework as of now, and should only be used with extreme care.
+:::
+
 Each contract instance refers to a contract class ID for its code. Upgrading a contract's implementation involves updating its current class ID to a new class ID, while retaining the original class ID for address verification.
 
 ## Original class ID
@@ -134,11 +145,3 @@ const updatedContract = UpdatedContract.at(contract.address, wallet);
 ```
 
 If you try to register a contract artifact that doesn't match the current contract class, the registration will fail.
-
-### Security considerations
-
-1. **Access control**: Implement proper access controls for upgrade functions. Consider using `set_update_delay` to customize the delay for your security requirements.
-
-2. **State compatibility**: Ensure the new implementation is compatible with existing state. Maintain the same storage layout to prevent data corruption.
-
-3. **Testing**: Test upgrades thoroughly in a development environment. Verify all existing functionality works with the new implementation.

docs/docs-developers/docs/aztec-nr/framework-description/functions/how_to_define_functions.md

@@ -49,7 +49,7 @@ Public functions operate on public state, similar to EVM contracts. They can wri
 
 Create offchain query functions using the `#[external("utility")]` annotation with `unconstrained`.
 
-Utility functions are standalone unconstrained functions that cannot be called from private or public functions. They are meant to be called by _applications_ to perform auxiliary tasks like querying contract state or processing offchain messages. Example:
+Utility functions are unconstrained functions that applications call to perform auxiliary tasks, like querying contract state or processing offchain messages. Their execution is never proven, so no guarantees are made on the correctness of their results. They can also be called from other utility functions and from private functions. See [utility calls](../calling_contracts.md#utility-calls). Public functions cannot call them. Example:
 
 #include_code get_counter /docs/examples/contracts/counter_contract/src/main.nr rust
 

docs/docs-developers/docs/foundational-topics/accounts/keys.md

@@ -66,7 +66,7 @@ The nullifier hiding key (`nhk`) — sometimes referred to in older documentatio
 |---------|----------|----------------|
 | Private (constrained) | `context.request_nhk_app(owner_npk_m_hash)` | Called on `&mut PrivateContext` |
 | Unconstrained | `get_nhk_app(owner_npk_m_hash)` | `use aztec::keys::getters::get_nhk_app` |
-| TypeScript (master key) | `deriveMasterNullifierHidingKey(secretKey)` | `import { deriveMasterNullifierHidingKey } from '@aztec/aztec.js/keys'` |
+| TypeScript (master key) | `deriveMasterNullifierHidingSecretKey(secretKey)` | `import { deriveMasterNullifierHidingSecretKey } from '@aztec/aztec.js/keys'` |
 | TypeScript (app-siloed) | `computeAppNullifierHidingKey(nhkM, app)` | `import { computeAppNullifierHidingKey } from '@aztec/aztec.js/keys'` |
 
 To get the owner's master nullifier public key hash (needed as input):
@@ -75,7 +75,7 @@ To get the owner's master nullifier public key hash (needed as input):
 let owner_npk_m_hash = get_public_keys(owner).npk_m_hash;
 ```
 
-`PublicKeys` exposes the nullifier, outgoing-viewing, and tagging keys directly as their hashes; only `ivpk_m` is held as a Grumpkin point (it is required as a point for address derivation and encrypt-to-address).
+`PublicKeys` exposes the nullifier, outgoing-viewing, tagging, message-signing, and fallback keys as their hashes; only `ivpk_m` is held as a Grumpkin point (it is required as a point for address derivation and encrypt-to-address).
 
 :::warning
 Do not compute nullifier keys by hand or derive custom blinding factors. The protocol kernel validates that `nhk_app` derives correctly from the master key — a hand-rolled value will fail verification.
@@ -159,7 +159,7 @@ The `Ovpk` (outgoing viewing key) and `Tpk` (tagging key) exist in the protocol'
 :::
 
 :::note
-The `Mspk` (master message-signing key) and `Fbpk` (master fallback key) hash slots exist in `PublicKeys` and participate in the address derivation above, but there is **no canonical derivation path for them yet**. `deriveKeys(secretKey)` stamps the canonical default hashes (`DEFAULT_MSPK_M_HASH`, `DEFAULT_FBPK_M_HASH`) into every account, so today they contribute no per-account entropy. When a real derivation lands in a future release, the address derived from the same secret will change — see the migration notes.
+The `Mspk` (master message-signing key) and `Fbpk` (master fallback key) are reserved for future protocol features (message signing and account recovery, respectively). Their public keys are derived per account and participate in the address derivation above. Unlike the other privacy keys, their **secret keys are held by the wallet, not the PXE**: the PXE receives only the corresponding public keys, since it is not trusted to hold these secrets.
 :::
 
 ## Key Management
@@ -215,9 +215,9 @@ Aztec's multi-key architecture is fundamental to its privacy and security model:
 | **Incoming Viewing** (`Ivpk_m`) | Decrypt received notes | No | No | Protocol (PXE) |
 | **Outgoing Viewing** (`Ovpk_m`) | Reserved | N/A | No | Protocol (PXE) |
 | **Tagging** (`Tpk_m`) | Reserved | N/A | No | Protocol (PXE) |
-| **Message-Signing** (`Mspk_m`) | Reserved | N/A | No | Protocol (PXE) |
-| **Fallback** (`Fbpk_m`) | Reserved | N/A | No | Protocol (PXE) |
-| **Signing** | Transaction authorization | N/A | Yes | Application |
+| **Message-Signing** (`Mspk_m`) | Reserved | N/A | No | Application (Wallet) |
+| **Fallback** (`Fbpk_m`) | Reserved | N/A | No | Application (Wallet) |
+| **Signing** | Transaction authorization | N/A | Yes | Account Contract (Wallet) |
 
 **Key takeaways:**
 

docs/docs-developers/docs/foundational-topics/call_types.md

@@ -187,11 +187,13 @@ Public functions can be called either directly in a public context (as shown abo
 
 ### Utility
 
-Contract functions marked with `#[external("utility")]` cannot be called as part of a transaction. They are only invoked by applications that interact with contracts for:
+Contract functions marked with `#[external("utility")]` are never proven as part of a transaction, even when called from a private function. They are invoked by applications that interact with contracts for:
 
 - **State queries**: Reading from both private and public state via an offchain client
 - **Local state management**: Modifying contract-related PXE state (e.g., processing logs in Aztec.nr)
 
+Utility functions can also be called from other utility functions, and from private functions as unconstrained code. Calls that cross a contract boundary require wallet authorization. See [utility calls](../aztec-nr/framework-description/calling_contracts.md#utility-calls) for how to make them.
+
 Since utility execution is unconstrained and relies heavily on oracle calls, no guarantees are made on the correctness of results. However, you can verify that the bytecode being executed is correct, since a contract's address includes a commitment to all of its utility functions.
 
 ### aztec.js

docs/docs-developers/docs/foundational-topics/pxe/execution_hooks.md

@@ -105,3 +105,7 @@ A general-purpose hook for *custom*, caller-defined requests. A contract reaches
 Pass a `resolveCustomRequest` hook when [creating the PXE](#configuring-hooks). It receives a `CustomRequest` with the issuing contract's address and class ID, the request `kind`, and the opaque `payload`, and returns the response. Because any contract can issue a request, the hook should check both the `kind` and the issuing contract before answering, dispatching on `kind` to the matching resolver.
 
 When the hook is absent, the request cannot be served and simulation fails.
+
+### Example: interactive handshakes
+
+The `HandshakeRegistry`'s `interactive_handshake` uses this hook to obtain the recipient's signed authorization. The payload carries what the signer needs to decide: who the recipient is, the handshake being authorized, and the chain context, but never the sender. The response carries what the registry needs to verify the recipient's signature in-circuit.

docs/docs-developers/docs/resources/migration_notes.md

@@ -9,6 +9,33 @@ Aztec is in active development. Each version may introduce breaking changes that
 
 ## TBD
 
+### [PXE] `pxe.updateContract` removed and `pxe.registerContract` no longer takes an artifact
+
+Registering classes and instances are now separate, unvalidated operations. `registerContractClass(artifact)` registers a class, `registerContract(instance)` registers an instance and no longer takes an artifact. `registerContract` does not check that PXE knows the contract's artifact: a missing artifact surfaces only when the contract is later simulated.
+
+**Migration:**
+
+- `pxe.registerContract` now takes the instance directly (its address preimage) and returns the derived address. Register the class separately via `registerContractClass`:
+
+```diff
+- await pxe.registerContract({ instance, artifact });
++ await pxe.registerContractClass(artifact);
++ await pxe.registerContract(instance);
+```
+
+  If you were calling it without an artifact, just drop the wrapping object: `pxe.registerContract({ instance })` becomes `pxe.registerContract(instance)`. The `wallet.registerContract(instance, artifact?, secretKeyOrKeys?)` convenience is unchanged and performs both registrations for you.
+
+- To make a new class's code available after an onchain upgrade, register the new artifact instead of calling `updateContract`:
+
+```diff
+- await pxe.updateContract(address, newArtifact);
++ await pxe.registerContractClass(newArtifact);
+```
+
+  The new class is used automatically once the upgrade takes effect on chain; no further PXE action is needed. Registering it beforehand is harmless: until the update activates, the node still resolves the contract's current class to the previous one, so it keeps running its old code.
+
+- `pxe.getContractInstance(address)` and `wallet.getContractMetadata(address).instance` now return the contract's **address preimage**, which no longer includes `currentContractClassId`.
+
 ### [Aztec.js] `AccountWithSecretKey` removed, read account keys from the `AccountManager` or PXE
 
 `AccountWithSecretKey` was a thin wrapper that bundled an account's transaction signer with its master secret key, used mainly to print or export the secret. It has been removed, and `AccountManager.getAccount()` now returns the plain `Account` signer. The wrapper's extra methods are no longer available on that value:
@@ -27,9 +54,11 @@ Aztec is in active development. Each version may introduce breaking changes that
 + const secretKey = accountManager.getSecretKey();
 ```
 
-To do what `AccountWithSecretKey` was meant for (exporting an account into a separate PXE or wallet), account registration now also accepts a full set of master secret keys instead of only a single seed. `wallet.registerContract(instance, artifact?, secretKeyOrKeys?)` and `pxe.registerAccount(secretKeyOrKeys, partialAddress)` take either an `Fr` (as before) or a `MasterSecretKeys` object (exported from `@aztec/aztec.js/keys`), and `pxe.getAccountSecretKeys(address)` returns those keys. These keys guard the account's privacy, so they should never be exposed to applications.
+To do what `AccountWithSecretKey` was meant for (exporting an account into a separate PXE or wallet), account registration accepts a full set of master secret keys instead of only a single seed. `wallet.registerContract(instance, artifact?, secretKeyOrKeys?)` takes either an `Fr` seed (as before) or a `MasterSecretKeys` object (exported from `@aztec/aztec.js/keys`), for an account whose privacy keys were generated independently rather than from one seed.
+
+The PXE never receives the seed nor the message-signing and fallback secret keys: it is not trusted to hold them. The wallet derives the account's privacy keys and passes the PXE only the four privacy secret keys (nullifier-hiding, incoming-viewing, outgoing-viewing, tagging) plus the message-signing and fallback *public* keys. Accordingly, `pxe.getAccountSecretKeys(address)` returns only those four privacy secret keys.
 
-**Impact**: Importing `AccountWithSecretKey`, or calling `getSecretKey()`/`getEncryptionSecret()` on the result of `getAccount()`, no longer compiles. The signer `getAccount()` returns is otherwise unchanged, and passing a single `Fr` to the registration methods keeps working.
+**Impact**: Importing `AccountWithSecretKey`, or calling `getSecretKey()`/`getEncryptionSecret()` on the result of `getAccount()`, no longer compiles. The signer `getAccount()` returns is otherwise unchanged, and passing a single `Fr` or a `MasterSecretKeys` to `wallet.registerContract` keeps working.
 
 
 ### [PXE] Unconstrained delivery defaults to a non-interactive handshake for external recipients

docs/docs-developers/docs/tutorials/js_tutorials/wallet-extension/04-accounts.md

@@ -44,7 +44,7 @@ export function deriveSigningKey(secretKey: Fr): GrumpkinScalar {
   return sha512ToGrumpkinScalar([secretKey, GeneratorIndex.IVSK_M]);
 }
 
-export function deriveMasterNullifierHidingKey(secretKey: Fr): GrumpkinScalar {
+export function deriveMasterNullifierHidingSecretKey(secretKey: Fr): GrumpkinScalar {
   return sha512ToGrumpkinScalar([secretKey, GeneratorIndex.NHK_M]);
 }
 ```

docs/examples/webapp-tutorial/src/fees.ts

@@ -30,7 +30,8 @@ export async function getSponsoredFPCContract() {
  */
 export async function registerSponsoredFPC(pxe: PXE) {
   const contract = await getSponsoredFPCContract();
-  await pxe.registerContract(contract);
+  await pxe.registerContractClass(contract.artifact);
+  await pxe.registerContract(contract.instance);
   return contract.instance.address;
 }
 // docs:end:register-fpc