fix(token-swap): compute amm math in u128 to avoid I64F64 overflow

[dev-jodee]

Summary

• deposit_liquidity, swap_exact_tokens_for_tokens, and withdraw_liquidity multiplied two u64 token amounts as I64F64, whose integer part maxes at 2^63-1. checked_mul().unwrap() therefore panics once the product reaches 2^63 — roughly 3 tokens per side at 9 decimals — so larger deposits, swaps and withdrawals fail.
• Compute the multiply/divide/sqrt in u128 instead (two u64s multiply to < 2^128), using u128::isqrt for the liquidity sqrt. Same floor results for valid inputs, overflow-free across the full u64 range.
• Drop the now-unused fixed dependency.

Test Plan

• anchor build succeeds locally (anchor-cli 1.0.2; exit 0, no errors/unused warnings; program + IDL built).
• Floor-preserving: I64F64::from_num(a*b).sqrt().to_num::<u64>() == (a*b).isqrt(); the I64F64 mul/div(+floor) paths == u128 integer mul/div for non-negative token amounts.
• Note: token-swap/anchor is listed in .github/.ghaignore ("can't test on localnet"), so CI won't exercise it — verified via local build + reasoning.

Closes #73

[greptile-apps]

Greptile Summary

This PR replaces I64F64 fixed-point arithmetic with u128 integer math in deposit_liquidity, swap_exact_tokens_for_tokens, and withdraw_liquidity, fixing a real overflow that caused panics for token amounts above roughly 3 tokens at 9 decimals. The fixed crate dependency is also removed.

• deposit_liquidity / withdraw_liquidity: The u128 migration is correct — products of two u64 values fit in u128, and the isqrt/division results never exceed u64::MAX for valid inputs, making the as u64 casts safe.
• swap_exact_tokens_for_tokens: The output calculation is correctly migrated, but the pre/post-trade invariant check (let invariant = pool_a.amount * pool_b.amount on line 55) still uses unguarded u64 multiplication and can overflow on large pools. Additionally, the post-trade comparison on line 129 uses pool_account_a.amount * pool_account_a.amount (squaring pool A) instead of pool_account_a.amount * pool_account_b.amount, making the safety guard logically meaningless for asymmetric pools.
• lib.rs: declare_id! was changed to a new program address without any mention in the PR description, which is a breaking change for existing deployments or client tooling referencing the old ID.

Confidence Score: 3/5

The overflow fix in the deposit and withdraw paths is correct, but the swap invariant check remains broken in two ways — overflow-prone u64 multiplication and a typo that compares against pool_a² instead of the true post-trade product — leaving the primary safety mechanism for swaps unreliable.

The deposit and withdraw changes are solid and solve the stated problem. However, swap_exact_tokens_for_tokens still has two defects in the invariant check: the plain u64 multiply on line 55 can overflow silently in release mode, and line 129 reads pool_account_a.amount * pool_account_a.amount when it should read pool_account_a.amount * pool_account_b.amount. Together these mean the constant-product safety guard can silently pass even when k decreases.

tokens/token-swap/anchor/programs/token-swap/src/instructions/swap_exact_tokens_for_tokens.rs — the invariant lines (55 and 129) need to be migrated to u128 and the pool_account_a typo on line 129 must be corrected.

Important Files Changed

Filename	Overview
tokens/token-swap/anchor/programs/token-swap/src/instructions/swap_exact_tokens_for_tokens.rs	Output calculation migrated from I64F64 to u128 (overflow fix), but the pre/post-trade invariant lines still use unchecked u64 multiplication and contain a typo (pool_account_a squared instead of pool_account_a × pool_account_b), leaving the safety guard broken.
tokens/token-swap/anchor/programs/token-swap/src/instructions/deposit_liquidity.rs	Liquidity calculation switched from I64F64 to u128 + isqrt(); the product of two u64 values fits in u128 and isqrt never exceeds u64::MAX, so the as u64 cast is safe. No issues found.
tokens/token-swap/anchor/programs/token-swap/src/instructions/withdraw_liquidity.rs	Withdrawal amounts computed in u128; result is always ≤ pool balance (u64) when amount ≤ supply, so the as u64 cast is safe. No issues found.
tokens/token-swap/anchor/programs/token-swap/Cargo.toml	Removed the now-unused fixed dependency; no other changes.
tokens/token-swap/anchor/programs/token-swap/src/lib.rs	declare_id! updated to a new program address without explanation in the PR description; this is a breaking change for any existing deployments or client code referencing the old ID.

Comments Outside Diff (2)

1. tokens/token-swap/anchor/programs/token-swap/src/instructions/swap_exact_tokens_for_tokens.rs, line 55 (link)

   Invariant calculation uses unchecked u64 multiplication

   pool_a.amount * pool_b.amount is a plain u64 multiplication. When both pool reserves exceed ~4.3 billion tokens (easily reachable at 9-decimal precision), this overflows: in a debug build it panics, in a release build it silently wraps to a smaller value. A wrapped invariant is then smaller than the true value, so the post-trade check (line 129) may pass even when k decreased — defeating the safety guard entirely. The PR fixes overflow in the output formula on the same call path but leaves this multiplication unchanged.

2. tokens/token-swap/anchor/programs/token-swap/src/instructions/swap_exact_tokens_for_tokens.rs, line 129 (link)

   Invariant check compares against pool_account_a squared instead of pool_account_a × pool_account_b

   The condition invariant > ctx.accounts.pool_account_a.amount * ctx.accounts.pool_account_a.amount checks pre-trade a × b against post-trade a², not the actual post-trade product a × b. For any pool where token A and token B are worth different amounts, these quantities are entirely unrelated, so the check can pass even when the constant-product invariant is genuinely violated. The final operand should be ctx.accounts.pool_account_b.amount. This line also still uses unchecked u64 multiplication (same overflow risk as line 55).

_{Reviews (1): Last reviewed commit: "fix(token-swap): compute amm math in u12..." | Re-trigger Greptile}