[runtime, indexer, simplex]: add phantom order support for passive price and liquidity indexing by dharjeezy · Pull Request #983 · polytope-labs/hyperbridge

dharjeezy · 2026-06-19T22:25:28Z

Passive phantom-order price & liquidity indexing

What & why

Indexes exchange rates and available liquidity from the LPs running the simplex filler, without any real funds moving. The runtime periodically emits synthetic "phantom" orders; fillers bid on them as if they were real; we read those bids to learn each LP's price and on-chain liquidity for a token pair, and store a snapshot.

How it works (end to end)

Runtime (pallet-intents-coprocessor) — governance sets a PhantomOrderConfig (destination chain, token pairs, interval_blocks). Each interval, on_initialize generates one phantom Order per pair:
- same-chain, zero output amount (it only probes price), and a deadline set to the latest confirmed height so it reads as already expired — it can never be filled for real.
- emits PhantomOrderRegistered and offchain-indexes the ABI-encoded order.
- on_finalize emits PhantomBidWindowExhausted when a pair's bid window closes (the indexer's trigger to aggregate).
- place_bid enforces phantom rules (one bid per filler, within the window); a governance guard requires bid_window < interval_blocks. Default window: 5 blocks (gargantua), 25 (nexus).
Simplex filler — subscribes to PhantomOrderRegistered, fetches the exact order from offchain storage, the FX strategy quotes the pair (e.g. USDC→cNGN) at its full policy price, builds a real fillOrder UserOp through the normal contract-interaction pipeline, and submits the bid (retracting the prior interval's bid in the same utility.batch).
SDK (aggregatePhantomBids) — the shared aggregation: pulls live bids via intents_getBidsForOrder, simulates each fill via eth_simulateV1 against the destination chain (injects escrow/balance/allowance via state overrides, confirms an OrderFilled log), measures the solver's liquidity for the output token (native ERC-20 + ERC-4626 vault positions), and reduces the bids to a liquidity-weighted median plus the quote range. Lives in the SDK so the indexer and tests share one implementation.
Indexer (SubQuery) —
- handlePhantomOrderRegistered → PhantomOrder entity.
- handlePhantomOrderPrices (on PhantomBidWindowExhausted) → calls aggregatePhantomBids and persists the result. Gateway resolved via INTENT_GATEWAY_V3_ADDRESSES; token addresses normalized to 20-byte.

Data model (GraphQL)

PhantomOrder — the probe: chain, tokenA/tokenB, standardAmount, created-at block.
PhantomOrderPriceSnapshot — per window close: tokenA/tokenB, standardAmount, lowestPrice/medianPrice (liquidity-weighted)/highestPrice, bidCount. The rate = medianPrice / standardAmount (apply token decimals).
PhantomOrderLpBalance — per solver per snapshot: that LP's measured liquidity for the output token (keyed {commitment}-{blockNumber}-{solver} so it's a time series).

Testing

Pallet unit tests (hooks, bid rules, window < interval guard).
SDK unit tests for the aggregation helpers (weightedMedian, extractFillData, buildSimulationOrder, slot derivation incl. the 20/32-byte normalization).
Full E2E (test:phantom-filler-e2e): a locally-built simnode emits the order, three real IntentFillers watch + quote USDC→cNGN + submit, and aggregatePhantomBids produces the snapshot — simulated against an anvil fork of Base mainnet (real gateway/tokens + eth_simulateV1). Wired into test-sdk.yml. Verified locally: quotes 1500/1510/1520 cNGN → weighted median 1510.

closes #977

…er balance charge

…ent phantom order price snapshot indexing handlers

… and gateway state override for phantom order simulation

…balance The median price now weights each solver's quote by that solver's total balance for the output token (native ERC-20 + ERC-4626 vault venues, summed on the destination chain). Solvers that can actually deliver size move the median more than ones quoting on thin liquidity; zero-balance quotes have no influence. - Add weightedMedian(entries) to phantom-simulation.helpers, returning the lower weighted median and falling back to the unweighted median when every weight is zero so a price is still reported. - handlePhantomOrderPrices collects {price, weight} quotes (weight = the balance it already fetches per solver) and uses weightedMedian for medianPrice; lowest/ highest remain the raw quote range. - Update the medianPrice schema docstring and add weightedMedian unit tests.

…antom-order

The indexer's package.json gained "@hyperbridge/sdk": "workspace:*" (the phantom handlers import @hyperbridge/sdk/intents-helpers) but pnpm-lock.yaml was not regenerated, so CI's frozen-lockfile install failed with ERR_PNPM_OUTDATED_LOCKFILE. Regenerated the lockfile; the only change is linking the indexer to the sdk package.

…/phantom-order

…20-byte token addrs - PhantomOrderPriceSnapshot gains standardAmount (denormalized from PhantomOrder) so the exchange rate is computable from one row without joining back to the order. - PhantomOrderLpBalance id is now {commitment}-{blockNumber}-{solver}, making LP liquidity a time series keyed to the snapshot whose weighted median it fed (was last-write-wins per {token}-{solver}, which lost history and snapshot linkage). Current liquidity = the row with the greatest blockNumber per (solver, token). - Route every entity token-address field through bytes32ToBytes20 so all stored addresses are 20-byte lowercase hex and join consistently (PhantomOrder.tokenA/B, snapshot tokenA/B; LpBalance.tokenAddress already did).

Reverts the "generate intent-gateway-v2-addresses from config" approach: - Drop the intentGatewayV2 schema field (configs/index.ts) and the 7 config-mainnet entries (intentGatewayV3 values from main are preserved). - Remove generateChainsIntentGatewayV2Addresses() and its codegen call. - Restore intent-gateway-v2-addresses.ts as a tracked, committed file (removed from .gitignore) so the handler's INTENT_GATEWAY_V2_ADDRESSES import keeps resolving.

…rop v2 addresses - handlePhantomOrderPrices now resolves the gateway from INTENT_GATEWAY_V3_ADDRESSES (the V3 proxy deployment) instead of the standalone v2 address map. - Delete src/intent-gateway-v2-addresses.ts and gitignore it. - Reword the storage-slot comment to reference "the IntentGateway" generically. The only remaining IntentGatewayV2 reference is the SDK's ABI export used to decode/encode fillOrder and compute the order commitment; that is the gateway's actual interface (the V3 address is an upgradeable proxy to it) and the SDK has no V3 ABI to swap to.

…etraction - submitBidWithRetraction now uses utility.batch instead of batchAll. The retraction runs first, so the old deposit is reclaimed even if the new place_bid fails (batch interrupts on failure without reverting completed calls). - Revert enqueueRetraction to its original single-arg form (drop the trackInStorage parameter); the phantom flow batches its retraction via submitBidWithRetraction and no longer routes through enqueueRetraction.

… blocks (nexus) PhantomOrderBidWindowBlocks was HOURS in both runtimes; set the fallback to 5 on gargantua and 25 on nexus. Drop the now-unused HOURS import from each ismp.rs.

…e + anvil) Refactor (the agreed "do 1 first" foundation): - Move the phantom price/liquidity aggregation out of the SubQuery handler into phantom-aggregation.ts as aggregatePhantomBids(), free of SubQuery store/global coupling so it can run in an integration test. handlePhantomOrderPrices is now a thin wrapper that resolves endpoints, calls it, and persists the entities. Extended E2E (run locally; gated out of the default jest run): - src/test/phantom-indexer.e2e.test.ts drives the whole flow in-process against a hyperbridge simnode + an anvil fork of Base mainnet: the runtime emits a phantom order, three in-process fillers watch (subscribePhantomOrders) and submit real fillOrder bids with distinct prices (submitBid), and aggregatePhantomBids reads the bids, simulates each via eth_simulateV1 on the fork, measures USDC liquidity, and returns the snapshot (asserted: bidCount, low/high, weighted median, LP rows). - jest.config ignores *.e2e.test.ts by default; run via `test:phantom-e2e`. NOTE: not yet wired into CI and not validated locally in this environment (needs the simnode binary + anvil). Intended for local iteration first per the plan.

_orders' inner mapping is keyed by `address`, so the storage-slot key must be the token left-padded to 32 bytes (abi.encode(address)). The caller passes the input token as a 20-byte address (phantom.tokenA), so concatenating it raw produced a wrong escrow slot — harmless only because _withdraw skips zero amounts, leaving the escrow injection dead. Normalise via toHex(BigInt(token), {size: 32}), which handles both the 20-byte address and 32-byte token-field forms, so the injected escrow now lands at the slot the contract reads. Add a regression test asserting both forms derive the same slot.

…SDC) Replaces the indexer-side, hand-built-UserOp E2E with a simplex-package test that boots multiple real IntentFiller instances (modelled on createFxOnlyIntentFiller), each pointed at the simnode + an anvil fork of Base. They watch for the phantom order through their own phantom-bidding subscription, quote USDC->cNGN with the FX strategy, build the fillOrder UserOp via the full ContractInteractionService pipeline, and submit — i.e. simplex's complete bid-submission path, not reconstructed components. The test asserts every bid lands and is discoverable via intents_getBidsForOrder. - token pair USDC -> cNGN, standard amount 1 USDC (1_000_000). - run with: pnpm --filter @hyperbridge/simplex test:phantom-filler-e2e - removes the now-superseded indexer e2e test + its jest gating/script. The aggregation logic stays factored out in phantom-aggregation.ts (handler + unit tests cover it). NOTE: not validated in this environment (needs the simnode binary + anvil); first cut for local iteration.

…x E2E Moves aggregatePhantomBids + the phantom simulation helpers from the indexer into @hyperbridge/sdk (exported from the VM2-safe intents-helpers entry), with the two indexer-specific config maps — per-token ERC-20 slots and the ERC-4626 vault map — passed in as parameters. This lets both the indexer handler and the simplex E2E share one implementation (the earlier cross-package @/-alias collision is gone). - sdk: new src/protocols/intents/phantom-aggregation.ts; re-exported from intents-helpers (bytes32->address inlined to keep that entry light). - indexer: handler imports aggregatePhantomBids from the SDK and passes TOKEN_SLOT_OVERRIDES + YIELD_VAULT_ADDRESSES; deletes the local copies; the helper unit test imports from the SDK and threads the overrides map. - simplex: the phantom-filler E2E now also runs aggregatePhantomBids on the real bids and asserts the snapshot (bidCount, price range, weighted median, cNGN LP balances) — full flow in one test. Validated locally: SDK build (incl. dts type-check) passes; runtime check of the moved functions from the built bundle passes. The simnode+anvil E2E still needs those services to run.

set_phantom_order_config rejects a config whose effective bid window is not strictly shorter than interval_blocks (10). The test set the window to 100, so the sudo'd config call failed silently (reported via Sudid, not ExtrinsicFailed) and no phantom order was generated. Use a window of 5 (< 10). Verified end to end against a local gargantua-1000 simnode + anvil fork of Base: three real IntentFillers watch, quote USDC->cNGN, and submit bids, and aggregatePhantomBids reduces them to a snapshot.

…into CI - fx: quotePhantomFill capped each output at the order's requested amount × (1 + maxOverfillBps). Phantom orders request a zero output (price probe), so the ceiling was 0 and every quote collapsed to 0. Skip the cap when output.amount == 0 and quote the full policy output. Verified: three fillers now quote 1500/1510/1520 cNGN. - phantom E2E: log each solver's decoded cNGN quote and the aggregation snapshot (bidCount, low/median/high, per-LP balances); assert lowestPrice > 0 so a zero-quote regression can't pass silently. - ci: add a phantom-filler-e2e job to test-sdk.yml (arc-runner-set) that builds the simnode, forks Base with anvil, and runs test:phantom-filler-e2e. Validated locally end to end: simnode (gargantua-1000) + anvil fork of Base; three real IntentFillers watch, quote USDC->cNGN, submit; aggregatePhantomBids reduces them to a snapshot (median 1510 cNGN, weighted by cNGN liquidity).

…l cap; install Go in E2E CI - intents-helpers: re-export HexString so the indexer's handler can import it. The type was bundled into the entry's d.ts as a local (unexported) declaration, so `import { type HexString }` failed the indexer's subql build with TS2459. Verified with ENV=local indexer build (clean). - fx: remove the overfill ceiling in quotePhantomFill entirely. Phantom orders request a zero output, so there is no requested amount to cap against; quote the full policy output. (Behaviourally identical to the prior zero-output skip — quotes stay e.g. 1500/1510/1520 cNGN.) - ci: install Go in the phantom-filler-e2e job; the node build's sp1-recursion-gnark-ffi build script compiles a Go library and fails without it.

…unit test - docker-compose.simnode.yml: never wired up (no simnode indexer project generation); the phantom E2E uses an in-process simnode + anvil instead. - phantom-simulation.helpers.test.ts: the helpers it covered now live in the SDK (@/protocols/intents/phantom-aggregation).

The helpers (extractFillData, buildSimulationOrder, tokenSlots, hasTokenSlotOverride, ordersStorageSlot, erc20*Slot, weightedMedian) moved to the SDK, so their unit tests move with them. Replaces the indexer's generated TOKEN_SLOT_OVERRIDES with a local fixture (the functions now take the overrides map as a parameter). Runs under the SDK's test:concurrent. Verified: 19 tests pass.

issues fixed

Record the destination chain where each LP balance was measured and include it in the id ({chain}-{commitment}-{blockNumber}-{solver}), so balances are unambiguous and queryable per chain. Verified with an ENV=local indexer build (codegen + subql build).

Previously aggregatePhantomBids measured each bidding solver's balance only for the phantom order's output token on its destination chain. Now it sweeps the solver's liquidity for every token configured in yieldVaults across every supported EVM chain (the indexer builds the per-chain RPC map from ENV_CONFIG's EVM-* entries), recording one PhantomOrderLpBalance per (solver, chain, token). The liquidity-weighted median is unchanged — still weighted by the output-token balance on the destination chain. - sdk: aggregatePhantomBids takes evmRpcUrls (chain -> rpc) instead of a single evmRpcUrl; new sweepSolverLiquidity over yieldVaults x chains. LpBalance gains `chain`. - indexer: build evmRpcUrls from ENV_CONFIG; id now {chain}-{tokenAddress}-{commitment}- {blockNumber}-{solver}. - e2e: pass evmRpcUrls + a cNGN yieldVaults entry; assert lp.chain. Verified: SDK + indexer (ENV=local) builds clean; simplex typechecks; full simnode+anvil E2E green (3 LP balances swept on EVM-8453, median 1510).

Only record a PhantomOrderLpBalance for tokens a solver actually holds, so the snapshot isn't padded with zero rows for every configured (chain, token) the solver is empty on.

… input-token unit standard_amount is the denominator of every published rate (medianPrice / standard_amount) and fails silently if wrong — a wrong value poisons every snapshot for the pair by an integer factor without reverting. Document loudly on PhantomTokenPair::standard_amount that it must be exactly 10^decimals(token_a), with a pointer from set_phantom_order_config.

Delete parachain/simtests/src/phantom_orders.rs and its module declaration.

add phantom order support for passive price and liquidity indexing

62c3163

Wizdave97 reviewed Jun 22, 2026

View reviewed changes

Comment thread modules/pallets/intents-coprocessor/src/lib.rs Outdated

Wizdave97 reviewed Jun 22, 2026

View reviewed changes

Comment thread modules/pallets/intents-coprocessor/src/lib.rs

Wizdave97 reviewed Jun 22, 2026

View reviewed changes

Comment thread ...packages/indexer/src/handlers/events/substrateChains/handlePhantomOrderRegistered.handler.ts

Wizdave97 reviewed Jun 22, 2026

View reviewed changes

Comment thread sdk/packages/indexer/src/handlers/events/substrateChains/handlePhantomBidPlaced.handler.ts Outdated

dharjeezy added 4 commits June 22, 2026 17:20

generate phantom order in pallet hook, governance config, remove solv…

e26564e

…er balance charge

update simnode indexer

4460afb

simulate bid

6eb80a1

merge main into dami/phantom-order

f00dad3

dharjeezy requested a review from Wizdave97 June 22, 2026 16:56

Wizdave97 reviewed Jun 22, 2026

View reviewed changes

Comment thread modules/pallets/intents-coprocessor/src/lib.rs Outdated

Wizdave97 reviewed Jun 22, 2026

View reviewed changes

Comment thread sdk/packages/simplex/src/services/ContractInteractionService.ts

dharjeezy added 4 commits June 23, 2026 13:25

expand PhantomOrderRegistered event with token pair fields and implem…

eb273ac

…ent phantom order price snapshot indexing handlers

doc schemas

3d34405

use SDK exposed abis and functions

d23b75d

rev

ce46fcb