Gateway controllers return incorrect data in GET /bot-orchestration/{bot_name}/history

[fengtality]

Description

The GET /bot-orchestration/{bot_name}/history endpoint returns incorrect data for Gateway-related controllers (AMM/LP strategies like Meteora CLMM).

Issues

1. Trade Fee schema shows incorrect token

The trade_fee.flat_fees[].token field shows the base asset (e.g., PERCOLATOR) instead of the native currency (e.g., SOL). Transaction fees on Solana are paid in SOL, not in the base token.

2. Only RANGE trades are returned

The endpoint only returns RANGE trade types, which correspond to positions being opened and closed. This misses the actual trading activity within the LP position.

This is the reason Hummingbot has a separate lphistory command instead of history for LP strategies - the standard history format doesn't capture LP-specific data correctly.

Example Response

curl -X 'GET' \
  'http://localhost:8000/bot-orchestration/blah-20260304-155319/history?days=0&verbose=false&timeout=30' \
  -H 'accept: application/json'

{
  "status": "success",
  "response": {
    "success": true,
    "data": {
      "data": {
        "trades": [
          {
            "market": "meteora/clmm",
            "trade_id": "5zCCyPKspk6kawk9Z2T7PMyW9GSVwSSShLgYDSc2dYKV2XAcxoAHBzjeoyBWT4vWj2iSqpLBwqAHZ2mm8Jk8M1XF",
            "price": "0.000011",
            "quantity": "345171.910192",
            "symbol": "PERCOLATOR-SOL",
            "trade_type": "RANGE",
            "base_asset": "PERCOLATOR",
            "quote_asset": "SOL",
            "raw_json": {
              "trade_fee": {
                "flat_fees": [
                  {
                    "token": "PERCOLATOR",  // ❌ Should be "SOL" (native currency)
                    "amount": "0.00007963700000000001"
                  }
                ]
              }
            }
          }
        ]
      }
    }
  }
}

Expected Behavior

1. trade_fee.flat_fees[].token should reflect the actual fee token (native currency like SOL for Solana)
2. Consider adding an lphistory endpoint or extending the history endpoint to return LP-specific data (fees earned, impermanent loss, etc.)

Environment

• Controller type: Gateway LP controller (e.g., conf_generic.lp_rebalancer_PERCOLATOR.yml)
• Connector: meteora/clmm

[fengtality]

Analysis

After investigating, this issue cannot be fixed in hummingbot-api alone. Here's why:

Root Cause

The /bot-orchestration/{bot_name}/history endpoint sends an MQTT history command to the Hummingbot client and returns the response. The data issues (incorrect fee token, missing LP trades) originate from:

1. TradeFill model - The standard history uses TradeFill records which store fees differently than LP operations
2. RANGE trade type - LP positions are stored as RANGE type trades, not individual swaps

Hummingbot Client Already Has the Solution

The client has a separate lphistory command (hummingbot/client/command/lphistory_command.py) that:

• Reads from RangePositionUpdate model (LP-specific data)
• Shows correct fees (base_fee, quote_fee, trade_fee_in_quote)
• Calculates proper P&L, fees collected, etc.

However, lphistory is not exposed via MQTT - only the regular history command is.

Required Changes

1. Hummingbot client (hummingbot repo):

• Add MQTT handler for lphistory command in hummingbot/remote_iface/mqtt.py
• Similar to existing history command handler

2. Hummingbot API (this repo):

• Add new endpoint GET /bot-orchestration/{bot_name}/lphistory
• Call the new MQTT lphistory command

Workaround

Until the client changes are made, users can:

1. Use the lphistory command directly in the Hummingbot terminal UI
2. Query the SQLite database directly for RangePositionUpdate records

Would you like me to create a PR for the client-side changes in the hummingbot repo?

[fengtality]

Analysis: LP History Data Architecture

Root Cause

• CEX strategies: Write to TradeFill table → MQTT history command reads it ✅
• LP strategies: Write to RangePositionUpdate table → MQTT history command misses it ❌

The Hummingbot client has a separate lphistory command that reads RangePositionUpdate, but it's not exposed via MQTT.

---

Option 1: Modify TradeFill to Handle LP Events

Work Required:

1. Modify hummingbot/model/trade_fill.py - add LP-specific fields (position_address, base_fee, quote_fee, lower_price, upper_price, etc.)
2. Modify hummingbot/connector/gateway/gateway_lp.py - emit TradeFill events for LP operations
3. Modify hummingbot/connector/markets_recorder.py - handle LP event → TradeFill conversion
4. Modify hummingbot/client/command/history_command.py - display LP data in appropriate format
5. Data migration for existing RangePositionUpdate records

Tradeoffs:

Pros	Cons
Single table for all history	Schema mismatch - TradeFill has amount/price/trade_type(BUY/SELL), LP has base_amount/quote_amount/order_action(ADD/REMOVE)
No MQTT handler changes	Breaking change for existing TradeFill consumers
API history endpoints work for all strategies	Semantic mismatch - LP events aren't "trades", they're position management
	Significant testing burden - must verify all CEX strategies still work
	Schema coupling - LP feature evolution affects CEX history

Maintainability: Poor - combines two conceptually different domains into one table, increasing complexity and coupling.

---

Option 2: Add lphistory to MQTT (Recommended)

Work Required:

Hummingbot client changes:

1. Add LPHistoryCommandMessage in hummingbot/connector/commlib/messages.py
2. Add /lphistory handler in hummingbot/remote_iface/mqtt.py (similar pattern to history)
3. Add get_lp_history_json() method (expose existing lphistory_command.py logic)

hummingbot-api changes:
4. Add lphistory MQTT command to bot orchestrator
5. Either:

• Add separate /bots/{instance}/lphistory endpoint, OR
• Modify history to auto-detect strategy type and call appropriate command

Tradeoffs:

Pros	Cons
Clean separation - trades vs LP positions	API needs to know which endpoint to call
No breaking changes to existing functionality	Two MQTT commands instead of one
Semantic clarity - each domain optimized for its use case	Controllers need to specify strategy type
Independent evolution - LP and CEX can evolve separately
Lower risk - existing CEX strategies untouched

Maintainability: Good - clear separation of concerns, each domain evolves independently, easier to understand and debug.

---

Recommendation

Option 2 (Add MQTT lphistory) is the better choice:

1. Lower risk - No changes to TradeFill means no risk of breaking CEX strategies
2. Better architecture - LP operations and CEX trades are fundamentally different; keeping them separate is correct
3. Less work - Adding an MQTT handler is straightforward; modifying TradeFill would require significant schema changes
4. Future-proof - As LP strategies evolve (new chains, new DEX types), the RangePositionUpdate schema can evolve independently

The main downside (API needing to know strategy type) can be mitigated by having the API detect the controller type from the bot config and call the appropriate MQTT command automatically.

---

Next Steps

This fix requires changes to the hummingbot client first (adding MQTT lphistory handler), then hummingbot-api can consume it. Tagging as hummingbot-client dependency.