Release#174
Merged
Merged
Conversation
Improve performance
…ocomplete support
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
…t-ranking refactor: adjust popular post ranking
Token-list eligibility, saveAllActiveAccounts, leaderboard active_accounts,
and trending metrics all filter tx_type IN ('buy','sell') grouped by
sale_address/address with no covering index, forcing sequential scans of
the largest table. Adds a general composite index plus two partial indexes
matching the exact predicate for the GROUP BY aggregate paths.
applyListEligibilityFilters and getTrendingEligibilityBreakdown joined an unbounded `GROUP BY sale_address` aggregate over the whole transactions table on every request (post counts were already materialized, trade counts were not). Adds a token_trade_eligibility_counts table, backfilled once in the migration and kept current by an incremental upsert in TransactionPersistenceService.saveTransaction (guarded against double counting on reorg/backward-sync replay of an already-persisted tx_hash).
TransactionService.saveTransaction's broadcast branch (syncTokenPrice, holder resync via the old updateTokenHolder incl. a full mdw re-fetch of every holder, trending score update) only ran when shouldBroadcast was true, and none of its three remaining callers (fix-failed-transactions, fast-pull-tokens, the by-hash controller fallback) ever pass it. Live buy/sell traffic already goes through TransactionProcessorService / TokenHolderService, which updates one holder row and never re-syncs the full set. Removes the unreachable branch and the now-orphaned deprecated method instead of leaving a landmine for a future caller.
The frontend paged a receiver's entire middleware transaction history in the browser to find TIP_POST:<id> payloads. Tip already has an indexed post_id column (IDX_TIPS_POST_ID); expose it as a query filter.
The frontend called middleware once per invitee to detect redeem txs. The invitation-redeem processor already records claim data on the row (invitee_address, status, status_updated_at, claim_tx_hash); adds an `inviter` filter to GET /invitations and derives claimed/claimer_address/ claimed_at/claim_tx_hash on each item so the frontend can fetch a single page instead of one middleware call per invitee.
The frontend fetched {mdw}/v3/aex9/{address} from the browser per token
page just for event_supply. Fetches and persists it as circulating_supply
during getTokeLivePrice (token sync / PullTokenInfoQueue) and exposes it
on the token DTO instead.
The frontend issued listTokenHolders({limit:1}) just to read meta.totalItems
for the "owned tokens" stat badge. Adds a cheap indexed COUNT(*) FROM
token_holders WHERE address = :addr AND balance > 0 alongside the existing
profile lookup in getAccount.
The frontend polled the heavy portfolio-history endpoint every 60s just to read the latest snapshot. getPortfolioHistory buckets its AE-node balance lookup to a 300-block window (fine for a daily chart, too stale for a live ticker), so this is a separate, cheaper path: current height/balance read directly, current token value from a single-height PnL batch call. Adds GET /accounts/:address/portfolio/value?convertTo=usd, cached 30s.
Every debounced keystroke fired 3 parallel requests (tokens, accounts, posts) from the frontend. Adds GET /search?q=&limit=, running the three lookups in parallel server-side (accounts reuses the existing AccountService.searchByNameOrAddress), each capped at limit and cached 30s.
computeEventItems called calculateTokenPnlsBatch once per candidate address (concurrency 8, cap 500) even though every candidate shares the same sample heights. Adds calculateTokenPnlsBatchForAddresses, which adds address as a grouping dimension to the same MATERIALIZED-CTE aggregation so the whole candidate set resolves in one query instead of up to 500.
The frontend fetched posts + token-creations + trades separately, then deduped/merged/sorted client-side with complex pagination. Adds GET /feed?sort=latest|hot&cursor=&limit=: - sort=latest: keyset-paginated UNION of posts, token creations, and trades (each source queried independently ordered by created_at DESC, merged and re-sliced in application code -- standard federated-keyset technique). - sort=hot: posts only via the existing PopularRankingService, since trades and token-creations are never ranked under "Hot" in the current product. Adds an index on token(unlisted, created_at), the only new query path introduced (token.created_at previously had no covering index).
Saving a post synchronously ran a RECURSIVE thread CTE + tips + reads aggregation (calculateTokenTrendingMetrics) per mentioned symbol on the request path. Adds an UpdateTrendingScoresQueue and TokensService.queueTrendingScoresForSymbols, deduped by symbol (jobId) with a 5s delay so a burst of posts mentioning the same symbol collapses into one recompute; post.service.ts now enqueues instead of awaiting the recompute inline.
createOrGetTopics looped a findOne + save per topic name (a post can mention several). Normalizes and dedupes the requested names, bulk-inserts any missing ones with a single INSERT ... ON CONFLICT (name) DO NOTHING (never clobbers an existing topic's accumulated post_count), then reads them all back with one IN() query. Scope note: this covers the topics half of the plan's batch-ingestion item. The other half (batching the per-transaction existence check and comment- count recompute across a whole middleware page) would need threading a per-page batch context through savePostFromTransaction and its private helpers; pullLatestPostsForContracts already runs loadPostsFromMdw concurrently per contract, so that context can't safely be shared instance state. Left alone rather than rushing a cross-cutting signature change.
queryTokensWithRanks ran RANK() OVER (ORDER BY market_cap ...) across the whole unlisted=false token table on every list request, regardless of the requested ordering. Adds a persisted rank column, backfilled in the migration and refreshed every 5 minutes by RefreshTokenRanksService; the query now reads token.rank as a plain column instead of recomputing the window function inline. Scope note: this covers the "persisted rank" half of the plan item. Keyset pagination and separately caching the filtered-count are left for a later pass -- both would change the tokens list API contract (cursor vs page, and a cache keyed per filter combination), which is a bigger and more disruptive change than the rank fix alone.
getChainNameForAccount verified each candidate .chain name against middleware with one sequential fetchJson call, so an account with several candidates (or one slow lookup) serialized the whole resolution. Extracts PortfolioService's private mapWithConcurrency into a shared util and uses it here with concurrency 8 and a 5s AbortSignal timeout per call (tighter than fetchJson's 30s default), reusing the existing fallback-to-historical- data behavior on any per-name failure.
migration:run defaults to transaction: 'all' (one batch transaction for every pending migration) since migrationsTransactionMode is not configured on the DataSource. CONCURRENTLY cannot run inside a transaction, so these 3 migrations declared `transaction = false` to opt out individually -- but TypeORM rejects a per-migration override that conflicts with a global 'all' batch (ForbiddenTransactionModeOverrideError), so the whole migration chain would have failed to run in any environment using the default mode (confirmed via the TGR eligibility integration suite, which runs real migrations against Postgres). Fixes by dropping CONCURRENTLY and the transaction override, matching this migration folder's existing convention (plain CREATE INDEX IF NOT EXISTS, see QueryHotPathIndexes) instead of introducing a new one.
recomputeRoomFromHolders and recomputeRoom already batched the token_balance and existing-membership reads, but still resolved each member's nostr pubkey via one IdentityService.getPubkeyForAddress DB read per member -- O(N) reads for a whole-room recompute with thousands of holders (identity service's cache only helps on repeat lookups, and each holder is normally looked up once). Adds IdentityService.getPubkeysForAddresses (one IN(...) read for every address not already cached) and threads a pubkeyOverride through recomputeMember, mirroring the existing balanceRaw/existing parameters, so the two batch/cursor recompute paths resolve pubkeys once per room/page instead of once per member. Scope note: this covers the read half of the plan item. Bulk-upserting the membership writes would need recomputeMember to stop writing per-member (it currently persists + emits tgr.eligibility.changed inline), which is shared by several single-member reactive call sites -- left alone rather than restructuring that per-member write/emit contract in this pass.
saveAllActiveAccounts (a full GROUP BY over the whole transactions table) ran unconditionally on every process start/restart when PULL_ACCOUNTS_ENABLED is on. ensureAccountFromTransactions already keeps individual accounts fresh incrementally, so the full rebuild only needs to run periodically as a consistency catch-up sweep -- moved to a daily 3am cron instead of onModuleInit.
processDueClaims processed up to 50 due claims sequentially (one on-chain claim at a time) every 30s. processClaimWithGuard already dedupes per-address, so different claims in the same batch are safe to run concurrently -- switched to mapWithConcurrency with concurrency 5.
fixOrphanedComments looped a per-comment validateParentPost read + a getCount()/UPDATE via updatePostCommentCount, in batches of 500. Replaces it with two set-based statements: one UPDATE unlinks every comment whose parent is still missing (post_id = NULL, converting it to a standalone post), and one UPDATE (via a single GROUP BY) recomputes total_comments for every parent whose count is stale -- which also catches up parents that only just appeared, since their comment-count update was a no-op while the parent row didn't exist yet (the actual race this cleanup exists to catch).
page size was already capped at 100, but page itself had no ceiling -- ?page=100000 forces Postgres to scan and discard every prior row before returning anything. Adds a page cap (500) to both the REST base controller and the GraphQL base resolver's shared validator.
…rer uses
buildScoredItems loaded full Post entities (every column: media, tx_args,
token_mentions, ...) plus the topics relation for up to 10k candidates on
every recompute (every 10 min, plus on-demand cold-cache fallback), but
the scoring pass only ever reads id/content/created_at and each topic's
name. Replaces the find({relations:['topics']}) call with a query builder
selecting just those columns.
…folioValue getCurrentPortfolioValue loads the AE wallet balance at chain tip (inclusive of the current block) but valued token holdings via calculateTokenPnlsBatch at snapshot_height = currentHeight, where the holdings aggregation uses the strict rule `tx.block_height < snapshot_height`. A trade mined in the current block was therefore reflected in the AE balance but omitted from token holdings, so a poll right after latest-block activity returned an inconsistent total. Value holdings at currentHeight + 1 so `block_height < currentHeight + 1` includes the current block, matching the balance. For integers this is identical to the old behavior for every address except those with a tx at exactly the tip block, so it only corrects the inconsistent case. The price lateral joins (`block_height <= snapshot_height`) are unaffected since no rows exist above currentHeight yet.
Improve performance
feat: web notification
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 76f27ab. Configure here.
| const seenIds = | ||
| cursor && cursor.ts === cutoff | ||
| ? [...new Set([...(cursor.seenIds ?? []), ...idsAtCutoffThisPage])] | ||
| : idsAtCutoffThisPage; |
There was a problem hiding this comment.
Feed seenIds grow without bound
Low Severity
When many items share the same created_at, seenIds is merged with the previous cursor’s set so ties can span pages. That set is not capped, so a large same-timestamp batch makes the cursor and every source’s NOT IN (:...excludeIds) clause grow without bound, despite the cursor util claiming it stays bounded by the page limit.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 76f27ab. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
High Risk
Touches notification auth/storage, broad schema migrations on hot tables, and proxy-trust configuration that affects rate limiting and IP-based limits if mis-set in production.
Overview
Large release that adds web in-app notification feed + Web Push (VAPID) plumbing (new
notifications/web_push_subscriptionsschema, env knobs,web-pushdependency) and documents the/notificationswebsocket namespace.Introduces a server-side home feed (
GET /api/feed) that merges posts, token creations, and trades with keyset cursors (latest) or popular-ranking offsets (hot), plus feed ranking tweaks (longer freshness window, stale penalties, daily score shuffle).Accounts & portfolio: typeahead
GET /accounts/search, batchGET /accounts/chain-names,holdings_counton account detail, lightweightGET :address/portfolio/value, and leaderboard event PnL batched across addresses in one SQL query instead of per-address fan-out.Ops & HTTP: optional
TRUST_PROXY+ sharedresolveClientIpfor Socket.IO handshakes, gzip viacompression, and deep pagination caps (default page ≤ 500, raised for MDW mirror tables).Performance & schema: many migrations (tips/token/transaction indexes, materialized token trade eligibility counts, persisted token
rankandcirculating_supply, GIN on posttoken_mentions), token list eligibility no longer aggregates trades on every request, AE pricing / DEX path-finding / CoinGecko fallback caching, and in-memory cache LRU 2k → 10k.Smaller API additions: invitations inviter filter with derived claim fields;
SearchModulewired inAppModule(full search surface may live outside this diff chunk).Reviewed by Cursor Bugbot for commit 76f27ab. Bugbot is set up for automated code reviews on this repo. Configure here.