Skip to content

PT-2516 - enhance pt-mongodb-index-check unused and duplicate analysis#1132

Open
jenunes wants to merge 1 commit into
percona:3.xfrom
jenunes:PT-2516_enhance_pt_mongodb_index_check
Open

PT-2516 - enhance pt-mongodb-index-check unused and duplicate analysis#1132
jenunes wants to merge 1 commit into
percona:3.xfrom
jenunes:PT-2516_enhance_pt_mongodb_index_check

Conversation

@jenunes

@jenunes jenunes commented May 28, 2026

Copy link
Copy Markdown

Multi-signal unused index scoring:

  • Collect $indexStats, listIndexes, collStats, and serverStatus
  • Score indexes into tiers: SAFE_TO_DROP, LIKELY_UNUSED, LOW_USAGE, MONITOR, KEEP
  • Hard guards for id, unique, TTL, hidden, partial, and system indexes
  • Warmup period detection (configurable --warmup-days)
  • Cross-reference duplicate prefix indexes with container usage

Duplicate detection improvements:

  • keyToken distinguishes hashed/text/2dsphere/2d from btree directions
  • Handle primitive.Symbol decoding (MongoDB 8.x)
  • Skip id as a droppable prefix candidate
  • Property-aware: partialFilterExpression, sparse, collation checked
  • Unique prefix warning when dropping would lose uniqueness
  • Rich sectioned text report with Reason, Action, optional index sizes

Sharded cluster support:

  • Aggregate $indexStats across shards (sum ops, oldest since)
  • NormalizeIndexStat for mongos responses missing top-level name/key
  • DeduplicateIndexRecords safety net

Connection and UX:

  • Ping after connect for immediate auth/network errors
  • Require --databases or --all-databases (or DB in URI)
  • Skip system databases (admin, config, local) and system.* collections
  • collStats IndexSizes uses dynamic map for arbitrary index names

Documentation:

  • README.rst rewritten with new flags, sample output, sharded behavior, edge cases (warmup, read preference, background builds), PTDEBUG usage

Tests:

  • 41 unit/regression tests covering scoring, duplicate detection, report rendering, system DB filtering, URI parsing, and sharded aggregation

  • The contributed code is licensed under GPL v2.0

  • Contributor Licence Agreement (CLA) is signed

  • util/update-modules has been ran

  • Documentation updated

  • Test suite update

Multi-signal unused index scoring:
- Collect $indexStats, listIndexes, collStats, and serverStatus
- Score indexes into tiers: SAFE_TO_DROP, LIKELY_UNUSED, LOW_USAGE, MONITOR, KEEP
- Hard guards for _id_, unique, TTL, hidden, partial, and system indexes
- Warmup period detection (configurable --warmup-days)
- Cross-reference duplicate prefix indexes with container usage

Duplicate detection improvements:
- keyToken distinguishes hashed/text/2dsphere/2d from btree directions
- Handle primitive.Symbol decoding (MongoDB 8.x)
- Skip _id_ as a droppable prefix candidate
- Property-aware: partialFilterExpression, sparse, collation checked
- Unique prefix warning when dropping would lose uniqueness
- Rich sectioned text report with Reason, Action, optional index sizes

Sharded cluster support:
- Aggregate $indexStats across shards (sum ops, oldest since)
- NormalizeIndexStat for mongos responses missing top-level name/key
- DeduplicateIndexRecords safety net

Connection and UX:
- Ping after connect for immediate auth/network errors
- Require --databases or --all-databases (or DB in URI)
- Skip system databases (admin, config, local) and system.* collections
- collStats IndexSizes uses dynamic map for arbitrary index names

Documentation:
- README.rst rewritten with new flags, sample output, sharded behavior,
  edge cases (warmup, read preference, background builds), PTDEBUG usage

Tests:
- 41 unit/regression tests covering scoring, duplicate detection,
  report rendering, system DB filtering, URI parsing, and sharded aggregation

Co-authored-by: Cursor <cursoragent@cursor.com>
@jenunes jenunes requested a review from svetasmirnova as a code owner May 28, 2026 18:04
@it-percona-cla

it-percona-cla commented May 28, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants