Rubric Gap Analysis — partition-avoid-hotspots enhancement: Add sequential/auto-incrementing ID anti-pattern

[jaydestro]

#51 — Rubric Gap Analysis — partition-avoid-hotspots enhancement: Add sequential/auto-incrementing ID anti-pattern

Field	Value
Type	Rule Enhancement
Target Rule	partition-avoid-hotspots
Severity	HIGH
Source	SCOPE Rubric Criteria — Partition Key Design, Criterion 2, check 4
Labels	enhancement, SCOPE, agent-kit, rule:partition

Summary

The partition-avoid-hotspots rule covers time-based keys and singleton keys as hotspot causes, and includes a "Popularity Skew Warning" about uneven traffic on high-cardinality keys. However, it does not warn against using auto-incrementing integers, timestamp-sortable IDs (Snowflake IDs, ULIDs), or other monotonically increasing values as partition keys. Sequential IDs concentrate all new writes on the "latest" physical partition because Cosmos DB maps partition key ranges to physical partitions — monotonically increasing values always land in the same range boundary.

Rubric Gap Analysis

This gap was identified During rubric criteria review. The sequential ID anti-pattern is distinct from both the time-based key and popularity skew patterns already documented:

• Time-based keys: Writes funnel into "today" — the rule covers this
• Popularity skew: High-cardinality keys with uneven traffic — the rule covers this
• Sequential IDs: Writes funnel into the "latest" range boundary — not covered

This is particularly common when agents migrate from relational databases where auto-increment primary keys are standard, or when using distributed ID generators (Snowflake, ULID, KSUID) that produce sortable but monotonically increasing values.

Evidence

Existing Rule Coverage

The rule currently documents:

• Time-based partition keys (Date = DateTime.UtcNow.ToString("yyyy-MM-dd"))
• Singleton partition keys (PartitionKey = "config")
• Popularity Skew Warning (viral user scenario)

None address monotonically increasing values.

Missing Anti-Pattern

// Anti-pattern: auto-incrementing integer as partition key
public class Event
{
    public string Id { get; set; }
    
    // Monotonically increasing — all new writes hit the SAME physical partition
    public long SequenceNumber { get; set; }  // ❌ 1, 2, 3, 4, 5, ...
}

// Cosmos DB maps partition key ranges to physical partitions:
// Physical partition A: keys 0–50,000
// Physical partition B: keys 50,001–100,000
// All new writes (50,001, 50,002, ...) hit partition B exclusively
// Partition A sits idle with historical data

// Anti-pattern: timestamp-sortable distributed IDs
public class Document
{
    public string Id { get; set; }
    
    // ULID/Snowflake IDs are lexicographically sortable by time
    public string DocumentId { get; set; }  // ❌ "01HYA3B..." — always increasing
}
// Same problem: newest IDs always map to the highest range boundary

Correct Guidance

// Use GUIDs or hash-based IDs — random distribution across keyspace
public class Event
{
    public string Id { get; set; }
    
    // GUID: randomly distributed across the entire keyspace
    public string EventId { get; set; }  // ✅ "a1b2c3d4-..." — random distribution
}

// Or use a domain property that distributes naturally
public class Event
{
    public string Id { get; set; }
    public string DeviceId { get; set; }  // ✅ Natural distribution across devices
    public long SequenceNumber { get; set; }  // Keep as non-PK property
}

Recommended Enhancement

Add to partition-avoid-hotspots after the singleton partition key anti-pattern:

// Anti-pattern: auto-incrementing or monotonically increasing partition key
public class Event
{
    public string Id { get; set; }
    public long SequenceNumber { get; set; }  // ❌ Monotonic — hot "latest" partition
}
// Snowflake IDs, ULIDs, KSUIDs, and auto-increment integers all concentrate
// new writes on the physical partition holding the highest key range.
// Use GUIDs, natural entity IDs, or hash-derived keys instead.

References

• Choosing a partition key — common pitfalls
• Physical partition design
• SCOPE Rubric: partition-key-design.md, Criterion 2, check 4

[TheovanKraay]

If an agent understands what partition keys are for, using a monotonically increasing value as the partition key is just another form of “all new writes funnel into one hot area,” which the current rule already teaches through time based keys and hotspot examples. I don't see a strong case for adding more copy to the context window of an agent unless we have actual evidence that agents are specifically generating sequential IDs as partition keys.