feat: add getMemoryStats endpoint to registry#486
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an admin-only getMemoryStats query endpoint to the registry canister to report runtime RTS memory metrics and per-data-structure {count, bytes} estimates (via Candid serialization sizing) for storage overhead analysis.
Changes:
- Extended the public Candid interface with
StructureStats,MemoryStats, andgetMemoryStats. - Implemented
getMemoryStatsin the main canister, including RTS heap/memory metrics and per-structure aggregation. - Added internal
getMemoryStatshelpers toDownloadLog,StorageManager, andUsersto report their respective structure stats.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| cli/declarations/main/main.did | Adds StructureStats/MemoryStats types and the getMemoryStats service method. |
| backend/main/main-canister.mo | Implements the admin-only query and aggregates stats across in-canister structures and submodules. |
| backend/main/DownloadLog.mo | Adds stats computation for download maps, snapshot buffers, and temp records. |
| backend/storage/storage-manager.mo | Adds stats computation for storage maps. |
| backend/main/Users.mo | Adds stats computation for the users map. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Change getMemoryStats from query to update to avoid hitting query instruction limits when serializing all entries via to_candid - Bump API_VERSION to 1.4 in backend and CLI (paired change) - Update cli/declarations/main/main.did.js and main.did.d.ts with StructureStats and MemoryStats types, and getMemoryStats method - Add unit tests for getMemoryStats in test/users.test.mo and test/download-log.test.mo Made-with: Cursor
Replace full-iteration to_candid calls with a systematic sample of up to 10,000 entries per data structure. The sampled byte total is extrapolated linearly (sampleBytes * total / sampled), so the result remains a useful approximation while keeping allocations bounded regardless of registry size. Made-with: Cursor
Sampling keeps to_candid calls bounded to ~10,000 per structure, so query instruction limits are no longer a concern. Made-with: Cursor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 11 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…eter environments)
72443ec to
c144cf3
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Suggestion: Extract memory stats logic into a shared module The sampling logic is duplicated across 4 files ( The idea: 1. New
2. Domain modules — each // Users.mo — before: ~40 lines of inline sampling
// Users.mo — after:
public func getMemoryStats() : { users : MemoryStatsHelper.StructureStats; names : MemoryStatsHelper.StructureStats } {
{
users = MemoryStatsHelper.sampleMapBytes(_users, func(k : Principal, v : Types.User) : Blob = to_candid((k, v)));
names = MemoryStatsHelper.sampleSetBytes(_names, func(k : Text) : Blob = to_candid(k));
}
};Similar for 3. Main actor — stays minimal: public query ({ caller }) func getMemoryStats() : async MemoryStatsHelper.MemoryStats {
assert (Utils.isAdmin(caller));
MemoryStatsHelper.collect(
{ packageVersions; packageConfigs; highestConfigs; packagePublications; ownersByPackage; maintainersByPackage; fileIdsByPackage; hashByFileId; packageFileStats; packageTestStats; packageBenchmarks; packageNotes; packageDocsCoverage },
downloadLog.getMemoryStats(),
storageManager.getMemoryStats(),
users.getMemoryStats(),
);
};This way:
|
Move the duplicated getMemoryStats sampling logic (sampleMapBytes, sampleBufferBytes, sampleIterBytes, sampleMapOfBuffersBytes) into a dedicated backend/main/MemoryStats.mo module. Domain modules (DownloadLog, Users, StorageManager) and the main actor now delegate to these shared helpers, reducing inline sampling code from ~180 lines to thin call-site wrappers. Made-with: Cursor
… into registry-memory-stats
- Collapse sampleMapBytes/sampleBufferBytes into the single
sampleIterBytes primitive; keep sampleMapOfBuffersBytes for the
two-level case
- Add statsForMap, statsForBuffer, statsForIter, statsForMapOfBuffers
convenience functions that return StructureStats directly
- Update all call sites to use the statsFor* helpers, removing the
repeated { count = ...; bytes = ... } boilerplate
- Rename MemoryStatsHelper → MemoryStats import in main-canister.mo
Made-with: Cursor
- Move MemoryStats type definition into MemoryStats.mo module
- Use record update syntax ({ dlStats and smStats and uStats with ... })
to eliminate manual field forwarding from sub-module stats
- Remove inaccurate comment about packageOwners stable storage clearing
Made-with: Cursor
Kamirus
left a comment
There was a problem hiding this comment.
Go for it, just one nit that might raise some warnings
This PR adds an admin-only query endpoint that reports entry counts and Candid-serialized byte counts for analyzing what data structures account for the most storage overhead in the registry canister.
getMemoryStatsis a query method gated behindUtils.isAdmin(caller). It returns:rts_heap_sizeandrts_memory_sizefrommo:prim{ count : Nat; bytes : Nat }for all data structures across the main actor,DownloadLog,StorageManager, andUsers