Question about engine-pair concept in Navy

[arungeorge83]

I see that a concept of Engine pairs (multiple pairs of BigHash & BlockCache) is added into the initialization sequence of Cachelib.
https://github.com/facebook/CacheLib/blob/main/cachelib/allocator/nvmcache/NavySetup.cpp#L241
And I don't see any documentation or much comments on the code about it.

Could you explain the main use cases of this concept?
Thanks

[haowu14]

Hi, the idea is similar to the "pools" in memory. If you have multiple use cases in the same cachelib instance, you can use pools to separate them in memory and engine pairs to separate them in NVM.

[arungeorge83]

Thanks.
A few questions more please.

1 - How does a 'key' or 'NVM IO' is directed to a particular navy engine pair? what is the mapping logic for the 'key -> engine pair' ?
2 - is there any mapping between memory pool and 'navy engine pair'
3 - Are you guys planning to publish any 'cachebench/test_configs/' with engine pair examples?

[jaesoo-fb]

Hi @arungeorge83

Let me answer the follow up questions.

The EnginePair is selected based on the policy provided in NavyConfig via NavyConfig::setEnginesSelector

2. The EngineSelector is a std::function taking HashedKey as input, meaning the EnginePair could be determined only based on the HashedKey.

using EnginesSelector = std::function<size_t(HashedKey)>;

3. No plan yet. Even the EnginePair is not supported in the cachebench's CacheConfig yet, either.

[arungeorge83]

Hey @jaesoo-fb @haowu14,
Thanks for the replies.

[pbhandar2]

Closing this discussion as it has been inactive. If this is still relevant, then please create a new discussion post and we will get back to you as soon as we can. Thank you!