Skip to content

Commit 47fa5c2

Browse files
beekldbeekld
authored
feat: add internal BigSegmentStoreWrapper with caching and status polling (#543)
## Summary Internal layer between the evaluator and a customer-provided `IBigSegmentStore`: hashes context keys, caches membership lookups, polls the store's metadata in the background to track availability/staleness, and broadcasts status changes. No caller yet — the evaluator and the public status provider come later in the stack. ## Design notes - Context keys are hashed to `base64(sha256(key))` using the standard base64 alphabet (not the existing URL-safe `Base64UrlEncode`), so the lookup key matches what the Relay Proxy writes. - Concurrent misses for the same key coalesce into a single store query rather than each hitting the store. - The background poll is built on `async::Delay` + `CancellationSource`. - A store error encountered during a membership lookup flips the store status to unavailable. The spec requires this; the Java and Go SDKs don't do it, so this is a deliberate divergence. - Changes `StoreMetadata::last_up_to_date` from a bare `milliseconds` to a `system_clock::time_point`, so staleness math can't mix epochs. This touches the Redis and DynamoDB stores (one line each) and their metadata tests. ## Test plan - [x] Added unit tests - [x] Full server-sdk suite green (499/499) - [x] DynamoDB source suite green (40/40) against DynamoDB Local - [x] Redis source suite green (37/37) against a live Redis <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes the public `IBigSegmentStore` contract (`noexcept`, `StoreMetadata` type) and adds threaded/async polling plus store-error handling that affects how staleness and availability are reported; evaluator is not wired yet, limiting immediate eval impact. > > **Overview** > Adds **`BigSegmentStoreWrapper`**, the internal layer in front of `IBigSegmentStore`: hashes context keys as **`base64(sha256(key))`** via new **`Base64Encode`** (RFC 4648, not URL-safe), uses an LRU/TTL **`MembershipCache`**, coalesces concurrent lookups for the same key, polls metadata on a timer for availability/staleness, and broadcasts status changes. Membership lookups return **`BigSegmentsStatus`** (healthy/stale/store error); a failed lookup marks the store unavailable (per spec). > > **`StoreMetadata::last_up_to_date`** is now a **`system_clock::time_point`** instead of raw milliseconds; Redis and DynamoDB stores and their tests are updated accordingly. **`IBigSegmentStore::GetMembership` / `GetMetadata`** are **`noexcept`**, with DynamoDB wrapping AWS calls in try/catch. > > Supporting pieces: **`big_segments_status.hpp`**, **`#include <variant>`** in `promise.hpp`, and unit tests for the wrapper, cache, and standard base64. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 0b492dd. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent e9c3c04 commit 47fa5c2

21 files changed

Lines changed: 1252 additions & 102 deletions

File tree

‎libs/internal/include/launchdarkly/async/promise.hpp‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
#include <memory>
88
#include <mutex>
99
#include <optional>
10+
#include <variant>
1011
#include <vector>
1112

1213
namespace launchdarkly::async {

‎libs/internal/include/launchdarkly/encoding/base_64.hpp‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,13 @@ namespace launchdarkly::encoding {
1313
*/
1414
std::string Base64UrlEncode(std::string const& input);
1515

16+
/**
17+
* Return a standard base64 encoded version of the input string, using the
18+
* RFC 4648 section 4 alphabet (with '+' and '/'). Unlike @ref Base64UrlEncode,
19+
* this is NOT URL-safe.
20+
* @param input The string to Base64 encode.
21+
* @return The encoded value.
22+
*/
23+
std::string Base64Encode(std::string const& input);
24+
1625
} // namespace launchdarkly::encoding

‎libs/internal/src/encoding/base_64.cpp‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
#include <launchdarkly/encoding/base_64.hpp>
22

3+
#include <openssl/evp.h>
4+
35
#include <algorithm>
46
#include <array>
57
#include <bitset>
68
#include <climits>
79
#include <cstddef>
10+
#include <vector>
811

912
static unsigned char const kEncodeSize = 4;
1013
static unsigned char const kInputBytesPerEncodeSize = 3;
@@ -75,4 +78,17 @@ std::string Base64UrlEncode(std::string const& input) {
7578
return out;
7679
}
7780

81+
std::string Base64Encode(std::string const& input) {
82+
if (input.empty()) {
83+
return {};
84+
}
85+
// EVP_EncodeBlock writes 4 output characters per 3 input bytes (rounded up)
86+
// plus a NUL terminator.
87+
std::vector<unsigned char> out(4 * ((input.size() + 2) / 3) + 1);
88+
int const written = EVP_EncodeBlock(
89+
out.data(), reinterpret_cast<unsigned char const*>(input.data()),
90+
static_cast<int>(input.size()));
91+
return std::string(reinterpret_cast<char const*>(out.data()), written);
92+
}
93+
7894
} // namespace launchdarkly::encoding

‎libs/internal/tests/base_64_test.cpp‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,11 @@
22

33
#include "launchdarkly/encoding/base_64.hpp"
44

5+
using launchdarkly::encoding::Base64Encode;
56
using launchdarkly::encoding::Base64UrlEncode;
67

78
TEST(Base64Encoding, CanEncodeString) {
8-
// Test vectors from RFC4668
9+
// Test vectors from RFC4648
910
// https://datatracker.ietf.org/doc/html/rfc4648#section-10
1011
EXPECT_EQ("", Base64UrlEncode(""));
1112
EXPECT_EQ("Zg==", Base64UrlEncode("f"));
@@ -15,3 +16,24 @@ TEST(Base64Encoding, CanEncodeString) {
1516
EXPECT_EQ("Zm9vYmE=", Base64UrlEncode("fooba"));
1617
EXPECT_EQ("Zm9vYmFy", Base64UrlEncode("foobar"));
1718
}
19+
20+
TEST(Base64Encoding, StandardCanEncodeString) {
21+
// Test vectors from RFC4648
22+
// https://datatracker.ietf.org/doc/html/rfc4648#section-10
23+
EXPECT_EQ("", Base64Encode(""));
24+
EXPECT_EQ("Zg==", Base64Encode("f"));
25+
EXPECT_EQ("Zm8=", Base64Encode("fo"));
26+
EXPECT_EQ("Zm9v", Base64Encode("foo"));
27+
EXPECT_EQ("Zm9vYg==", Base64Encode("foob"));
28+
EXPECT_EQ("Zm9vYmE=", Base64Encode("fooba"));
29+
EXPECT_EQ("Zm9vYmFy", Base64Encode("foobar"));
30+
}
31+
32+
TEST(Base64Encoding, StandardUsesNonUrlSafeAlphabet) {
33+
// "???" encodes to a value ending in '/' under the standard alphabet and
34+
// '_' under the URL-safe one; ">>>" exercises '+' versus '-'.
35+
EXPECT_EQ("Pz8/", Base64Encode("???"));
36+
EXPECT_EQ("Pz8_", Base64UrlEncode("???"));
37+
EXPECT_EQ("Pj4+", Base64Encode(">>>"));
38+
EXPECT_EQ("Pj4-", Base64UrlEncode(">>>"));
39+
}

‎libs/server-sdk-dynamodb-source/include/launchdarkly/server_side/integrations/dynamodb/dynamodb_big_segment_store.hpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,8 @@ class DynamoDBBigSegmentStore final : public IBigSegmentStore {
6161
DynamoDBClientOptions options = {});
6262

6363
[[nodiscard]] GetMembershipResult GetMembership(
64-
std::string const& context_hash) const override;
65-
[[nodiscard]] GetMetadataResult GetMetadata() const override;
64+
std::string const& context_hash) const noexcept override;
65+
[[nodiscard]] GetMetadataResult GetMetadata() const noexcept override;
6666

6767
~DynamoDBBigSegmentStore() override;
6868

‎libs/server-sdk-dynamodb-source/src/dynamodb_big_segment_store.cpp‎

Lines changed: 97 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -64,100 +64,115 @@ DynamoDBBigSegmentStore::DynamoDBBigSegmentStore(
6464
DynamoDBBigSegmentStore::~DynamoDBBigSegmentStore() = default;
6565

6666
IBigSegmentStore::GetMembershipResult DynamoDBBigSegmentStore::GetMembership(
67-
std::string const& context_hash) const {
68-
Aws::DynamoDB::Model::GetItemRequest request;
69-
request.SetTableName(table_name_);
70-
request.SetConsistentRead(true);
71-
request.AddKey(kPartitionKey,
72-
Aws::DynamoDB::Model::AttributeValue{user_namespace_});
73-
request.AddKey(kSortKey,
74-
Aws::DynamoDB::Model::AttributeValue{context_hash});
75-
76-
auto outcome = client_->GetItem(request);
77-
if (!outcome.IsSuccess()) {
78-
return tl::make_unexpected(outcome.GetError().GetMessage());
79-
}
80-
81-
auto const& item = outcome.GetResult().GetItem();
82-
83-
std::vector<std::string> included;
84-
std::vector<std::string> excluded;
85-
86-
// GetSS() silently returns an empty vector if the attribute is not
87-
// actually a String Set, so check the type explicitly before reading.
88-
if (auto const it = item.find(kBigSegmentsIncludedAttribute);
89-
it != item.end()) {
90-
if (it->second.GetType() !=
91-
Aws::DynamoDB::Model::ValueType::STRING_SET) {
92-
return tl::make_unexpected(
93-
std::string("DynamoDB Big Segments '") +
94-
kBigSegmentsIncludedAttribute +
95-
"' is not of type STRING_SET");
96-
}
97-
for (auto const& ref : it->second.GetSS()) {
98-
included.emplace_back(ref);
67+
std::string const& context_hash) const noexcept {
68+
try {
69+
Aws::DynamoDB::Model::GetItemRequest request;
70+
request.SetTableName(table_name_);
71+
request.SetConsistentRead(true);
72+
request.AddKey(kPartitionKey,
73+
Aws::DynamoDB::Model::AttributeValue{user_namespace_});
74+
request.AddKey(kSortKey,
75+
Aws::DynamoDB::Model::AttributeValue{context_hash});
76+
77+
auto outcome = client_->GetItem(request);
78+
if (!outcome.IsSuccess()) {
79+
return tl::make_unexpected(outcome.GetError().GetMessage());
9980
}
100-
}
101-
if (auto const it = item.find(kBigSegmentsExcludedAttribute);
102-
it != item.end()) {
103-
if (it->second.GetType() !=
104-
Aws::DynamoDB::Model::ValueType::STRING_SET) {
105-
return tl::make_unexpected(
106-
std::string("DynamoDB Big Segments '") +
107-
kBigSegmentsExcludedAttribute +
108-
"' is not of type STRING_SET");
81+
82+
auto const& item = outcome.GetResult().GetItem();
83+
84+
std::vector<std::string> included;
85+
std::vector<std::string> excluded;
86+
87+
// GetSS() silently returns an empty vector if the attribute is not
88+
// actually a String Set, so check the type explicitly before reading.
89+
if (auto const it = item.find(kBigSegmentsIncludedAttribute);
90+
it != item.end()) {
91+
if (it->second.GetType() !=
92+
Aws::DynamoDB::Model::ValueType::STRING_SET) {
93+
return tl::make_unexpected(
94+
std::string("DynamoDB Big Segments '") +
95+
kBigSegmentsIncludedAttribute +
96+
"' is not of type STRING_SET");
97+
}
98+
for (auto const& ref : it->second.GetSS()) {
99+
included.emplace_back(ref);
100+
}
109101
}
110-
for (auto const& ref : it->second.GetSS()) {
111-
excluded.emplace_back(ref);
102+
if (auto const it = item.find(kBigSegmentsExcludedAttribute);
103+
it != item.end()) {
104+
if (it->second.GetType() !=
105+
Aws::DynamoDB::Model::ValueType::STRING_SET) {
106+
return tl::make_unexpected(
107+
std::string("DynamoDB Big Segments '") +
108+
kBigSegmentsExcludedAttribute +
109+
"' is not of type STRING_SET");
110+
}
111+
for (auto const& ref : it->second.GetSS()) {
112+
excluded.emplace_back(ref);
113+
}
112114
}
113-
}
114115

115-
return Membership::FromSegmentRefs(included, excluded);
116+
return Membership::FromSegmentRefs(included, excluded);
117+
} catch (std::exception const& e) {
118+
return tl::make_unexpected(e.what());
119+
}
116120
}
117121

118122
IBigSegmentStore::GetMetadataResult DynamoDBBigSegmentStore::GetMetadata()
119-
const {
120-
Aws::DynamoDB::Model::GetItemRequest request;
121-
request.SetTableName(table_name_);
122-
request.SetConsistentRead(true);
123-
request.AddKey(kPartitionKey,
124-
Aws::DynamoDB::Model::AttributeValue{metadata_namespace_});
125-
request.AddKey(kSortKey,
126-
Aws::DynamoDB::Model::AttributeValue{metadata_namespace_});
127-
128-
auto outcome = client_->GetItem(request);
129-
if (!outcome.IsSuccess()) {
130-
return tl::make_unexpected(outcome.GetError().GetMessage());
131-
}
123+
const noexcept {
124+
try {
125+
Aws::DynamoDB::Model::GetItemRequest request;
126+
request.SetTableName(table_name_);
127+
request.SetConsistentRead(true);
128+
request.AddKey(
129+
kPartitionKey,
130+
Aws::DynamoDB::Model::AttributeValue{metadata_namespace_});
131+
request.AddKey(
132+
kSortKey,
133+
Aws::DynamoDB::Model::AttributeValue{metadata_namespace_});
134+
135+
auto outcome = client_->GetItem(request);
136+
if (!outcome.IsSuccess()) {
137+
return tl::make_unexpected(outcome.GetError().GetMessage());
138+
}
132139

133-
auto const& item = outcome.GetResult().GetItem();
134-
if (item.empty()) {
135-
return std::nullopt;
136-
}
140+
auto const& item = outcome.GetResult().GetItem();
141+
if (item.empty()) {
142+
return std::nullopt;
143+
}
137144

138-
auto const it = item.find(kBigSegmentsSyncTimeAttribute);
139-
if (it == item.end()) {
140-
// "absent" sync time is treated as never synchronized rather than
141-
// an error; the wrapper marks the store stale based on the
142-
// resulting nullopt.
143-
return std::nullopt;
144-
}
145+
auto const it = item.find(kBigSegmentsSyncTimeAttribute);
146+
if (it == item.end()) {
147+
// "absent" sync time is treated as never synchronized rather than
148+
// an error; the wrapper marks the store stale based on the
149+
// resulting nullopt.
150+
return std::nullopt;
151+
}
145152

146-
auto const& raw = it->second.GetN();
147-
if (raw.empty()) {
148-
return tl::make_unexpected(
149-
"DynamoDB Big Segments 'synchronizedOn' is empty or not type N");
150-
}
153+
auto const& raw = it->second.GetN();
154+
if (raw.empty()) {
155+
return tl::make_unexpected(
156+
"DynamoDB Big Segments 'synchronizedOn' is empty or not type "
157+
"N");
158+
}
151159

152-
errno = 0;
153-
char* end = nullptr;
154-
long long const parsed = std::strtoll(raw.c_str(), &end, 10);
155-
if (errno != 0 || end == raw.c_str() || *end != '\0') {
156-
return tl::make_unexpected(
157-
"DynamoDB Big Segments 'synchronizedOn' is not a valid integer");
158-
}
160+
errno = 0;
161+
char* end = nullptr;
162+
long long const parsed = std::strtoll(raw.c_str(), &end, 10);
163+
if (errno != 0 || end == raw.c_str() || *end != '\0') {
164+
return tl::make_unexpected(
165+
"DynamoDB Big Segments 'synchronizedOn' is not a valid "
166+
"integer");
167+
}
159168

160-
return StoreMetadata{std::chrono::milliseconds{parsed}};
169+
// The stored value is a Unix-epoch millisecond count: system_clock's
170+
// epoch.
171+
return StoreMetadata{std::chrono::system_clock::time_point{
172+
std::chrono::milliseconds{parsed}}};
173+
} catch (std::exception const& e) {
174+
return tl::make_unexpected(e.what());
175+
}
161176
}
162177

163178
} // namespace launchdarkly::server_side::integrations

‎libs/server-sdk-dynamodb-source/tests/dynamodb_big_segment_store_test.cpp‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,8 @@ TEST_F(DynamoDBBigSegmentTests, GetMetadataWithEmptyPrefix) {
163163
ASSERT_TRUE(result);
164164
ASSERT_TRUE(result->has_value());
165165
ASSERT_EQ(result->value().last_up_to_date,
166-
std::chrono::milliseconds{1700000000000LL});
166+
std::chrono::system_clock::time_point{
167+
std::chrono::milliseconds{1700000000000LL}});
167168
}
168169

169170
TEST_F(DynamoDBBigSegmentTests, GetMembershipRejectsMalformedIncluded) {
@@ -182,7 +183,8 @@ TEST_F(DynamoDBBigSegmentTests, GetMetadataReturnsSyncTime) {
182183
ASSERT_TRUE(result);
183184
ASSERT_TRUE(result->has_value());
184185
ASSERT_EQ(result->value().last_up_to_date,
185-
std::chrono::milliseconds{1700000000000LL});
186+
std::chrono::system_clock::time_point{
187+
std::chrono::milliseconds{1700000000000LL}});
186188
}
187189

188190
TEST_F(DynamoDBBigSegmentTests, GetMetadataAbsentSyncTimeReturnsNoMetadata) {

‎libs/server-sdk-redis-source/include/launchdarkly/server_side/integrations/redis/redis_big_segment_store.hpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,8 @@ class RedisBigSegmentStore final : public IBigSegmentStore {
5555
Create(std::string uri, std::string prefix);
5656

5757
[[nodiscard]] GetMembershipResult GetMembership(
58-
std::string const& context_hash) const override;
59-
[[nodiscard]] GetMetadataResult GetMetadata() const override;
58+
std::string const& context_hash) const noexcept override;
59+
[[nodiscard]] GetMetadataResult GetMetadata() const noexcept override;
6060

6161
~RedisBigSegmentStore() override;
6262

‎libs/server-sdk-redis-source/src/redis_big_segment_store.cpp‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ RedisBigSegmentStore::RedisBigSegmentStore(
5757
RedisBigSegmentStore::~RedisBigSegmentStore() = default;
5858

5959
IBigSegmentStore::GetMembershipResult RedisBigSegmentStore::GetMembership(
60-
std::string const& context_hash) const {
60+
std::string const& context_hash) const noexcept {
6161
std::string const include_key = include_key_prefix_ + context_hash;
6262
std::string const exclude_key = exclude_key_prefix_ + context_hash;
6363

@@ -76,7 +76,8 @@ IBigSegmentStore::GetMembershipResult RedisBigSegmentStore::GetMembership(
7676
return Membership::FromSegmentRefs(included, excluded);
7777
}
7878

79-
IBigSegmentStore::GetMetadataResult RedisBigSegmentStore::GetMetadata() const {
79+
IBigSegmentStore::GetMetadataResult RedisBigSegmentStore::GetMetadata()
80+
const noexcept {
8081
sw::redis::OptionalString raw;
8182
try {
8283
raw = redis_->get(sync_time_key_);
@@ -98,7 +99,9 @@ IBigSegmentStore::GetMetadataResult RedisBigSegmentStore::GetMetadata() const {
9899
"Redis Big Segments synchronized_on is not a valid integer");
99100
}
100101

101-
return StoreMetadata{std::chrono::milliseconds{parsed}};
102+
// The stored value is a Unix-epoch millisecond count: system_clock's epoch.
103+
return StoreMetadata{std::chrono::system_clock::time_point{
104+
std::chrono::milliseconds{parsed}}};
102105
}
103106

104107
} // namespace launchdarkly::server_side::integrations

‎libs/server-sdk-redis-source/tests/redis_big_segment_store_test.cpp‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,8 @@ TEST_F(RedisBigSegmentTests, GetMetadataWithEmptyPrefix) {
143143
ASSERT_TRUE(result);
144144
ASSERT_TRUE(result->has_value());
145145
ASSERT_EQ(result->value().last_up_to_date,
146-
std::chrono::milliseconds{1700000000000LL});
146+
std::chrono::system_clock::time_point{
147+
std::chrono::milliseconds{1700000000000LL}});
147148
}
148149

149150
TEST_F(RedisBigSegmentTests, GetMetadataReturnsSyncTime) {
@@ -153,7 +154,8 @@ TEST_F(RedisBigSegmentTests, GetMetadataReturnsSyncTime) {
153154
ASSERT_TRUE(result);
154155
ASSERT_TRUE(result->has_value());
155156
ASSERT_EQ(result->value().last_up_to_date,
156-
std::chrono::milliseconds{1700000000000LL});
157+
std::chrono::system_clock::time_point{
158+
std::chrono::milliseconds{1700000000000LL}});
157159
}
158160

159161
TEST_F(RedisBigSegmentTests, GetMetadataRejectsMalformedSyncTime) {

0 commit comments

Comments
 (0)