Skip to content

feat: Implement a Bigtable Emulator (WIP)#15247

Closed
marete wants to merge 248 commits intogoogleapis:mainfrom
Unoperate:for_google
Closed

feat: Implement a Bigtable Emulator (WIP)#15247
marete wants to merge 248 commits intogoogleapis:mainfrom
Unoperate:for_google

Conversation

@marete
Copy link
Copy Markdown

@marete marete commented Jul 1, 2025

This change is Reviewable

dopiera and others added 30 commits December 3, 2024 15:33
Logic from Table::MutateRow() will be moved here and this commit
proposes the general structure of this class, for early review.
And unconditionally return an error status for the aggregation
mutations, which we don't currently support.
…ion.

- Introduces a RowTransaction class to proxy mutations and record undo
  logs in case we needs to rewind.
- Re-implements the SetCell mutation in the RowTransaction class.
- Introduces all Undo types/actions for SetCell and records them in an
  undo stack.
- For SetCell only and to facilitate testing with only SetCell
  mutations, introduces the use of RowTransaction in doing the mutation.
- clang-format is run on all affected files (which has removed some
  headers, e.t.c.

A test for SetCell is coming up and is not currently included here.
Populate a schema protobuf with the minimal fields and call
Table::Create();
…utation.

There is a crash later on, in the code that sets up the undo log, that
will be fixed in an upcoming commit.
marete and others added 13 commits July 10, 2025 21:05
Without this, GetCurrentTime which exists as a macro in Windows is
expanded, leading to this compilation error:
https://github.com/googleapis/google-cloud-cpp/actions/runs/16149454809/job/45577395445#step:12:3259

This method of fixing this problem is documented and recommended here:
https://protobuf.dev/support/migration/#getcurrenttime
Without this, GetCurrentTime which exists as a macro in Windows is
expanded, leading to this compilation error:
https://github.com/googleapis/google-cloud-cpp/actions/runs/16149454809/job/45577395445#step:12:3259

This method of fixing this problem is documented and recommended here:
https://protobuf.dev/support/migration/#getcurrenttime
… a size_t.

The compiler spits out an error in pedantic mode because on 32 bit
platforms, there is a usually the possibility of an overflow. However,
the quantity is guaranteed to be 32
bit (https://github.com/googleapis/googleapis/blob/74657e8a6690b249c048f685124ee3b8473b70b4/google/bigtable/v2/data.proto#L497)
so use int32_t to store it and fix the MSVC compiler warning and failure.
…gth are adjacent.

This will be used in a subsequent PR to correctly determine if 2 Bigtable row
keys are adjacent, given that a Bigtable row key must be at most 4KiB
in length.
@scotthart
Copy link
Copy Markdown
Member

/gcbrun

marete and others added 12 commits July 17, 2025 23:13
The failures of the build were due to:

- Missing includes leading to errors in template type instantiation
- column_family_test.cc was included in the common library causing
- duplication definitions at link time.

Not sure why these problems did not manifest on Linux which builds and
test cleanly before this change, even with -pedantic and
-pedantic-errors turned on.
The failures of the build were due to:

- Missing includes leading to errors in template type instantiation
- column_family_test.cc was included in the common library causing duplication definitions at link time.

Not sure why these problems did not manifest on Linux which builds and
test cleanly before this change, even with -pedantic and
-pedantic-errors turned on.
@marete
Copy link
Copy Markdown
Author

marete commented Jul 22, 2025

Closing this PR - we will import the code for the emulator into a new repository.

1 similar comment
@marete
Copy link
Copy Markdown
Author

marete commented Jul 22, 2025

Closing this PR - we will import the code for the emulator into a new repository.

@marete marete closed this Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants