Skip to content

Support simulation mode for unit test targets#13290

Open
tclinkenbeard-oai wants to merge 5 commits into
apple:mainfrom
tclinkenbeard-oai:dev/tclinkenbeard/library-tests-simulation-mode
Open

Support simulation mode for unit test targets#13290
tclinkenbeard-oai wants to merge 5 commits into
apple:mainfrom
tclinkenbeard-oai:dev/tclinkenbeard/library-tests-simulation-mode

Conversation

@tclinkenbeard-oai
Copy link
Copy Markdown
Collaborator

This PR extends unit test targets (e.g. fdbserver_commitproxy_test) to support running unit tests in simulation. This is not possible for flow_test, which doesn't have access to the simulator. Validated by running unit test targets with the simulation flag.

@tclinkenbeard-oai tclinkenbeard-oai force-pushed the dev/tclinkenbeard/library-tests-simulation-mode branch from 37679e3 to 55636d1 Compare May 28, 2026 02:44
@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: 37679e3
  • Duration 0:04:08
  • Result: ❌ FAILED
  • Error: git checkout failed with exit status 128: fatal: unable to read tree (37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4) for primary source and source version 37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-ide on Linux RHEL 9

  • Commit ID: 37679e3
  • Duration 0:04:13
  • Result: ❌ FAILED
  • Error: git checkout failed with exit status 128: fatal: unable to read tree (37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4) for primary source and source version 37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: 37679e3
  • Duration 0:04:18
  • Result: ❌ FAILED
  • Error: git checkout failed with exit status 128: fatal: unable to read tree (37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4) for primary source and source version 37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: 37679e3
  • Duration 0:04:21
  • Result: ❌ FAILED
  • Error: git checkout failed with exit status 128: fatal: unable to read tree (37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4) for primary source and source version 37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: 37679e3
  • Duration 0:04:21
  • Result: ❌ FAILED
  • Error: git checkout failed with exit status 128: fatal: unable to read tree (37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4) for primary source and source version 37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: 37679e3
  • Duration 0:05:20
  • Result: ❌ FAILED
  • Error: git checkout failed with exit status 128: fatal: unable to read tree (37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4) for primary source and source version 37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: 37679e3
  • Duration 0:05:33
  • Result: ❌ FAILED
  • Error: git checkout failed with exit status 128: fatal: unable to read tree (37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4) for primary source and source version 37679e3b07c2d4f6dcb1fbc2d372462f69a08cc4
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-ide on Linux RHEL 9

  • Commit ID: 55636d1
  • Duration 0:23:00
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: 55636d1
  • Duration 0:46:09
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: 55636d1
  • Duration 0:50:01
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: 55636d1
  • Duration 0:53:54
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: 55636d1
  • Duration 0:54:57
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: 55636d1
  • Duration 1:03:19
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: 55636d1
  • Duration 1:12:16
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-ide on Linux RHEL 9

  • Commit ID: 81c7c18
  • Duration 0:20:46
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: 81c7c18
  • Duration 0:32:45
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: 81c7c18
  • Duration 0:45:00
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: 81c7c18
  • Duration 0:46:06
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: 81c7c18
  • Duration 0:50:12
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: 81c7c18
  • Duration 0:53:51
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: 81c7c18
  • Duration 1:02:15
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: 38ddd5b
  • Duration 0:22:49
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: 38ddd5b
  • Duration 0:22:58
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: 38ddd5b
  • Duration 0:23:01
  • Result: ❌ FAILED
  • Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: 38ddd5b
  • Duration 0:24:30
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-ide on Linux RHEL 9

  • Commit ID: 38ddd5b
  • Duration 0:25:59
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: 38ddd5b
  • Duration 0:26:07
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: 38ddd5b
  • Duration 0:41:43
  • Result: ❌ FAILED
  • Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-ide on Linux RHEL 9

  • Commit ID: 958a680
  • Duration 0:22:26
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: 958a680
  • Duration 0:34:06
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: 958a680
  • Duration 0:46:32
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: 958a680
  • Duration 0:47:13
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: 958a680
  • Duration 0:50:50
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: 958a680
  • Duration 0:53:58
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: 958a680
  • Duration 1:06:50
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

Copy link
Copy Markdown
Collaborator Author

@tclinkenbeard-oai tclinkenbeard-oai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generated by Codex.

What is it trying to do?

This CL adds a --simulation execution mode to standalone unit-test targets backed by fdbclient, fdbrpc, and fdbserver, while rejecting that option for unsupported targets such as flow_test. It initializes Sim2, simulated filesystems, per-process transports, an HTTP server process for simulated HTTP/mock-S3 tests, simulation-aware knobs, and filters noSim/ tests by default when simulation mode is selected. It also adds the missing fdbserver_core dependency required by the mocks3 unit-test target after the new server-knob initialization dependency.

Is it correct?

Yes, based on code inspection. The runner only selects simulation setup for targets that supply an initializer, and the initializer is awaited before executing tests. The new startup path establishes the simulated network before tracing or test execution, switches execution to the test process, and creates the simulated filesystem and transport state expected by affected unit tests. The HTTP process initialization is consistent with the existing simulated HTTP registration path used by HTTPServer.cpp and MockS3Server.cpp.

I also checked the apparent lifetime concern around discarding the value returned from FlowTransport::bind(): bind() retains the listener future internally, and its return value is used by other callers only to observe listener errors, so this is not a correctness issue for these simulated bindings.

This change does not add serialized messages, durable records, or independently versioned outputs, so I found no protocol-compatibility boundary requiring version gating. It also does not place new work on a production hot path; the added filtering and initialization occur only in standalone unit-test invocation.

At the current head, all visible PR checks are succeeding, including FoundationDB CI PR Builder, clang, clang arm, clang-ide, macOS, macOS_m1, PR Cluster Tests, and Test Boost CONFIG Mode on Windows. Earlier build failures were on the previous commit before the mocks3 dependency fix.

No build, unit-test, or simulation validation was run as part of this review.

Are there bugs?

I did not find any correctness bugs.

Are there omissions?

There is no automated regression coverage added for the new command-line mode, unsupported-target rejection, or the default noSim/ filtering behavior. The PR description states that the simulation invocation was exercised, and the current public CI results are green, so I do not think this omission blocks the change; a focused runner-level regression would still make this behavior easier to preserve.

Are there better ways of doing things?

A small test covering --simulation option handling and noSim/ selection behavior would be preferable to relying only on invocation-level validation. I do not see a necessary implementation rewrite beyond that.

Should this CL be LGTMd?

Yes, LGTM. I inspected the runner changes, simulator/process/bootstrap initialization, knob setup, HTTP/mock-S3 support path, and target linkage update. The highest remaining risk is untested regression in the new option/filtering behavior, which does not appear blocking given the scoped code path and successful current checks.

Copy link
Copy Markdown
Contributor

@saintstack saintstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition. Worth a mention in documentation/sphinx/source/testing.rst ? So it'll get noticed/used?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants