Skip to content

Extend test harness to support envconfig #2163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
THardy98 wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Extend test harness to support envconfig #2163

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9fd00d4
extend test harness to run on arbitrary external servers
THardy98 Jul 6, 2026
b03eba6
remove unnecessary helpers, clarify docstrings
THardy98 Jul 6, 2026
1480be8
remove TEMPORAL_TEST_RUN_ID
THardy98 Jul 6, 2026
107b6c4
simplify api
THardy98 Jul 6, 2026
9d1d2ef
clarify doc string
THardy98 Jul 6, 2026
5fe9024
remove helper
THardy98 Jul 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"build:watch": "pnpm run build:protos && tsc --build --watch packages/*/tsconfig.json",
"build:protos": "tsx ./packages/proto/scripts/compile-proto.ts",
"test": "pnpm --recursive --parallel --workspace-concurrency=1 --stream run test",
"test:external": "pnpm --dir packages/test run test:external",
"test:watch": "pnpm --recursive --stream run test:watch",
"ci-stress": "node ./packages/test/lib/load/run-all-stress-ci-scenarios.js",
"ci-nightly": "node ./packages/test/lib/load/run-all-nightly-scenarios.js",
Expand Down
1 change: 1 addition & 0 deletions packages/test-helpers/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"license": "MIT",
"dependencies": {
"@temporalio/common": "workspace:*",
"@temporalio/envconfig": "workspace:*",
"@temporalio/proto": "workspace:*",
"@temporalio/testing": "workspace:*",
"@temporalio/worker": "workspace:*",
Expand Down
20 changes: 17 additions & 3 deletions packages/test-helpers/src/environment.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
import type { LocalTestWorkflowEnvironmentOptions } from '@temporalio/testing';
import { workflowInterceptorModules as defaultWorkflowInterceptorModules } from '@temporalio/testing';
import { loadClientConnectConfig } from '@temporalio/envconfig';
import type { BundlerPlugin, WorkflowBundleWithSourceMap, BundleOptions } from '@temporalio/worker';
import { bundleWorkflowCode, DefaultLogger } from '@temporalio/worker';
import { defineSearchAttributeKey, SearchAttributeType } from '@temporalio/common/lib/search-attributes';
import { TestWorkflowEnvironment } from './wrappers';
import { baseBundlerIgnoreModules } from './bundler';
import { isSet } from './flags';

export const defaultDynamicConfigOptions = [
'system.enableActivityEagerExecution=true',
Expand Down Expand Up @@ -76,14 +78,26 @@ export async function createLocalTestEnvironment(
}

/**
* Create a test workflow environment, using an existing server if TEMPORAL_SERVICE_ADDRESS is set,
* otherwise creating a local one.
* Create a test workflow environment.
*
* Uses envconfig for the test server connection when TEMPORAL_TEST_ENV_CONFIG_SERVER is truthy, uses
* TEMPORAL_SERVICE_ADDRESS as a legacy existing-server shortcut when set, otherwise creates a local environment.
*/
export async function createTestWorkflowEnvironment(
opts?: LocalTestWorkflowEnvironmentOptions
): Promise<TestWorkflowEnvironment> {
let env: TestWorkflowEnvironment;
if (process.env.TEMPORAL_SERVICE_ADDRESS) {
if (isSet(process.env.TEMPORAL_TEST_ENV_CONFIG_SERVER, false)) {
const { namespace, connectionOptions } = loadClientConnectConfig();
const { address, apiKey, metadata, tls } = connectionOptions;
env = await TestWorkflowEnvironment.createFromExistingServer({
address,
namespace,
connectionOptions: { apiKey, metadata, tls },
client: opts?.client,
plugins: opts?.plugins,
});
} else if (isSet(process.env.TEMPORAL_SERVICE_ADDRESS, false)) {
env = await TestWorkflowEnvironment.createFromExistingServer({
address: process.env.TEMPORAL_SERVICE_ADDRESS,
plugins: opts?.plugins,
Expand Down
1 change: 1 addition & 0 deletions packages/test-helpers/src/helpers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ export function helpers<TEnv extends AnyTestWorkflowEnvironment = TestWorkflowEn
async createWorker(workerOpts?: Partial<WorkerOptions>): Promise<Worker> {
return await Worker.create({
connection: env.nativeConnection,
namespace: env.namespace,
workflowBundle,
taskQueue,
showStackTraceSources: true,
Expand Down
1 change: 1 addition & 0 deletions packages/test-helpers/tsconfig.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
"references": [
{ "path": "../client" },
{ "path": "../common" },
{ "path": "../envconfig" },
{ "path": "../proto" },
{ "path": "../testing" },
{ "path": "../worker" },
Expand Down
131 changes: 131 additions & 0 deletions packages/test/EXTERNAL_SERVER_TESTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# `@temporalio/test`

## External Server Test Coverage

`pnpm run test:external` runs the subset of SDK integration tests that currently use the shared configurable test
harness and are expected to work against an existing Temporal server.

When `TEMPORAL_TEST_ENV_CONFIG_SERVER` is truthy (`1`, `t`, or `true`), the shared harness uses envconfig to
configure the test server connection. Envconfig resolves settings from environment variables, `TEMPORAL_CONFIG_FILE`,
`TEMPORAL_PROFILE`, and default TOML discovery.

- `TEMPORAL_TEST_ENV_CONFIG_SERVER`
- `TEMPORAL_ADDRESS`
- `TEMPORAL_NAMESPACE`
- `TEMPORAL_API_KEY`
- `TEMPORAL_CONFIG_FILE`
- `TEMPORAL_PROFILE`
- `TEMPORAL_TLS_CLIENT_CERT_PATH`
- `TEMPORAL_TLS_CLIENT_KEY_PATH`
- `TEMPORAL_TLS_CLIENT_CERT_DATA`
- `TEMPORAL_TLS_CLIENT_KEY_DATA`
- `TEMPORAL_TLS_SERVER_CA_CERT_PATH`
- `TEMPORAL_TLS_SERVER_CA_CERT_DATA`
- `TEMPORAL_TLS_SERVER_NAME`
- `TEMPORAL_GRPC_META_*`

`TEMPORAL_SERVICE_ADDRESS` remains supported as a legacy shortcut for connecting to an existing server without
envconfig. Use `TEMPORAL_TEST_ENV_CONFIG_SERVER` for namespace, auth, TLS, metadata, and TOML configuration.

Example:

```sh
pnpm run build
TEMPORAL_TEST_ENV_CONFIG_SERVER=true \
TEMPORAL_ADDRESS=namespace.account.tmprl.cloud:7233 \
TEMPORAL_NAMESPACE=namespace.account \
TEMPORAL_API_KEY=... \
pnpm run test:external
```

TOML example:

```sh
pnpm run build
TEMPORAL_TEST_ENV_CONFIG_SERVER=true \
TEMPORAL_CONFIG_FILE=/path/to/temporal.toml \
pnpm run test:external
```

`workflowEnvironmentOpts.server.extraArgs` only apply when the harness starts a local server. In external mode, the
target server must already have equivalent dynamic config and feature flags enabled.

## Current External Runner

| Test file | Notes |
| -------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| `test-default-activity.ts` | Shared harness. |
| `test-integration-activities.ts` | Shared harness. |
| `test-integration-cancellation-scopes.ts` | Shared harness. |
| `test-integration-replay-and-flags.ts` | Shared harness; time-skipping-only block is local-only. |
| `test-integration-reserved-prefixes.ts` | Shared harness. |
| `test-integration-update.ts` | Shared harness; assumes external server long-poll behavior is compatible with the assertions. |
| `test-integration-update-interceptors.ts` | Shared harness. |
| `test-integration-workflows-with-recorded-logs.ts` | Shared harness. |
| `test-isolation.ts` | Shared harness. |
| `test-nexus-operation-timeouts.ts` | Requires Nexus endpoint creation/operator permission. |
| `test-nexus-signal-linking.ts` | Requires Nexus and CHASM signal backlink server support. |
| `test-nexus-standalone.ts` | Requires standalone Nexus operation server support. |
| `test-nexus-temporal-operation.ts` | Requires standalone Nexus operation and CHASM callback server support. |
| `test-random-stream-reset.ts` | Shared harness. |
| `test-worker-tuner.ts` | Shared harness. |
| `test-workflow-async-local-storage.ts` | Shared harness. |
| `test-workflow-cancellation.ts` | Shared harness. |
| `test-workflow-fail-on-errors-policy.ts` | Shared harness. |
| `test-workflow-nexus-cancellation.ts` | Requires Nexus endpoint creation/operator permission. |

## Follow-Up Coverage Inventory

| Test file | Status / notes |
| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `test-async-completion.ts` | Uses default `Connection.connect()` in integration setup; needs shared harness connection. |
| `test-bundler.ts` | Uses default client/worker connection in integration block; needs shared harness connection. |
| `test-client-connection.ts` | Tests client behavior against local fake gRPC servers; not a cloud server test. |
| `test-client-errors.ts` | Direct `TestWorkflowEnvironment.createLocal()`. |
| `test-custom-payload-codec.ts` | Uses default client/worker connection in integration block. |
| `test-custom-payload-converter.ts` | Uses default client/worker connection in integration block. |
| `test-default-workflow.ts` | Direct `TestWorkflowEnvironment.createLocal()`. |
| `test-ephemeral-server.ts` | Intentionally tests local ephemeral/time-skipping servers. |
| `test-failure-converter.ts` | Direct `TestWorkflowEnvironment.createLocal()`. |
| `test-integration-split-one.ts` | Shared harness, but multi-codec environments share task queues; also has custom Search Attribute and hard-coded `default` namespace assertions. |
| `test-integration-split-two.ts` | Shared harness, but multi-codec environments share task queues and require custom Search Attributes. |
| `test-integration-split-three.ts` | Shared harness, but multi-codec environments share task queues; otherwise likely addable after validation. |
| `test-integration-workflow-info.ts` | Mostly shared harness; one test uses `createLocalTestEnvironment()` for custom Search Attribute setup. |
| `test-integration-workflow-start.ts` | Shared harness; has extra native worker connections and eager activity/start-delay behavior that need external validation. |
| `test-interceptors.ts` | Uses default client/worker connection in integration block. |
| `test-local-activities.ts` | Direct local environment setup; local activities do not require a local server but this file needs shared harness conversion. |
| `test-metrics-custom.ts` | Shared harness with local Prometheus scrape endpoint; likely addable after external validation. |
| `test-native-connection.ts` | Primarily local fake gRPC/default-connection behavior; not a cloud release test. |
| `test-native-connection-headers.ts` | Local fake gRPC server tests; not a cloud release test. |
| `test-nexus-codec-converter-errors.ts` | Shared harness; requires Nexus endpoint creation/operator permission and external validation. |
| `test-nexus-handler.ts` | Direct local environment with Nexus dynamic config; needs shared harness conversion or local-only classification. |
| `test-nexus-workflow-caller.ts` | Shared harness; has a hard-coded `default` namespace request and requires Nexus endpoint creation/operator permission. |
| `test-nyc-coverage.ts` | Uses default client/worker connection in integration block. |
| `test-patch-and-condition.ts` | Uses default client/worker connection in integration block. |
| `test-payload-converter.ts` | Uses default client/worker connection in integration block. |
| `test-plugins.ts` | Direct local environments and plugin-specific local connection assertions. |
| `test-runtime.ts` | Mixes shared harness and default native connections; runtime-focused tests need local-only review. |
| `test-runtime-otel.ts` | Direct local environment with local telemetry endpoint. |
| `test-runtime-prometheus.ts` | Direct local environment with local Prometheus endpoint. |
| `test-schedules.ts` | Uses default connection and operator Search Attribute registration; needs envconfig connection and pre-provisioned/managed Search Attributes. |
| `test-serialization-context.ts` | Shared harness but has hard-coded `workflow.default.*` expectations. |
| `test-serialization-context-codec.ts` | Shared harness but has hard-coded `workflow.default.*` and `activity.default.*` expectations. |
| `test-signal-query-patch.ts` | Direct `TestWorkflowEnvironment.createLocal()`. |
| `test-signals-are-always-delivered.ts` | Uses default client/worker connection in integration block. |
| `test-sinks.ts` | Uses default connection, operator Search Attribute registration, and hard-coded `default` namespace expectations. |
| `test-standalone-activities.ts` | Shared harness but has fixed task queue and local-only server dynamic config assumptions. |
| `test-temporal-cloud.ts` | Separate cloud smoke tests with bespoke cloud env vars; not part of envconfig external runner. |
| `test-testenvironment.ts` | Intentionally tests time-skipping test environment. |
| `test-typed-search-attributes.ts` | Uses local namespace/server Search Attribute setup through operator service. |
| `test-worker-connection-replacement.ts` | Intentionally creates a second local environment. |
| `test-worker-debug-mode.ts` | Uses default client/worker connection in integration block. |
| `test-worker-deployment-versioning.ts` | Shared harness; extra native connections now use env auth/TLS, but server feature support needs validation. |
| `test-worker-exposes-abortcontroller.ts` | Uses default client/worker connection in integration block. |
| `test-worker-exposes-textencoderdecoder.ts` | Uses default client/worker connection in integration block. |
| `test-worker-lifecycle.ts` | Uses default client/worker connection in integration block and has runtime/lifecycle-local assertions. |
| `test-worker-no-activities.ts` | Uses default client/worker connection in integration block. |
| `test-worker-no-workflows.ts` | Uses default client/worker connection in integration block. |
| `test-worker-poller-autoscale.ts` | Direct local environment with local Prometheus endpoint. |
| `test-worker-versioning.ts` | Uses default client/worker connection in integration block. |
| `test-workflow-log-interceptor.ts` | Direct `TestWorkflowEnvironment.createLocal()`. |
| `test-workflow-unhandled-rejection-crash.ts` | Uses default client/worker connection in integration block. |
1 change: 1 addition & 0 deletions packages/test/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
"build:ts": "tsc --build",
"build:protos": "tsx ./scripts/compile-proto.ts",
"test": "ava ./lib/test-*.js",
"test:external": "ava ./lib/test-default-activity.js ./lib/test-integration-activities.js ./lib/test-integration-cancellation-scopes.js ./lib/test-integration-replay-and-flags.js ./lib/test-integration-reserved-prefixes.js ./lib/test-integration-update.js ./lib/test-integration-update-interceptors.js ./lib/test-integration-workflows-with-recorded-logs.js ./lib/test-isolation.js ./lib/test-nexus-operation-timeouts.js ./lib/test-nexus-signal-linking.js ./lib/test-nexus-standalone.js ./lib/test-nexus-temporal-operation.js ./lib/test-random-stream-reset.js ./lib/test-worker-tuner.js ./lib/test-workflow-async-local-storage.js ./lib/test-workflow-cancellation.js ./lib/test-workflow-fail-on-errors-policy.js ./lib/test-workflow-nexus-cancellation.js",
"test:watch": "ava --watch ./lib/test-*.js",
"test:bun": "bun run -b ava ./lib/test-*.js"
},
Expand Down
Loading
Loading