Expose an `id` for concurrent test runners (like `JEST_WORKER_ID`)

[blimmer]

What is the problem this feature will solve?

When running tests concurrently (the default setting in the Node test runner), it's common practice to split concurrent test runs across multiple local resources (like locally running servers, databases, etc).

Many other popular test runners offer this feature via an environment variable:

• jest: JEST_WORKER_ID
• mocha: MOCHA_WORKER_ID
• vitest: VITEST_POOL_ID

This makes it very easy to set up the proper database connection. For example, here's a snippet from my codebase:

if (process.env.STAGE === "test" && process.env.JEST_WORKER_ID) {
  process.env.DB_DATABASE = `myapp_tests_${process.env.JEST_WORKER_ID}`;
}

Then, in my database docker container, I create n databases, one for each worker. For example, if I run jest with --maxWorkers 8, I create myapp_tests_1 -> myapp_tests_8. During tests, the parallel test suites talk to a different database to avoid deadlocks.

What is the feature you are proposing to solve the problem?

The simplest solution for users coming from other frameworks would be to provide an environment variable just like the other test runners (e.g., NODE_TEST_WORKER_ID).

However, if that's not feasible, adding a workerId to the TestContext could also be a good solution. This solution is not as ideal as the environment variable because people would need to pass the test context to whatever establishes their db/server/etc connection. In my case, at least, this would require refactoring of code. Right now, I rely on that variable being set at file import time.

What alternatives have you considered?

I considered trying to find a solution based on pid, but it's not reliable enough. @redyetidev mentioned in Slack that there might be a solution using hooks. I have not dug into this yet.

[cjihrig]

We could add an environment variable fairly easily, but I have a question:

if (process.env.STAGE === "test" && process.env.JEST_WORKER_ID) {
  process.env.DB_DATABASE = `myapp_tests_${process.env.JEST_WORKER_ID}`;
}

There is already a NODE_TEST_CONTEXT environment variable that can be used to determine that the test runner is being used (if you actually need something like that). Why not use something like crypto.randomUUID() to create a unique ID for each file?

[blimmer]

The problem is that, as an external step before the test runner is launched, I:

• Start up the postgres instance via docker compose
• Run an init script to create n databases (based on the parallelism I run the tests with)
• Run my migration logic (node-pg-migrate) against all the test databases
• Create a flush_database() function in each test database to run beforeEach test

I don't want to do this for each test file because the migration logic is quite slow. It's much faster to migrate once and have a fast flush_database function to reset database state between tests.

So a UUID, for my use case, would not be appropriate. Having a sequential integer between 1 and n (where n is the max concurrency) is required for the database setup logic that occurs outside the test runner context.

[cjihrig]

Got it. Thanks for the explanation. I think we should add support for this.

[0hmX]

I can work on this! will pick this up!

[cjihrig]

Thanks @cu8code. Please be sure to handle both cases of test isolation. When --experimental-test-isolation=process, each child process should get an environment variable from 1 to N. When --experimental-test-isolation=none, only the value of 1 should be used.

[hpatel292-seneca]

Hi @cu8code, are you working on this? If not I can give it a try.

[0hmX]

I am working on this 👍

[JakobJingleheimer]

Is worker.threadId possibly what you're looking for?