Migrate tests/workflows to node:test and update docs

Replace the legacy Jest-based test setup and dotenv loader with Node's built-in test runner and native env-file flags. GitHub Actions workflows were reworked: job 'test' renamed to 'local_seams', new 'integration_seams' (creates .env from secrets, sets up Node 24, caches npm, runs npm run test:integration) and coordinated bump_version job added; CD jobs now run npm run test:integration for smoke checks. Documentation and guides (.github/copilot-instructions.md, CONFIG.md, CONTRIBUTING.md, README.md) updated to reflect Node native env-file usage, updated test script names (test:local, test:integration, test:coverage) and testing guidance. Removed import of env-loader from bin/tpen3_services.js. Large set of legacy Jest tests under __tests__ and various class test folders were removed and replaced by new test suites under test/local and test/integration; package.json/package-lock were updated accordingly.

Author: cubap

.github/copilot-instructions.md

@@ -16,7 +16,7 @@ TPEN Services is a Node.js Express API service for TPEN3 (Transcription for Pale
 
 ### Environment Configuration
 
-TPEN Services uses a layered configuration approach with `--import ./env-loader.js` using the dotenv package:
+TPEN Services uses a layered configuration approach with Node native env-file flags:
 
 - `config.env` - Safe defaults (committed to repo, no secrets)
   - Works out-of-the-box for local Docker/MongoDB/MariaDB
@@ -33,10 +33,10 @@ TPEN Services uses a layered configuration approach with `--import ./env-loader.
   - Contains actual secrets and environment-specific values
   - Overrides values from `config.env`
 
-Configuration loading order (via `--import ./env-loader.js` using the dotenv package):
+Configuration loading order (via Node CLI env-file flags in npm scripts):
 
 1. `config.env` is loaded first (provides safe defaults)
-2. `.env.{NODE_ENV}` is loaded second (environment-specific: .env.development, .env.production, .env.test)
+2. `.env.development` is loaded second (development defaults)
 3. `.env` is loaded last (local/server overrides - HIGHEST PRIORITY)
 
 This allows developers to work immediately with sensible defaults while keeping secrets out of the repository.
@@ -48,19 +48,15 @@ This allows developers to work immediately with sensible defaults while keeping
 - Start the application: `npm start` or `npm run dev`
 - Test the root endpoint: `curl http://localhost:3011/` -- should return HTML containing the TPEN3 Services index (heading + welcome text)
 - Run unit tests that don't require databases: `npm run unitTests` -- many tests pass without database connections
-- Run existence tests: `npm run existsTests` -- validates route registration and class imports
 - Run all tests: `npm run allTests` -- Full test suite confirming full app functionality
 - ALWAYS wait for full test completion. Tests may appear to hang but should complete within 2 minutes. 
 - NOTE: Application may crash after serving initial requests due to database connection attempts - this is expected behavior without running MongoDB/MariaDB.
 
 ### Test Categories Available
 - `npm run allTests` -- Full test suite which requires .env settings
-- `npm run unitTests` -- Core unit tests (some require databases)
-- `npm run existsTests` -- Route and class existence validation (database-independent)
-- `npm run functionsTests` -- Function-level tests
-- `npm run E2Etests` -- End-to-end API tests
-- `npm run dbTests` -- Database-specific tests (require running databases)
-- `npm run authTest` -- Authentication tests (require Auth0 configuration)
+- `npm run unitTests` -- Fast local seam checks
+- `npm run E2Etests` -- CI-oriented integration seam checks
+- `npm run test:coverage` -- Coverage report via c8
 
 ### Expected Test Behavior
 - Tests requiring databases will timeout/fail without MongoDB/MariaDB running
@@ -108,8 +104,8 @@ Required for external services:
 1. Do not overwrite the existing .env file.  If an .env file does not exist or is not populated then copy environment configuration: `cp .env.development .env`
 2. Install dependencies with `npm install`
 3. Make code changes
-4. Test with: `npm run existsTests` (fast, database-independent)
-5. For all other tests use `npm run allTests`
+4. Test with: `npm run unitTests` (fast, database-independent)
+5. For full validation use `npm run allTests`
 6. Test manually: `curl http://localhost:3011/` and relevant endpoints
 
 NEVER CANCEL long-running commands. Application builds and tests are designed to complete within documented timeouts. Always wait for completion to ensure accurate validation of changes.

.github/workflows/cd_dev.yaml

@@ -86,4 +86,4 @@ jobs:
       - name: Run smoke tests
         run: |
           cd /srv/node/tpen-services/
-          SMOKE_TEST_URL=https://dev.api.t-pen.org node __tests__/smoke.test.js
+          npm run test:integration

.github/workflows/cd_prod.yaml

@@ -86,4 +86,4 @@ jobs:
       - name: Run smoke tests
         run: |
           cd /srv/node/tpen-services/
-          SMOKE_TEST_URL=https://api.t-pen.org node __tests__/smoke.test.js
+          npm run test:integration

.github/workflows/ci_dev.yaml

@@ -3,7 +3,7 @@ on:
   pull_request:
     branches: development
 jobs:
-  test:
+  local_seams:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@master
@@ -50,8 +50,62 @@ jobs:
       - name: Install dependencies and run the test
         run: |
           npm install
-          npm run E2Etests
+          npm run test:local
 
+  integration_seams:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@master
+      - name: Create .env from secrets
+        uses: SpicyPizza/create-envfile@v2
+        with:
+          fail_on_empty: false
+          sort_keys: false
+          envkey_DOWN: ${{ secrets.DOWN }}
+          envkey_SERVERURL: ${{ secrets.SERVERURL }}
+          envkey_MONGODBNAME: ${{ secrets.MONGODBNAME }}
+          envkey_MONGODB: ${{ secrets.MONGODB }}
+          envkey_TPENPROJECTS: ${{ secrets.TPENPROJECTS }}
+          envkey_TPENGROUPS: ${{ secrets.TPENGROUPS }}
+          envkey_TPENUSERS: ${{ secrets.TPENUSERS }}
+          envkey_MARIADBNAME: ${{ secrets.MARIADBNAMEDEV }}
+          envkey_MARIADB: ${{ secrets.MARIADB }}
+          envkey_MARIADBUSER: ${{ secrets.MARIADBUSERDEV }}
+          envkey_MARIADBPASSWORD: ${{ secrets.MARIADBPASSWORDDEV }}
+          envkey_RERUMIDPREFIX: ${{ secrets.RERUMIDPREFIXDEV }}
+          envkey_TINYPEN: ${{ secrets.TINYPENDEV }}
+          envkey_AUDIENCE: ${{ secrets.AUDIENCE }}
+          envkey_DOMAIN: ${{ secrets.DOMAIN }}
+          envkey_TPEN_SUPPORT_EMAIL: ${{ secrets.TPEN_SUPPORT_EMAIL }}
+          envkey_SMTP_HOST: ${{ secrets.SMTP_HOST }}
+          envkey_SMTP_PORT: ${{ secrets.SMTP_PORT }}
+          envkey_TPEN_EMAIL_CC: ${{ secrets.TPEN_EMAIL_CC }}
+      - name: Setup Node.js
+        uses: actions/setup-node@master
+        with:
+          node-version: "24"
+      - name: Cache node modules
+        uses: actions/cache@master
+        env:
+          cache-name: cache-node-modules
+        with:
+          path: ~/.npm
+          key: ${{ runner.os }}-build-${{ env.cache-name }}-${{
+            hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-build-${{ env.cache-name }}-
+            ${{ runner.os }}-build-
+            ${{ runner.os }}-
+      - name: Install dependencies and run the test
+        run: |
+          npm install
+          npm run test:integration
+
+  bump_version:
+    runs-on: ubuntu-latest
+    needs: [local_seams, integration_seams]
+    steps:
+      - uses: actions/checkout@master
       - name: Bump version (patch + prerelease)
         run: |
           git config user.name "github-actions[bot]"

.github/workflows/ci_prod.yaml

@@ -3,7 +3,7 @@ on:
   pull_request:
     branches: main
 jobs:
-  test:
+  local_seams:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@master
@@ -50,8 +50,62 @@ jobs:
       - name: Install dependencies and run the test
         run: |
           npm install
-          npm run E2Etests
+          npm run test:local
 
+  integration_seams:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@master
+      - name: Create .env from secrets
+        uses: SpicyPizza/create-envfile@v2
+        with:
+          fail_on_empty: false
+          sort_keys: false
+          envkey_DOWN: ${{ secrets.DOWN }}
+          envkey_SERVERURL: ${{ secrets.SERVERURL }}
+          envkey_MONGODBNAME: ${{ secrets.MONGODBNAME }}
+          envkey_MONGODB: ${{ secrets.MONGODB }}
+          envkey_TPENPROJECTS: ${{ secrets.TPENPROJECTS }}
+          envkey_TPENGROUPS: ${{ secrets.TPENGROUPS }}
+          envkey_TPENUSERS: ${{ secrets.TPENUSERS }}
+          envkey_MARIADBNAME: ${{ secrets.MARIADBNAME }}
+          envkey_MARIADB: ${{ secrets.MARIADB }}
+          envkey_MARIADBUSER: ${{ secrets.MARIADBUSER }}
+          envkey_MARIADBPASSWORD: ${{ secrets.MARIADBPASSWORD }}
+          envkey_RERUMIDPREFIX: ${{ secrets.RERUMIDPREFIX }}
+          envkey_TINYPEN: ${{ secrets.TINYPEN }}
+          envkey_AUDIENCE: ${{ secrets.AUDIENCE }}
+          envkey_DOMAIN: ${{ secrets.DOMAIN }}
+          envkey_TPEN_SUPPORT_EMAIL: ${{ secrets.TPEN_SUPPORT_EMAIL }}
+          envkey_SMTP_HOST: ${{ secrets.SMTP_HOST }}
+          envkey_SMTP_PORT: ${{ secrets.SMTP_PORT }}
+          envkey_TPEN_EMAIL_CC: ${{ secrets.TPEN_EMAIL_CC }}
+      - name: Setup Node.js
+        uses: actions/setup-node@master
+        with:
+          node-version: "24"
+      - name: Cache node modules
+        uses: actions/cache@master
+        env:
+          cache-name: cache-node-modules
+        with:
+          path: ~/.npm
+          key: ${{ runner.os }}-build-${{ env.cache-name }}-${{
+            hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-build-${{ env.cache-name }}-
+            ${{ runner.os }}-build-
+            ${{ runner.os }}-
+      - name: Install dependencies and run the test
+        run: |
+          npm install
+          npm run test:integration
+
+  bump_version:
+    runs-on: ubuntu-latest
+    needs: [local_seams, integration_seams]
+    steps:
+      - uses: actions/checkout@master
       - name: Bump version (minor + prerelease)
         run: |
           git config user.name "github-actions[bot]"

CONFIG.md

@@ -2,7 +2,7 @@
 
 ## Configuration Architecture
 
-TPEN Services uses a layered configuration approach to separate safe defaults from environment-specific secrets. Configuration is loaded using `--import ./env-loader.js` with the dotenv package:
+TPEN Services uses a layered configuration approach to separate safe defaults from environment-specific secrets. Configuration is loaded using Node native env-file flags in npm scripts:
 
 1. **`config.env`** (committed) - Safe defaults for development
 2. **`.env`** (gitignored) - Environment-specific overrides
@@ -47,10 +47,10 @@ cp .env.production .env
 # Optional: Database connections, SMTP, etc.
 ```
 
-The application loads configuration via `--import ./env-loader.js` using the dotenv package in this order:
+The application loads configuration via Node flags in this order:
 
 1. `config.env` - provides safe defaults
-2. `.env.{NODE_ENV}` - environment-specific overrides (.env.development, .env.production, .env.test)
+2. `.env.development` - development template values
 3. `.env` - local/server overrides (HIGHEST PRIORITY)
 
 ### What Goes Where?

CONTRIBUTING.md

@@ -126,7 +126,7 @@ curl http://localhost:3011
 
 ## Testing
 
-TPEN Services uses Jest for testing with multiple test suites:
+TPEN Services uses Node.js built-in test runner (`node:test`) with `c8` for coverage:
 
 ### Run All Tests
 ```bash
@@ -135,31 +135,18 @@ npm run allTests
 
 ### Run Specific Test Suites
 ```bash
-# Unit tests only
+# Fast local seam checks
 npm run unitTests
 
-# End-to-end tests
+# CI-oriented integration seam checks
 npm run E2Etests
 
-# Database tests
-npm run dbTests
-
-# Authentication tests
-npm run authTest
-
-# User class tests
-npm run userClassTests
-
-# Import functionality tests
-npm run importTests
-
-# Member invitation tests
-npm run inviteMemberTests
+# Coverage report
+npm run test:coverage
 ```
 
 ### Test Requirements
-- Some tests require database connections and will be skipped if databases are not available
-- Authentication tests require proper Auth0 configuration ( currently skipped )
+- Current suites focus on API seam and validation contract regressions, and are designed to run without live DB dependencies
 - Ensure your `.env` file is properly configured before running tests
 
 ## Code Style and Best Practices
@@ -179,7 +166,7 @@ npm run inviteMemberTests
 - **API Routes**: Organized in feature folders (`project/`, `manifest/`, `line/`, etc.)
 - **Classes**: Located in `classes/` directory
 - **Utilities**: Helper functions in `utilities/` directory
-- **Tests**: Use `__tests__` directories or `.test.js` suffix
+- **Tests**: Use the `test/local` and `test/integration` directories with `.test.js` files
 
 ### Error Handling
 - Use appropriate HTTP status codes
@@ -245,7 +232,7 @@ TPEN-services/
 ├── line/              # Line/annotation routes
 ├── userProfile/       # User profile routes
 ├── utilities/         # Helper functions
-├── __tests__/         # Global tests
+├── test/              # node:test suites (local and integration)
 ├── app.js             # Express app configuration
 ├── package.json       # Dependencies and scripts
 ├── config.env         # Safe defaults (committed)
@@ -271,7 +258,7 @@ TPEN-services/
 - [TPEN Project Homepage](https://t-pen.org/TPEN3)
 - [Express.js Documentation](https://expressjs.com/)
 - [MongoDB Documentation](https://docs.mongodb.com/)
-- [Jest Testing Framework](https://jestjs.io/)
+- [Node.js Test Runner](https://nodejs.org/api/test.html)
 - [Auth0 Documentation](https://auth0.com/docs)
 
 ## License

README.md

@@ -36,7 +36,7 @@ Services required by TPEN interfaces in order to interact with data.
 
 ### Configuration Files
 
-TPEN Services uses a layered configuration approach with `--import ./env-loader.js` using the dotenv package:
+TPEN Services uses a layered configuration approach with Node's native env-file support:
 
 - **`config.env`** - Safe defaults, committed to repository
   - Works out-of-the-box for local Docker/MongoDB/MariaDB
@@ -58,10 +58,10 @@ TPEN Services uses a layered configuration approach with `--import ./env-loader.
   - Contains your specific settings and secrets
   - Overrides values from `config.env`
 
-Configuration is loaded via `--import ./env-loader.js` using the dotenv package:
+Configuration is loaded via Node CLI flags in npm scripts:
 
 1. `config.env` is loaded first (provides safe defaults)
-2. `.env.{NODE_ENV}` is loaded second (environment-specific: .env.development, .env.production, .env.test)
+2. `.env.development` is loaded second (development defaults)
 3. `.env` is loaded last (local/server overrides - HIGHEST PRIORITY)
 
 ### Environment Variables
@@ -79,14 +79,17 @@ See [CONFIG.md](./CONFIG.md) for complete configuration documentation.
 ## Testing
 
 ```bash
-# Run all tests
+# Fast local regression suite (node:test)
+npm test
+
+# Run local + integration seams
 npm run allTests
 
-# Run specific test suites
-npm run unitTests      # Core unit tests
-npm run existsTests    # Route/class validation
-npm run E2Etests       # End-to-end API tests
-npm run dbTests        # Database tests
+# Run only CI-oriented integration seams
+npm run E2Etests
+
+# Coverage with c8
+npm run test:coverage
 ```
 
 ## Deployment