CQL Test Server for CQL Tests Runner (#360)

Implement a test server that exposes a `$cql` target to allow the CQL Tests Runner to run tests against the `cql-execution`. The test server is a sub-project at `test-server` with its own `package.json` file and corresponding scripts. See test-server/README.md for further details.

Author: cmoesel

.github/workflows/ci-workflow.yml

@@ -29,7 +29,7 @@ jobs:
       - uses: actions/setup-node@v6
         with:
           node-version: 24.x
-      - run: npm install
+      - run: npm ci
       - run: npm run lint
       - run: npm run prettier
       - run: ./bin/check_for_nonassertive_tests.sh
@@ -49,7 +49,7 @@ jobs:
         with:
           timezone: America/New_York
       - run: date +"%Y-%m-%d %T"
-      - run: npm install
+      - run: npm ci
       - run: npx nyc --reporter=lcov npm test
         env:
           CI: true
@@ -58,3 +58,18 @@ jobs:
         uses: codecov/codecov-action@v5
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
+  test-server:
+    name: Check test-server
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+      - uses: actions/setup-node@v6
+        with:
+          node-version: 24.x
+      - run: npm ci
+      - run: npm ci
+        working-directory: test-server
+      - run: npm run build
+        working-directory: test-server
+      - run: npm run check
+        working-directory: test-server

.prettierignore

@@ -5,6 +5,8 @@ node_modules
 .vscode
 coverage
 cql4browsers.js
+# the following uses its own prettier config
+test-server
 # the following are generated by the test generator
 test/elm/*/data.js
 test/elm/external/Included.js

README.md

@@ -350,6 +350,10 @@ modification to the test data text file, you can setup a process to _watch_ for
 regenerate the `data.js` files every time it detects changes in the source text file.  Simply
 execute `npm run watch:test-data`.
 
+# Using the Test Server and CQL Tests Runner
+
+See [test-server/README.md](test-server/README.md).
+
 # Pull Requests
 
 If TypeScript source code is modified, `cql4browsers.js` needs to be included in the pull request,

bin/validate_cql4browsers.sh

@@ -13,7 +13,7 @@ if [ $? != 0 ]; then
   echo "cql4browsers.js not found. Run this script from the base repository directory."
   exit 1
 fi
-npm install
+npm ci
 
 # comm -3 only returns lines that differ between the two files. If none are different, diff will be empty
 diff=`diff ./examples/browser/cql4browsers.js.original ./examples/browser/cql4browsers.js`

eslint.config.mjs

@@ -19,6 +19,7 @@ export default defineConfig([
     '**/.eslintrc.json',
     '**/lib',
     '**/lib-test',
+    'test-server',
     'examples/browser/cql4browsers.js',
     'test/elm/library/data-with-namespace.js'
   ]),

test-server/.gitignore

@@ -0,0 +1,32 @@
+lib-cov
+*.seed
+*.log
+*.csv
+*.dat
+*.out
+*.pid
+*.gz
+*.swp
+
+pids
+logs
+results
+tmp
+
+# API keys and secrets
+.env
+
+# Dependency directory
+node_modules
+
+# Editors
+.vscode
+.idea
+*.iml
+
+# OS metadata
+.DS_Store
+Thumbs.db
+
+# Ignore built ts files
+dist/**/*

test-server/.prettierrc

@@ -0,0 +1,8 @@
+{
+  "trailingComma": "none",
+  "arrowParens": "avoid",
+  "endOfLine": "auto",
+  "printWidth": 100,
+  "tabWidth": 2,
+  "singleQuote": true
+}

test-server/README.md

@@ -0,0 +1,122 @@
+# CQL Test Server
+
+A barebones implementation of the FHIR [$cql](https://build.fhir.org/ig/HL7/cql-ig/en/OperationDefinition-cql-cql.html) operation that evaluates CQL expressions using the CQL execution engine and uses the [CQL FHIR Type Mapping](https://build.fhir.org/ig/HL7/cql-ig/en/conformance.html#fhir-type-mapping) to return results as a FHIR `Parameters` resource. This server is intended to be used as a target for the [cql-tests-runner](https://github.com/cqframework/cql-tests-runner) and is tuned specifically for that use case.
+
+## Endpoint
+
+- POST `/fhir/$cql`
+  - Example request:
+    ```json
+    {
+      "resourceType": "Parameters",
+      "parameter": [{ "name": "expression", "valueString": "1 + 2" }]
+    }
+    ```
+  - Example response:
+    ```json
+    {
+      "resourceType": "Parameters",
+      "parameter": [{
+          "extension": [{
+            "url": "http://hl7.org/fhir/StructureDefinition/cqf-cqlType",
+            "valueString": "System.Integer"
+          }],
+          "name": "return",
+          "valueInteger": 3
+      }]
+    }
+    ```
+  - Errors:
+    - HTTP 400 if the `expression` parameter is missing or empty
+    - HTTP 500 if there is an unexpected / unrecoverable server-side error
+
+## Prerequisites
+
+This project uses Node.js. It has been tested using Node 24 but should work with any recent LTS version.
+
+## Developing Scripts
+
+The `package.json` file defines scripts to support development of the test-server source code. These include:
+
+- `npm run lint`: run eslint against the test-server source code
+- `npm run lint:fix`: fix eslint errors (as possible)
+- `npm run prettier`: run prettier against the test-server source code
+- `npm run prettier:fix`: fix prettier violations (as possible)
+- `npm test`: run the unit tests
+
+Scripts for building and running the server are documented below.
+
+## Start The Server
+
+From this directory:
+
+### Install dependencies:
+
+```sh
+npm install
+```
+
+### Development mode:
+
+```sh
+npm run dev
+```
+
+The development mode builds the `cql-execution` source (to ensure it uses the latest changes) before running the server. It watches for changes to the `test-server` source and hot-reloads as necessary. By default, it uses the javascript CQL-to-ELM translator and listens on `http://localhost:8000`.
+
+_NOTE: The watch mode will NOT detect or hot-reload changes in the `cql-execution` source. You must stop and restart the `test-server` to pick up changes in the `cql-execution` source._
+
+# Production mode:
+
+```sh
+npm run build
+npm start
+```
+
+The production mode builds the `cql-execution` source and `test-server` source then runs the server _without_ watching for changes. By default, it uses the javascript CQL-to-ELM translator and listens on `http://localhost:8000`.
+
+### Configuration
+
+- `PORT` (optional): Port to bind. Defaults to `8000`.
+- `USE_TRANSLATION_SERVICE` (optional, default: `false`): If `true`, the server will use a local [CQL Translation Service](https://github.com/cqframework/cql-translation-service) for CQL-to-ELM. In this case, the CQL Translation service must be running and available at `http://localhost:8080`. If `false`, the server will use the javascript CQL-to-ELM module to compile CQL.
+
+You can export these variables before running the server or place them in a `.env` file:
+
+```ini
+PORT=8000
+USE_TRANSLATION_SERVICE=false
+```
+
+## Quick Test With curl
+
+```sh
+curl -X POST http://localhost:8000/fhir/\$cql \
+  -H "Content-Type: application/fhir+json" \
+  -d '{
+    "resourceType": "Parameters",
+    "parameter": [
+      { "name": "expression", "valueString": "1 + 2" }
+    ]
+  }'
+```
+
+## Use With cql-tests-runner
+
+This server is designed to be a `$cql` target for the [cql-tests-runner](https://github.com/cqframework/cql-tests-runner). To execute the test-runner against this server:
+
+1. Download or clone the `cql-tests-runner` repository.
+
+2. Start this test-server (see above), e.g. at `http://localhost:8000`.
+
+3. In cql-tests-runner folder, create or edit a [test runner configuration](https://github.com/cqframework/cql-tests-runner?tab=readme-ov-file#configuration-settings) that points to the test-server endpoint. If you are using port 8000, you may choose to use the existing [conf/cql-execution-local.json](https://github.com/cqframework/cql-tests-runner/blob/main/conf/cql-execution-local.json) file.
+
+4. Run the tests from the cql-tests-runner folder:
+
+  ```sh
+  npm install
+  git submodule update --init --recursive
+  npx tsx src/bin/cql-tests.ts run-tests conf/cql-execution-local.json ./results
+  ```
+
+The runner will POST each test expression to `http://localhost:8000/fhir/$cql` and expect FHIR `Parameters` responses per the mapping. The test-server logs each incoming expression, the raw result, and the result mapped to `FHIR Parameters`. When the run is done, you can find the results in the cql-tests-runner folder's `results` sub-folder.
+

test-server/eslint.config.mjs

@@ -0,0 +1,52 @@
+import { defineConfig, globalIgnores } from 'eslint/config';
+import globals from 'globals';
+import tsParser from '@typescript-eslint/parser';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import js from '@eslint/js';
+import { FlatCompat } from '@eslint/eslintrc';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const compat = new FlatCompat({
+  baseDirectory: __dirname,
+  recommendedConfig: js.configs.recommended,
+  allConfig: js.configs.all
+});
+
+export default defineConfig([
+  globalIgnores(['dist', 'node_modules']),
+  {
+    extends: compat.extends('plugin:@typescript-eslint/recommended', 'prettier'),
+    languageOptions: {
+      globals: {
+        ...globals.node,
+        ...globals.mocha
+      },
+      parser: tsParser,
+      ecmaVersion: 6,
+      sourceType: 'module',
+      parserOptions: {
+        ecmaFeatures: { experimentalObjectRestSpread: true }
+      }
+    },
+    rules: {
+      '@typescript-eslint/no-require-imports': 'off',
+      '@typescript-eslint/no-explicit-any': 'off',
+      'no-unused-vars': 'off', // redundant to rule below
+      '@typescript-eslint/no-unused-vars': ['error', { argsIgnorePattern: '^_' }],
+      indent: ['error', 2, { SwitchCase: 1 }],
+      'no-console': ['warn', { allow: ['warn', 'error'] }],
+      'no-loss-of-precision': 'off',
+      quotes: ['error', 'single', { allowTemplateLiterals: true, avoidEscape: true }],
+      semi: ['error', 'always'],
+      curly: 'error'
+    }
+  },
+  {
+    files: ['test/**/*.{js,ts}', 'examples/**/*.js', 'bin/**/*.js'],
+    rules: {
+      '@typescript-eslint/no-var-requires': 'off'
+    }
+  }
+]);