Skip to content

Add k6 performance tests#3193

Merged
rhysyngsun merged 1 commit into
mainfrom
performance-tests
Apr 17, 2026
Merged

Add k6 performance tests#3193
rhysyngsun merged 1 commit into
mainfrom
performance-tests

Conversation

@rhysyngsun

@rhysyngsun rhysyngsun commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

What are the relevant tickets?

Part of the work described in https://github.com/mitodl/hq/discussions/10883

Description (What does it do?)

Adds k6 testing for APIs and frontend pages that are publicly accessible.

Note: there is some work around authentication included, it's still a bit of a work in progress but I'd rather not throw it out so I'm keeping it in this PR as is causes no harm as long as the environment variables aren't set.

How can this be tested?

You can cd load_testing and then run k6 run learn.local.ts to run against your local system.

@github-actions

github-actions Bot commented Apr 13, 2026

Copy link
Copy Markdown

OpenAPI Changes

No changes detected

View full changelog

Unexpected changes? Ensure your branch is up-to-date with main (consider rebasing).

Comment thread load_testing/backend/client/v0/api.ts Fixed
@rhysyngsun rhysyngsun marked this pull request as ready for review April 15, 2026 20:50
Comment thread load_testing/backend/test.ts Outdated
@shanbady shanbady self-assigned this Apr 17, 2026
@shanbady shanbady self-requested a review April 17, 2026 13:53

@shanbady shanbady left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looked good and ran once i tweaked a few things for my local setup.
I think most folks access the api and frontend via different ports vs hostnames - I also dont know if anyone has ssl setup.

Some suggestions I have that could make this an *almost zero-dependency one-liner to run locally

in learn.local.ts:

  • remove the api.* from the local hostname
  • use port-mapped hosts.
  • drop https

should look like this:

module.exports = require("./learn.ts").configure({
  apiBaseUrl: "http://learn.odl.local:8063",
  frontendBaseUrl: "http://learn.odl.local:8062",
  browserOptions: {
    ignoreHTTPSErrors: true,
  },
})

Then we can remove the "install k6 locally" step and just use their docker image and run directly from the root:

docker run --rm -i -v $PWD/load_testing:/app  --add-host learn.odl.local:host-gateway  grafana/k6:master-with-browser run /app/learn.local.ts

Comment thread load_testing/config.ts
Comment on lines +7 to +9
ignoreHTTPSErrors: Object.hasOwn(__ENV, "BROWSER_IGNORE_HTTPS_ERRORS")
? __ENV.BROWSER_IGNORE_HTTPS_ERRORS
: false,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The BROWSER_IGNORE_HTTPS_ERRORS environment variable is a string, but it's used as a boolean. The string "false" is truthy, leading to inverted logic.
Severity: MEDIUM

Suggested Fix

Explicitly check if the environment variable string is equal to "true" to correctly convert it to a boolean value before assigning it to the ignoreHTTPSErrors option.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: load_testing/config.ts#L7-L9

Potential issue: The `ignoreHTTPSErrors` option in the k6 browser context expects a
boolean value. The code assigns this value directly from the
`__ENV.BROWSER_IGNORE_HTTPS_ERRORS` environment variable, which k6 provides as a string.
In JavaScript, any non-empty string, including the string `"false"`, is evaluated as
truthy. Consequently, when a user sets `BROWSER_IGNORE_HTTPS_ERRORS=false` to enable
HTTPS error checking, the string `"false"` is assigned, which is treated as `true`. This
results in HTTPS errors being ignored, which is the opposite of the user's intent.

@shanbady shanbady left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved 👍 @rhysyngsun

i'll leave it up to you/non-blocker if you want to use the one-liner docker command over installing k6 on the host machine.

Comment on lines +72 to +73
"is status 200": (r) => r.response.status === 200,
"has results": (_) => results.length > 0,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: If the API returns an empty results array, randomItem(results) returns undefined, causing a TypeError when resource.id is accessed in the subsequent loop.
Severity: MEDIUM

Suggested Fix

Add a conditional guard to ensure the for loop that processes the results only executes if the results array is not empty. For example, wrap the loop in an if (results && results.length > 0) block.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: load_testing/backend/test.ts#L72-L73

Potential issue: In the backend load test, the code fetches a list of resources from an
API. If the API returns an empty `results` array, which is a valid response, the code
proceeds to a loop. Inside this loop, `randomItem(results)` is called, which returns
`undefined`. The subsequent line then attempts to access `resource.id`, which triggers a
`TypeError` because `resource` is `undefined`. This error will crash the virtual user,
disrupting the load test. The existing `check` for `results.length > 0` only records a
metric and does not prevent the error-producing code from executing.

Comment on lines +15 to +17
await carousel
.locator("article")
.nth(randomIntBetween(0, articlesCount - 1))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: If articlesCount is 0, the code attempts to click on an element in an empty locator, which will throw an error and crash the load test's virtual user.
Severity: MEDIUM

Suggested Fix

Add a conditional check to ensure articlesCount is greater than 0 before attempting to click on an article. This will prevent the script from trying to access an element in an empty collection and avoid the runtime error.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: load_testing/frontend/test.ts#L15-L17

Potential issue: In the load test script, if the carousel locator finds no `article`
elements, `articlesCount` will be 0. The subsequent call to `randomIntBetween(0,
articlesCount - 1)` will evaluate to `randomIntBetween(0, -1)`, which returns 0. The
code then attempts to execute `.nth(0).click()` on an empty locator. This action will
throw an unhandled error, causing the k6 virtual user iteration to crash. This can
happen under realistic test conditions, such as an empty database or slow page load,
leading to inaccurate performance metrics.

@rhysyngsun rhysyngsun merged commit d19b004 into main Apr 17, 2026
14 checks passed
@rhysyngsun rhysyngsun deleted the performance-tests branch April 17, 2026 20:39
@odlbot odlbot mentioned this pull request Apr 20, 2026
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants