Add k6 performance tests#3193
Conversation
OpenAPI ChangesNo changes detected Unexpected changes? Ensure your branch is up-to-date with |
13ccfc4 to
d63b3c1
Compare
shanbady
left a comment
There was a problem hiding this comment.
looked good and ran once i tweaked a few things for my local setup.
I think most folks access the api and frontend via different ports vs hostnames - I also dont know if anyone has ssl setup.
Some suggestions I have that could make this an *almost zero-dependency one-liner to run locally
in learn.local.ts:
- remove the api.* from the local hostname
- use port-mapped hosts.
- drop https
should look like this:
module.exports = require("./learn.ts").configure({
apiBaseUrl: "http://learn.odl.local:8063",
frontendBaseUrl: "http://learn.odl.local:8062",
browserOptions: {
ignoreHTTPSErrors: true,
},
})
Then we can remove the "install k6 locally" step and just use their docker image and run directly from the root:
docker run --rm -i -v $PWD/load_testing:/app --add-host learn.odl.local:host-gateway grafana/k6:master-with-browser run /app/learn.local.ts
5e5b36e to
4be5496
Compare
| ignoreHTTPSErrors: Object.hasOwn(__ENV, "BROWSER_IGNORE_HTTPS_ERRORS") | ||
| ? __ENV.BROWSER_IGNORE_HTTPS_ERRORS | ||
| : false, |
There was a problem hiding this comment.
Bug: The BROWSER_IGNORE_HTTPS_ERRORS environment variable is a string, but it's used as a boolean. The string "false" is truthy, leading to inverted logic.
Severity: MEDIUM
Suggested Fix
Explicitly check if the environment variable string is equal to "true" to correctly convert it to a boolean value before assigning it to the ignoreHTTPSErrors option.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: load_testing/config.ts#L7-L9
Potential issue: The `ignoreHTTPSErrors` option in the k6 browser context expects a
boolean value. The code assigns this value directly from the
`__ENV.BROWSER_IGNORE_HTTPS_ERRORS` environment variable, which k6 provides as a string.
In JavaScript, any non-empty string, including the string `"false"`, is evaluated as
truthy. Consequently, when a user sets `BROWSER_IGNORE_HTTPS_ERRORS=false` to enable
HTTPS error checking, the string `"false"` is assigned, which is treated as `true`. This
results in HTTPS errors being ignored, which is the opposite of the user's intent.
shanbady
left a comment
There was a problem hiding this comment.
approved 👍 @rhysyngsun
i'll leave it up to you/non-blocker if you want to use the one-liner docker command over installing k6 on the host machine.
| "is status 200": (r) => r.response.status === 200, | ||
| "has results": (_) => results.length > 0, |
There was a problem hiding this comment.
Bug: If the API returns an empty results array, randomItem(results) returns undefined, causing a TypeError when resource.id is accessed in the subsequent loop.
Severity: MEDIUM
Suggested Fix
Add a conditional guard to ensure the for loop that processes the results only executes if the results array is not empty. For example, wrap the loop in an if (results && results.length > 0) block.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: load_testing/backend/test.ts#L72-L73
Potential issue: In the backend load test, the code fetches a list of resources from an
API. If the API returns an empty `results` array, which is a valid response, the code
proceeds to a loop. Inside this loop, `randomItem(results)` is called, which returns
`undefined`. The subsequent line then attempts to access `resource.id`, which triggers a
`TypeError` because `resource` is `undefined`. This error will crash the virtual user,
disrupting the load test. The existing `check` for `results.length > 0` only records a
metric and does not prevent the error-producing code from executing.
feb46db to
e59392f
Compare
e59392f to
329ac80
Compare
| await carousel | ||
| .locator("article") | ||
| .nth(randomIntBetween(0, articlesCount - 1)) |
There was a problem hiding this comment.
Bug: If articlesCount is 0, the code attempts to click on an element in an empty locator, which will throw an error and crash the load test's virtual user.
Severity: MEDIUM
Suggested Fix
Add a conditional check to ensure articlesCount is greater than 0 before attempting to click on an article. This will prevent the script from trying to access an element in an empty collection and avoid the runtime error.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: load_testing/frontend/test.ts#L15-L17
Potential issue: In the load test script, if the carousel locator finds no `article`
elements, `articlesCount` will be 0. The subsequent call to `randomIntBetween(0,
articlesCount - 1)` will evaluate to `randomIntBetween(0, -1)`, which returns 0. The
code then attempts to execute `.nth(0).click()` on an empty locator. This action will
throw an unhandled error, causing the k6 virtual user iteration to crash. This can
happen under realistic test conditions, such as an empty database or slow page load,
leading to inaccurate performance metrics.
What are the relevant tickets?
Part of the work described in https://github.com/mitodl/hq/discussions/10883
Description (What does it do?)
Adds k6 testing for APIs and frontend pages that are publicly accessible.
Note: there is some work around authentication included, it's still a bit of a work in progress but I'd rather not throw it out so I'm keeping it in this PR as is causes no harm as long as the environment variables aren't set.
How can this be tested?
You can
cd load_testingand then runk6 run learn.local.tsto run against your local system.