Add customUpCodes support for HTTP monitors (Backend)#3690
Add customUpCodes support for HTTP monitors (Backend)#3690harsh-aghara wants to merge 7 commits into
Conversation
Adds a customUpCodes field to the monitor schema to allow users to specify which HTTP status codes should be treated as UP instead of DOWN. Includes core logic in HttpProvider and unit tests. Refs bluewave-labs#3657
ajhollid
left a comment
There was a problem hiding this comment.
I haven't had a chance to do a full review yet, but off the bat this should use the http codes defined in the http library, these should not be arbitrary or self defined numbers.
The http library already properly defines the codes and should be used for type discrimination.
Addresses reviewer feedback by enforcing strict validation on the array. Previously, the Zod schema accepted any arbitrary number, which could allow invalid or self-defined HTTP codes (e.g., 5000) to be saved to the database. Refs bluewave-labs#3657
@ajhollid I just pushed a commit that updates the Zod schemas to validate customUpCodes directly against http.STATUS_CODES. It now uses the http library for discrimination and will reject any fake or arbitrary numbers (like 9999) from being saved. I also added full unit tests for this behavior. |
Br0wnHammer
left a comment
There was a problem hiding this comment.
Overall, the PR looks solid and clean. I have one concern:
got follows redirects by default, so a 301/302 never shows up as the final status; the redirect target's code does. That means adding 3xx codes to `customUpCodes' won't actually work here (the 301 test only passes because the response is mocked). Should 3xx be supported, or should we drop the 301 example?
ajhollid
left a comment
There was a problem hiding this comment.
This needs a bit more work, please see my comments in the review.
Thanks!
| uptimePercentage?: number; | ||
| notifications: string[]; | ||
| tags: string[]; | ||
| customUpCodes: number[]; |
There was a problem hiding this comment.
Let's properly define an HttpStatusCode type and a set we can use to verify
export const HttpStatusCodes = Object.keys(http.STATUS_CODES).map(Number);
export const HttpStatusCodeSet = new Set(HttpStatusCodes);
export type HttpStatusCode = number;
and use it here to properly type this:
customUpCodes: HttpStatusCode[];
| import { DnsRecordTypes, MonitorMatchMethods, MonitorStatuses, MonitorTypes, PageSpeedStrategies } from "@/types/monitor.js"; | ||
| import http from "node:http"; | ||
|
|
||
| const httpStatusCode = z.number().refine((code) => code.toString() in http.STATUS_CODES, { message: "Must be a valid HTTP status code" }); |
There was a problem hiding this comment.
This can now be simplified
const httpStatusCode = z.number().refine((code) => HttpStatusCodeSet.has(code), { message: "Must be a valid HTTP status code" });
There was a problem hiding this comment.
simplified this, thanks for the review!
|
|
||
| private handleHttpError<T>(error: unknown, monitor: Monitor): MonitorStatusResponse<T> { | ||
| if (error instanceof HTTPError || error instanceof RequestError) { | ||
| const isCustomUp = error instanceof HTTPError && (monitor.customUpCodes ?? []).includes(error.response?.statusCode); |
There was a problem hiding this comment.
This is very hard to read and has a superfluous instance assertion. Why not do something like the following:
const statusCode = error.response?.statusCode;
const statusUp = statusCode !== undefined && monitor.customUpCodes.includes(statusCode);
...
return {
monitorId: monitor.id,
teamId: monitor.teamId,
type: monitor.type,
status: statusUp,
code: statusCode ?? NETWORK_ERROR,
message: error.message,
responseTime: error.timings?.phases?.total ?? 0,
timings: error.timings,
payload: null as T,
};
Something along those lines is clearer and much easier to read.
There was a problem hiding this comment.
Done.
broke down code into more readable parts.
| teamId: monitor.teamId, | ||
| type: monitor.type, | ||
| status: response.ok && matchResult.ok, | ||
| status: ((monitor.customUpCodes ?? []).includes(response.statusCode) || response.ok) && matchResult.ok, |
There was a problem hiding this comment.
This difficult to read line then simply becomes
status: statusUp && matchResult.ok,
It's also worth noting that this is acutally not needed since got throws on non 2xx responses.
This is harmless though and we can leave it in in case we decide to change the got default behaviour.
| } | ||
|
|
||
| async pollForResults(measurementId: string, timeoutMs: number = MAX_POLL_TIMEOUT_MS): Promise<GeoCheckResult[]> { | ||
| async pollForResults(measurementId: string, timeoutMs: number = MAX_POLL_TIMEOUT_MS, customUpCodes: number[] = []): Promise<GeoCheckResult[]> { |
There was a problem hiding this comment.
Correct type can be used here
There was a problem hiding this comment.
Done.
Updated globalPingService.ts to use correct type.
@Br0wnHammer Good catch, you are right. Since got follows redirects by default, it will return the final status code instead of the 3xx code in production. To handle this, I see two options:
I think Option 1 is the safer choice. |
Add HttpStatusCode types and simplify provider logic
Thanks for making changes here, this is looking good 🚀 I think the issue around redirects isn't straightforward, especially since I don't know how the geo checks handle redirects. Additionally, What about extracting a utility to deal with status that is independent of something like that, which we can use in both the |
…ok and modified utils test accordingly
|
@ajhollid I've implemented the isStatusUp utility as requested, so the UP status is now determined independently of got. |
ajhollid
left a comment
There was a problem hiding this comment.
The advanced matcher is ignored in the error path. This is as it should have been before this change, but now that there are codes that could be considered as "up" in the error path, the advanced matcher should be run in those cases.
| const statusCode = error.response?.statusCode; | ||
| const statusUp = isStatusUp(statusCode, monitor.customUpCodes); | ||
|
|
||
| return { | ||
| monitorId: monitor.id, | ||
| teamId: monitor.teamId, | ||
| type: monitor.type, | ||
| status: false, | ||
| code: error.response?.statusCode ?? NETWORK_ERROR, | ||
| status: statusUp, | ||
| code: statusCode ?? NETWORK_ERROR, |
There was a problem hiding this comment.
There is an issue here.
If we accept that there are error codes that may actually be considered as "Up", we need to run the advanced matcher here as well.
If a user sets advanced matching and a custom code, their advanced matcher will be ignored.
There was a problem hiding this comment.
Fixed this in the latest commit.
It now properly runs the advanced matcher for custom up codes too.
Thank you for the review.
When customUpCodes mark an HTTPError as up, parse the response body, apply advancedMatcher.validate, and set status to statusUp && matchResult.ok. Return parsed payload and extracted values to match the success path.
…us-codes-backend # Conflicts: # server/src/service/infrastructure/network/HttpProvider.ts
ajhollid
left a comment
There was a problem hiding this comment.
There's too much code duplication in the error branch, it's nearly identical to the success branch. Let's extract it to a helper.
| const { jsonPath } = monitor; | ||
| const body = error.response?.body ?? ""; | ||
| const contentType = error.response?.headers?.["content-type"] || ""; | ||
| const isJson = contentType.includes("application/json"); | ||
|
|
||
| if (jsonPath && !isJson) { | ||
| return { | ||
| monitorId: monitor.id, | ||
| teamId: monitor.teamId, | ||
| type: monitor.type, | ||
| status: false, | ||
| code: statusCode ?? NETWORK_ERROR, | ||
| message: "Response is not JSON", | ||
| responseTime, | ||
| timings: error.timings, | ||
| payload: body as unknown as T, | ||
| }; | ||
| } | ||
|
|
||
| let payload: T; | ||
| if (isJson) { | ||
| try { | ||
| payload = JSON.parse(body) as T; | ||
| } catch { | ||
| payload = body as unknown as T; | ||
| } | ||
| } else { | ||
| payload = body as unknown as T; | ||
| } | ||
|
|
||
| const matchResult = this.advancedMatcher.validate<T>(payload, monitor); | ||
|
|
||
| return { | ||
| monitorId: monitor.id, | ||
| teamId: monitor.teamId, | ||
| type: monitor.type, | ||
| status: false, | ||
| code: error.response?.statusCode ?? NETWORK_ERROR, | ||
| message: error.message, | ||
| responseTime: error.timings?.phases?.firstByte ?? error.timings?.phases?.total ?? 0, | ||
| status: statusUp && matchResult.ok, | ||
| code: statusCode ?? NETWORK_ERROR, |
There was a problem hiding this comment.
This code is nearly identical to the code in the success branch. This should be extracted to a private helper function
There was a problem hiding this comment.
Sure i can create a private method to obey DRY.
Describe your changes
This PR adds backend support for a
customUpCodesfield to HTTP monitors. This feature allows users to specify which non-2xx HTTP status codes (e.g.,401 Unauthorized,301 Moved Permanently) should be treated as a successful "UP" status instead of the default "DOWN".Key Changes:
customUpCodes: number[]to the MongooseMonitorschema and TypeScript types, defaulting to[].monitorValidation.tsto validate the new field on monitor creation and updates.HttpProviderto checkcustomUpCodesduring both success paths and error paths (whengotthrows anHTTPError).GlobalPingServiceandgeoChecksServiceto ensure geographically distributed HTTP checks also respect the custom status codes.httpProvider.test.tsto cover error scenarios, success overrides, empty arrays, and matcher precedence.Write your issue number after "Fixes "
Refs #3657
Please ensure all items are checked off before requesting a review. "Checked off" means you need to add an "x" character between brackets so they turn into checkmarks.
npm run formatin server and client directories, which automatically formats your code.