API Performance & Error Analytics

[yash-pouranik]

Goal: Provide project owners with insights into their API's health and speed.

Context: The dashboard currently only shows total requests but lacks data on latency (response time) and error rates.

Implementation Requirements:

• Create a global middleware in apps/public-api to record the response-time and status code of every request.
• Store this data efficiently (e.g., using a dedicated Analytics collection or aggregation logic).
• Create endpoints in apps/dashboard-api that return "Average Response Time" and "Error %" for a specific time range.
• Ensure the logging process is asynchronous so it does not impact main API performance.

[MdTowfikomer]

Hi @yash-pouranik, could you assign this issue to me? I’d like to work on implementing the API latency and error rate analytics.

[avanithumballi]

@yash-pouranik I can implement the analytics middleware, storage, and dashboard endpoints with proper async logging. Please assign this to me.

[yash-pouranik]

@MdTowfikomer
@avanithumballi

I have to choose b/w you so can u both please share ur Approach.

[avanithumballi]

@yash-pouranik this is my approach:

1. Middleware – Add a post‑response hook in apps/public-api to record responseTime, statusCode, endpoint, and projectId. Log asynchronously (e.g., setImmediate or queue) to avoid slowing down the API.

2. Storage – Store logs in a dedicated ApiAnalytics collection with indexes on timestamp and projectId. Enable TTL (auto‑delete after 90 days) to keep size in check.

3. Dashboard Endpoints – Create two new APIs in apps/dashboard-api:

GET /api/analytics/latency?range returns average response time per endpoint (overall + breakdown).

GET /api/analytics/errors?range returns error percentage (status ≥ 400).

Use MongoDB aggregation for time‑range filtering and grouping. Add a reasonable timeout to prevent slow queries from blocking the dashboard.

4. Testing – Unit tests for middleware, integration tests for endpoints. Verify no performance degradation with load testing.

[yash-pouranik]

Thanks for sharing the approach, @avanithumballi ! It’s technically sound and covers the key performance concerns (async logging and storage management).

I am assigning this issue to you. A few things to keep in mind for the implementation:

Model Location: Please define the new ApiAnalytics model in packages/common/src/models/ so it can be shared across apps.
Middleware: You can integrate the latency logic into the existing logger in apps/public-api/src/middlewares/api_usage.js or keep it side-by-side.
Precision: Use process.hrtime() to capture high-resolution latency.
TTL Duration: Instead of 90 days, let's keep the data for 365 days (1 year) to give project owners better long-term insights. Please make this value configurable via an environment variable (e.g., ANALYTICS_TTL_DAYS).
TTL Index: Don't forget to include the TTL index in the schema to ensure the cleanup works as planned.
Dashboard API: If possible, try to include these new metrics in the existing GET /api/projects/:projectId/analytics response to minimize frontend changes.

Waiting for the PR... Lets goooooooooooooooo.

[avanithumballi]

@yash-pouranik thanks for assigning it to me. i will keep in mind abt these guidelines