feat: add cron job to cleanup old queued form responses (calcom#22035)

* feat: add cron job to cleanup old queued form responses

- Add cleanup cron job for App_RoutingForms_QueuedFormResponse table
- Delete records with null actualResponseId older than 1 hour
- Schedule to run twice daily (every 12 hours) via vercel.json
- Follow existing cron job patterns for authentication and structure

Co-Authored-By: hariom@cal.com <hariom@cal.com>

* fix: correct date comparison logic to delete older records

- Change from lte to gte in createdAt comparison
- Now properly deletes records older than 1 hour

Co-Authored-By: hariom@cal.com <hariom@cal.com>

* refactor: move cleanup logic to routing forms lib and import in API endpoint

- Create cleanupExpiredQueuedFormResponses function in routing forms lib
- Update API endpoint to import and use the new cleanup function
- Follow established patterns for code organization in Cal.com
- Maintain same functionality and authentication logic

Co-Authored-By: hariom@cal.com <hariom@cal.com>

* refactor: move entire cron logic to routing-forms cron folder

- Create routing-forms/cron/queuedFormResponseCleanup.ts with complete handler
- Move authentication, HTTP handling, and database operations to routing-forms
- Update API endpoint to simply import and call the cron handler
- Remove old lib cleanup function
- Follow user's preferred organizational pattern

Co-Authored-By: hariom@cal.com <hariom@cal.com>

* fix: correct date comparison logic to delete records older than 1 hour

- Change from gte (greater than or equal) to lt (less than) cutoffTime
- Now properly deletes records where createdAt < cutoffTime (older than 1 hour)
- Previous logic was deleting newer records instead of older ones

Co-Authored-By: hariom@cal.com <hariom@cal.com>

* fixes

* refactor: reorganize queuedFormResponse into domain-driven structure

- Move repository and service files to routingForm/queuedFormResponse subdirectory
- Clean up unused methods and tests for better maintainability
- Fix import paths and type definitions
- Ensure all tests pass and maintain backward compatibility

* fix: align QueuedFormResponse interface types with Prisma schema

- Update actualResponseId type from string to number to match Prisma Int type
- Make updatedAt nullable (Date | null) to match Prisma DateTime? type
- Fix import path for QueuedFormResponseRepository interface

Co-Authored-By: hariom@cal.com <hariombalhara@gmail.com>

* refactor: reorganize queuedFormResponse into domain-driven structure

- Move repository and service files to routingForm/queuedFormResponse subdirectory
- Clean up unused methods and tests for better maintainability
- Fix import paths and type definitions
- Ensure all tests pass and maintain backward compatibility

* test: add comprehensive where clause validation tests and missing test coverage

* chore: update cron schedule to weekly for 7-day retention period

* revert: restore 12-hour cron schedule for better batch management

* Add integration tests

* refactor: apply test data builder pattern to queuedFormResponse integration tests

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: hariom@cal.com <hariom@cal.com>
Co-authored-by: Hariom <hariombalhara@gmail.com>

Author: 3 people

apps/web/app/api/cron/queuedFormResponseCleanup/route.ts

@@ -0,0 +1,5 @@
+import { defaultResponderForAppDir } from "app/api/defaultResponderForAppDir";
+
+import { handleQueuedFormResponseCleanup } from "@calcom/app-store/routing-forms/cron/queuedFormResponseCleanup";
+
+export const POST = defaultResponderForAppDir(handleQueuedFormResponseCleanup);

apps/web/vercel.json

@@ -4,6 +4,10 @@
       "path": "/api/cron/calendar-cache-cleanup",
       "schedule": "0 5 * * *"
     },
+    {
+      "path": "/api/cron/queuedFormResponseCleanup",
+      "schedule": "0 */12 * * *"
+    },
     {
       "path": "/api/tasks/cron",
       "schedule": "* * * * *"

packages/app-store/routing-forms/cron/queuedFormResponseCleanup.ts

@@ -0,0 +1,47 @@
+import type { NextRequest } from "next/server";
+import { NextResponse } from "next/server";
+
+import logger from "@calcom/lib/logger";
+import { PrismaQueuedFormResponseRepository } from "@calcom/lib/server/repository/routingForm/queuedFormResponse/PrismaQueuedFormResponseRepository";
+import { QueuedFormResponseService } from "@calcom/lib/server/service/routingForm/queuedFormResponse/QueuedFormResponseService";
+import prisma from "@calcom/prisma";
+
+function validateRequest(request: NextRequest) {
+  const apiKey = request.headers.get("authorization") || request.nextUrl.searchParams.get("apiKey");
+
+  if (![process.env.CRON_API_KEY, `Bearer ${process.env.CRON_SECRET}`].includes(`${apiKey}`)) {
+    return NextResponse.json({ message: "Not authenticated" }, { status: 401 });
+  }
+
+  return null;
+}
+
+export async function handleQueuedFormResponseCleanup(request: NextRequest) {
+  const authError = validateRequest(request);
+  if (authError) {
+    return authError;
+  }
+
+  try {
+    const queuedFormResponseRepo = new PrismaQueuedFormResponseRepository(prisma);
+    const queuedFormResponseService = new QueuedFormResponseService({
+      logger,
+      queuedFormResponseRepo,
+    });
+
+    const result = await queuedFormResponseService.cleanupExpiredResponses({
+      batchSize: 1000,
+    });
+
+    return NextResponse.json({
+      status: "success",
+      data: result,
+    });
+  } catch (error) {
+    console.error("Error during queued form response cleanup:", error);
+    return NextResponse.json(
+      { status: "error", message: "Failed to cleanup queued form responses" },
+      { status: 500 }
+    );
+  }
+}