refactor(polling): consolidate polling services into provider handler pattern (#4035)

* refactor(polling): consolidate polling services into provider handler pattern

Eliminate self-POST anti-pattern and extract shared boilerplate from 4 polling
services into a clean handler registry mirroring lib/webhooks/providers/.

- Add processPolledWebhookEvent() to processor.ts for direct in-process webhook
  execution, removing HTTP round-trips that caused Lambda 403/timeout errors
- Extract shared utilities (markWebhookFailed/Success, fetchActiveWebhooks,
  runWithConcurrency, resolveOAuthCredential, updateWebhookProviderConfig)
- Create PollingProviderHandler interface with per-provider implementations
- Consolidate 4 identical route files into single dynamic [provider] route
- Standardize concurrency to 10 across all providers
- No infra changes needed — Helm cron paths resolve via dynamic route

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* polish(polling): extract lock TTL constant and remove unnecessary type casts

- Widen processPolledWebhookEvent body param to accept object, eliminating
  `as unknown as Record<string, unknown>` double casts in all 4 handlers
- Extract LOCK_TTL_SECONDS constant in route, tying maxDuration and lock TTL
  to a single value

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(polling): address PR review feedback

- Add archivedAt filters to fetchActiveWebhooks query, matching
  findWebhookAndWorkflow in processor.ts to prevent polling archived
  webhooks/workflows
- Move provider validation after auth check to prevent provider
  enumeration by unauthenticated callers
- Fix inconsistent pollingIdempotency import path in outlook.ts to
  match other handlers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(polling): use literal for maxDuration segment config

Next.js requires segment config exports to be statically analyzable
literals. Using a variable reference caused build failure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: waleedlatif1

…s/sim/app/api/webhooks/poll/rss/route.ts …pp/api/webhooks/poll/[provider]/route.tsapps/sim/app/api/webhooks/poll/rss/route.ts renamed to apps/sim/app/api/webhooks/poll/[provider]/route.ts apps/sim/app/api/webhooks/poll/rss/route.ts renamed to apps/sim/app/api/webhooks/poll/[provider]/route.ts

@@ -3,31 +3,36 @@ import { type NextRequest, NextResponse } from 'next/server'
 import { verifyCronAuth } from '@/lib/auth/internal'
 import { acquireLock, releaseLock } from '@/lib/core/config/redis'
 import { generateShortId } from '@/lib/core/utils/uuid'
-import { pollRssWebhooks } from '@/lib/webhooks/rss-polling-service'
+import { pollProvider, VALID_POLLING_PROVIDERS } from '@/lib/webhooks/polling'
 
-const logger = createLogger('RssPollingAPI')
+const logger = createLogger('PollingAPI')
 
-export const dynamic = 'force-dynamic'
-export const maxDuration = 180 // Allow up to 3 minutes for polling to complete
+/** Lock TTL in seconds — must match maxDuration so the lock auto-expires if the function times out. */
+const LOCK_TTL_SECONDS = 180
 
-const LOCK_KEY = 'rss-polling-lock'
-const LOCK_TTL_SECONDS = 180 // Same as maxDuration (3 min)
+export const dynamic = 'force-dynamic'
+export const maxDuration = 180
 
-export async function GET(request: NextRequest) {
+export async function GET(
+  request: NextRequest,
+  { params }: { params: Promise<{ provider: string }> }
+) {
+  const { provider } = await params
   const requestId = generateShortId()
-  logger.info(`RSS webhook polling triggered (${requestId})`)
 
+  const LOCK_KEY = `${provider}-polling-lock`
   let lockValue: string | undefined
 
   try {
-    const authError = verifyCronAuth(request, 'RSS webhook polling')
-    if (authError) {
-      return authError
+    const authError = verifyCronAuth(request, `${provider} webhook polling`)
+    if (authError) return authError
+
+    if (!VALID_POLLING_PROVIDERS.has(provider)) {
+      return NextResponse.json({ error: `Unknown polling provider: ${provider}` }, { status: 404 })
     }
 
     lockValue = requestId
     const locked = await acquireLock(LOCK_KEY, lockValue, LOCK_TTL_SECONDS)
-
     if (!locked) {
       return NextResponse.json(
         {
@@ -40,21 +45,21 @@ export async function GET(request: NextRequest) {
       )
     }
 
-    const results = await pollRssWebhooks()
+    const results = await pollProvider(provider)
 
     return NextResponse.json({
       success: true,
-      message: 'RSS polling completed',
+      message: `${provider} polling completed`,
       requestId,
       status: 'completed',
       ...results,
     })
   } catch (error) {
-    logger.error(`Error during RSS polling (${requestId}):`, error)
+    logger.error(`Error during ${provider} polling (${requestId}):`, error)
     return NextResponse.json(
       {
         success: false,
-        message: 'RSS polling failed',
+        message: `${provider} polling failed`,
         error: error instanceof Error ? error.message : 'Unknown error',
         requestId,
       },