fix(lockdown): Update maxAge for Lockdown Queries (firecrawl#3411)

* fix(lockdown): match cache rows regardless of stealth proxy state

When proxy:auto retries silently upgrade to stealthProxy, the cache row
is written with is_stealth=true even though the user never asked for
stealth. The lockdown lookup queried with is_stealth=false (because
buildFeatureFlags returns an empty set for lockdown), so the strict SQL
match missed every popular bot-protected URL.

Lockdown now issues both stealth variants in parallel and uses the
union, sorted by recency. The two-call fan-out only happens when
options.lockdown is set, so normal scrapes are unaffected.

Adds a snip that seeds with proxy:'stealth' and verifies the lockdown
lookup still hits.

* fix(lockdown): cap default maxAge at 100 years to avoid TIMESTAMP overflow

Number.MAX_SAFE_INTEGER ms is ~285,000 years. Inside the index lookup
the SQL function does NOW() - INTERVAL '1ms' * p_max_age_ms, which
underflows Postgres's TIMESTAMP range (4713 BC) and silently returns no
rows. Lockdown then reports a cache miss for URLs that are very much
cached (e.g. firecrawl.dev itself).

100 years is 'any cache, any age' for practical use and stays well
inside both BIGINT and Postgres date arithmetic.

* fix(lockdown): use 1 year default maxAge instead of 100

Previous commit used 100 years; 1 year is sufficient for lockdown's
practical purpose of serving any cached version and is a friendlier
default.

* revert(lockdown): drop is_stealth fan-out, keep only maxAge fix

Isolating the maxAge change to verify it's sufficient. If prod still
misses on cached URLs after this, the is_stealth fan-out (or another
filter mismatch) is also needed.

* fix(lockdown): use 2 year default maxAge

Author: abimaelmartell

apps/api/src/__tests__/snips/v2/types-validation.test.ts

@@ -470,14 +470,14 @@ describe("V2 Types Validation", () => {
         expect(result.lockdown).toBe(true);
       });
 
-      it("should default maxAge to MAX_SAFE_INTEGER when lockdown is true and maxAge is unset", () => {
+      it("should default maxAge to ~2 years when lockdown is true and maxAge is unset", () => {
         const input: ScrapeRequestInput = {
           url: "https://example.com",
           lockdown: true,
         };
 
         const result = scrapeRequestSchema.parse(input);
-        expect(result.maxAge).toBe(Number.MAX_SAFE_INTEGER);
+        expect(result.maxAge).toBe(2 * 365 * 24 * 60 * 60 * 1000);
       });
 
       it("should preserve maxAge when lockdown is true and maxAge is provided", () => {

apps/api/src/controllers/v2/types.ts

@@ -644,7 +644,10 @@ const extractTransformImpl = <T extends ScrapeOptionsBase | undefined>(
   }
 
   if (obj.lockdown && obj.maxAge === undefined) {
-    result = { ...result, maxAge: Number.MAX_SAFE_INTEGER };
+    // 2 years in ms. Number.MAX_SAFE_INTEGER lands ~285,000 years which
+    // overflows Postgres TIMESTAMP arithmetic in the index lookup and silently
+    // returns no rows. 2 years covers any practical cache retention window.
+    result = { ...result, maxAge: 2 * 365 * 24 * 60 * 60 * 1000 };
   }
 
   return result as T extends undefined ? undefined : T;