Optimize q filter to use pk__in subqueries instead of set operations

The `q` complex filter composed its NOT/AND/OR operands with QuerySet
.difference()/.intersection()/.union(), which compile to SQL
EXCEPT/INTERSECT/UNION. For NOT in particular, the left operand is the
full unfiltered queryset, so evaluating an expression like
`repository_version=X AND NOT repository_version=Y` seq-scans the entire
content table and sorts/hashes every leg, spilling to disk on large
repositories.

Rewrite the three operand evaluators to compose with `pk__in` subqueries:

  - _NotAction:  qs.exclude(pk__in=expr.evaluate(qs).values("pk"))
  - _AndAction:  chained qs.filter(pk__in=...) semi-joins
  - _OrAction:   OR of Q(pk__in=...) subqueries

This collapses the plan to indexed anti-/semi-joins with no full-table
scan, no SetOp, and no disk-spilling sorts. Results are unchanged (each
operand still filters the same base queryset on a unique pk, so the
implicit de-duplication of the set operations is preserved); the
existing q filter functional tests pass without modification.

Benchmarked on a 200k-unit repository computing a 1000-unit diff
between two repository versions (steady-state, work_mem=4MB):

  Version sizes    Before (set-ops)    After (pk__in)
  ---------------  ------------------  ----------------
  199k units       ~1400 ms            ~680 ms
   50k units       ~450 ms             ~150 ms

Planning time also drops (e.g. ~1.4 ms -> ~0.5 ms) as the large inlined
operand lists are replaced by parameterized subqueries.

Closes #7831

Assisted by: Claude Opus 4.8

Author: Andrew Cheng

CHANGES/7831.bugfix

@@ -0,0 +1,3 @@
+Optimized the ``q`` complex filter to compose ``NOT``/``AND``/``OR`` operands using ``pk__in``
+subqueries instead of the ``EXCEPT``/``INTERSECT``/``UNION`` set operations, avoiding full-table
+scans and disk-spilling sorts on large content sets.

pulpcore/filters.py

@@ -215,27 +215,29 @@ def __init__(self, tokens):
             self.complexity = self.expr.complexity + 1
 
         def evaluate(self, qs):
-            return qs.difference(self.expr.evaluate(qs))
+            return qs.exclude(pk__in=self.expr.evaluate(qs).values("pk"))
 
     class _AndAction:
         def __init__(self, tokens):
             self.exprs = tokens[0]
             self.complexity = sum((expr.complexity for expr in self.exprs)) + 1
 
         def evaluate(self, qs):
-            return (
-                self.exprs[0]
-                .evaluate(qs)
-                .intersection(*[expr.evaluate(qs) for expr in self.exprs[1:]])
-            )
+            result = qs
+            for expr in self.exprs:
+                result = result.filter(pk__in=expr.evaluate(qs).values("pk"))
+            return result
 
     class _OrAction:
         def __init__(self, tokens):
             self.exprs = tokens[0]
             self.complexity = sum((expr.complexity for expr in self.exprs)) + 1
 
         def evaluate(self, qs):
-            return self.exprs[0].evaluate(qs).union(*[expr.evaluate(qs) for expr in self.exprs[1:]])
+            query = models.Q()
+            for expr in self.exprs:
+                query |= models.Q(pk__in=expr.evaluate(qs).values("pk"))
+            return qs.filter(query)
 
     def __init__(self, *args, **kwargs):
         self.filterset = kwargs.pop("filter").parent