feat: conditional data masking with row-level filters in access policies (cube-js#10803)

* feat: conditional data masking with row-level filters in access policies

When an access policy grants full member_level access with row_level
filters, and another policy provides masked access, the masking should
be conditional on the row filter. Previously, masking was skipped
entirely when any policy granted full access, even if that access was
restricted by row-level filters.

Now, masked members with conditional full access generate:
  CASE WHEN {rowFilter} THEN {originalValue} ELSE {maskedValue} END

This ensures that rows matching the row filter see unmasked values,
while rows outside the filter range see masked values.

Changes:
- CompilerApi: Distinguish unconditional vs conditional full access
  when determining masking. Track row filters for conditionally
  accessible members as memberMaskFilters.
- BaseQuery: Add conditionalMemberMaskSql() and maskFilterToSql()
  methods to generate CASE WHEN SQL for masked members with
  associated row filters.
- NormalizedQuery type: Add memberMaskFilters field.
- Tests: Add 4 new tests for conditional masking behavior.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* refactor: extend maskedMembers type to carry filter info inline

Instead of a separate memberMaskFilters field, extend maskedMembers to
support both string and {member, filter} object formats:
  maskedMembers: (string | { member: string, filter: FilterItem })[]

This ensures Tesseract/native SQL planner also supports conditional
masking. The Rust MaskedSqlNode now generates CASE WHEN SQL when a
member has an associated mask filter.

Changes:
- CompilerApi: Emit maskedMembers with inline filter objects
- BaseQuery.js: Parse both formats from maskedMembers array
- Rust (base_query_options.rs): Add MaskedMemberItem enum (untagged)
- Rust (query_tools.rs): Store mask filters alongside masked members
- Rust (masked.rs): Generate CASE WHEN for conditional masks
- Rust test fixtures: Update to use MaskedMemberItem type
- NormalizedQuery type: Update maskedMembers type signature
- query.js: Update Joi validation for new format
- smoke-rbac.test.ts: Add conditional masking test case
- conditional_masking_test.yaml: Add fixture for smoke test

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* fix: resolve lint, fmt, and stack overflow issues

- Fix infinite recursion in conditionalMemberMaskSql by adding
  skipMaskFor context guard to prevent re-entrance
- Fix CompilerApi logic: only apply conditional masking when a
  memberMasking policy actually exists (prevents false positive
  masking for cubes that only have row_level filters without
  memberMasking)
- Fix lint: object-property-newline and object-curly-spacing in tests
- Run cargo fmt on all Rust files

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* refactor: use standard filter pipeline for mask filter SQL generation

BaseQuery.js: Replace custom and/or rendering in maskFilterToSql with
the standard extractFiltersAsTree + initFilter + filterToWhere() pipeline
that all other filters use.

Rust MaskedSqlNode: Replace custom render_native_filter +
render_filter_condition with FilterCompiler::add_item +
FilterItem::to_sql using a standard VisitorContext - the same approach
used by QueryProperties and Select for all other filter rendering.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* fix: resolve lint no-continue errors and Rust RefCell borrow panic

- CompilerApi.ts: Replace continue statements with nested if/else
  to satisfy the no-continue lint rule
- Rust MaskedSqlNode: Fix RefCell already borrowed panic by using
  VisitorContext::new_with_node_processor with self.input (the
  underlying EvaluateSqlNode) instead of SqlNodesFactory::new()
  which creates a node processor chain that includes MaskedSqlNode,
  causing re-entrant borrow when evaluating filter member SQL
- Add VisitorContext::new_with_node_processor constructor for
  creating contexts with custom node processors

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* fix: prevent infinite recursion when filter member is also masked

The stack overflow occurred when a mask filter referenced a member that
was itself masked (e.g., product_id has a mask filter on product_id).
The evaluateSymbolSql for the filter member would re-enter the masking
code, creating infinite recursion.

Fix: Use skipMasking context flag to disable all masking during CASE
WHEN evaluation (both the filter SQL and the original value SQL).
Also set currentMember to null to prevent memberChildren cycle tracking
that caused hasMultiStageMembers infinite loop.

Added test case that verifies no recursion when filter member is masked.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* refactor: use { member, filter? } struct for maskedMembers, remove validation

- maskedMembers now always uses { member: string, filter?: FilterItem }
  objects instead of string | object alternatives
- Remove maskedMembers from Joi query validation schema since users
  should never provide masking params in the query directly
- Rust MaskedMemberItem: replace untagged enum with simple struct
- Update all consumers and tests to use object format

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* fix: keep maskedMembers in Joi schema for internal query validation

The normalizeQuery validation runs after applyRowLevelSecurity sets
maskedMembers on the query, so the field must be allowed in the schema.
Use Joi.array().items(Joi.object()) to accept the internal format
without exposing strict typing to users.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* fix: prevent maskedMembers injection from user queries and rewrites

- Strip maskedMembers from user query before first normalizeQuery
  validation so users cannot inject masking params
- After queryRewrite, always restore maskedMembers from the post-RLS
  query, ensuring rewrites cannot override RLS-determined masking

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* fix: throw error if maskedMembers is provided in user query

Instead of silently stripping maskedMembers, throw a UserError if the
user attempts to pass it in their query. The Joi schema keeps the
correct type definition for internal validation after RLS sets it.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* refactor: reuse standard CASE WHEN templates for conditional masking

BaseQuery.js: Use caseWhenStatement() method (same template used by
measure filters and dimension case types) instead of inline template
string.

Rust MaskedSqlNode: Use templates.case() from PlanSqlTemplates (same
template used by CaseSqlNode for dimension case rendering) instead of
inline format! string.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* refactor: use BaseGroupFilter for multiple mask filter clauses

Instead of manually joining clauses with ' AND ', create a proper
and-group filter via newGroupFilter when there are multiple filter
items, reusing the standard BaseGroupFilter.filterToWhere() rendering.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* docs: add TODO for FILTER_PARAMS support in mask filter SQL

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* fix: OR across policies, AND within policy for mask filters

When multiple policies grant conditional full access, the member should
be unmasked when ANY policy's filter matches (OR), not when ALL match
(AND). Within a single policy, multiple filters are still AND'd.

Example with two policies:
  Policy A: filters = [region = 'RESEARCH', lock = 0]
  Policy B: filters = [region = 'DEMO']

Before: region = 'RESEARCH' AND lock = 0 AND region = 'DEMO' (always false)
After:  (region = 'RESEARCH' AND lock = 0) OR (region = 'DEMO')

This is consistent with row-level security which uses union (OR) across
policies for row visibility.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* test: add smoke test for OR across multiple conditional mask policies

Adds a test that verifies when a user has two conditional mask roles:
  - conditional_mask_role: product_id <= 3
  - conditional_mask_role_extra: product_id = 5

The price is unmasked when EITHER filter matches (OR across policies),
confirming the fix for the AND vs OR bug.

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

* refactor: remove unused memberPathArray parameter from conditionalMemberMaskSql

Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Author: paveltiunov

packages/cubejs-api-gateway/src/gateway.ts

@@ -1343,6 +1343,10 @@ class ApiGateway {
         currentQuery = this.parseMemberExpressionsInQuery(currentQuery);
       }
 
+      if ((currentQuery as any).maskedMembers) {
+        throw new UserError('maskedMembers cannot be provided in the query');
+      }
+
       return {
         normalizedQuery: (normalizeQuery(currentQuery, persistent, cacheMode)),
         hasExpressionsInQuery
@@ -1372,6 +1376,8 @@ class ApiGateway {
             context
           ) : queryWithRlsFilters;
 
+          rewrittenQuery.maskedMembers = queryWithRlsFilters.maskedMembers;
+
           // applyRowLevelSecurity may add new filters which may contain raw member expressions
           // if that's the case, we should run an extra pass of parsing here to make sure
           // nothing breaks down the road

packages/cubejs-api-gateway/src/query.js

@@ -195,7 +195,10 @@ const querySchema = Joi.object().keys({
   responseFormat: Joi.valid('default', 'compact', 'columnar'),
   subqueryJoins: Joi.array().items(subqueryJoin),
   joinHints: Joi.array().items(joinHint),
-  maskedMembers: Joi.array().items(Joi.string()),
+  maskedMembers: Joi.array().items(Joi.object().keys({
+    member: Joi.string().required(),
+    filter: Joi.object(),
+  })),
 });
 
 const normalizeQueryOrder = order => {

packages/cubejs-api-gateway/src/types/query.ts

@@ -166,7 +166,7 @@ interface NormalizedQuery extends Query {
   filters?: NormalizedQueryFilter[];
   rowLimit?: null | number;
   order?: { id: string; desc: boolean }[];
-  maskedMembers?: string[];
+  maskedMembers?: { member: string; filter?: any }[];
 }
 
 export {

packages/cubejs-schema-compiler/src/adapter/BaseQuery.js

@@ -253,7 +253,14 @@ export class BaseQuery {
       securityContext: {},
       ...this.options.contextSymbols,
     };
-    this.maskedMembers = new Set(this.options.maskedMembers || []);
+    this.maskedMembers = new Set();
+    this.memberMaskFilters = {};
+    for (const item of this.options.maskedMembers || []) {
+      this.maskedMembers.add(item.member);
+      if (item.filter) {
+        this.memberMaskFilters[item.member] = item.filter;
+      }
+    }
     this.compilerCache = this.compilers.compiler.compilerCache;
     this.queryCache = this.compilerCache.getQueryCache({
       measures: this.options.measures,
@@ -3299,13 +3306,18 @@ export class BaseQuery {
 
     this.safeEvaluateSymbolContext().currentMember = memberPath;
     try {
-      if (this.maskedMembers && this.maskedMembers.has(memberPath) && !memberExpressionType) {
+      if (this.maskedMembers && this.maskedMembers.has(memberPath) && !memberExpressionType &&
+          !this.safeEvaluateSymbolContext().skipMasking) {
         // In ungrouped queries, only apply static masks to measures.
         // SQL masks (mask.sql) reference columns that don't apply per-row.
         const isMeasure = type === 'measure';
         const isUngrouped = this.options.ungrouped;
         const hasSqlMask = symbol.mask && typeof symbol.mask === 'object' && symbol.mask.sql;
         if (!isMeasure || !isUngrouped || !hasSqlMask) {
+          const maskFilter = this.memberMaskFilters && this.memberMaskFilters[memberPath];
+          if (maskFilter) {
+            return this.conditionalMemberMaskSql(cubeName, name, symbol, maskFilter);
+          }
           return this.memberMaskSql(cubeName, name, symbol);
         }
       }
@@ -3486,6 +3498,37 @@ export class BaseQuery {
     return this.defaultMaskSql(symbol.type);
   }
 
+  conditionalMemberMaskSql(cubeName, name, symbol, maskFilter) {
+    const maskedSql = this.memberMaskSql(cubeName, name, symbol);
+    const result = this.evaluateSymbolSqlWithContext(
+      () => {
+        const filterSql = this.maskFilterToSql(maskFilter);
+        if (!filterSql) {
+          return maskedSql;
+        }
+        const originalSql = this.autoPrefixAndEvaluateSql(cubeName, symbol.sql);
+        return this.caseWhenStatement([{ sql: filterSql, label: originalSql }], maskedSql);
+      },
+      { skipMasking: true, currentMember: null }
+    );
+    return result;
+  }
+
+  maskFilterToSql(filter) {
+    if (!filter) return null;
+    const filterItems = this.extractFiltersAsTree([filter]);
+    if (!filterItems.length) return null;
+    const initialized = filterItems.map(this.initFilter.bind(this));
+    if (initialized.length === 1) {
+      return initialized[0].filterToWhere();
+    }
+    const groupFilter = this.newGroupFilter({
+      operator: 'and',
+      values: initialized,
+    });
+    return groupFilter.filterToWhere();
+  }
+
   defaultMaskSql(memberType) {
     const envMasks = {
       string: getEnv('accessPolicyMaskString'),

packages/cubejs-schema-compiler/test/unit/transpilers.test.ts

@@ -410,7 +410,7 @@ describe('Transpilers', () => {
       {
         measures: ['Test.count'],
         dimensions: ['Test.secret'],
-        maskedMembers: ['Test.secret'],
+        maskedMembers: [{ member: 'Test.secret' }],
       }
     );
     const sql = query.buildSqlAndParams();

packages/cubejs-schema-compiler/test/unit/yaml-schema.test.ts

@@ -1665,7 +1665,7 @@ cubes:
         {
           measures: ['orders.count'],
           dimensions: ['orders.status'],
-          maskedMembers: ['orders.status'],
+          maskedMembers: [{ member: 'orders.status' }],
           contextSymbols: {
             securityContext: { cubeCloud: { userAttributes: { hasStatusAccess: true } } }
           }
@@ -1806,7 +1806,7 @@ views:
         const query = new PostgresQuery(compilers, {
           measures: ['users_secure_view.users_count'],
           dimensions: ['users_secure_view.users_city_sensitive_masked'],
-          maskedMembers: ['users_secure_view.users_city_sensitive_masked'],
+          maskedMembers: [{ member: 'users_secure_view.users_city_sensitive_masked' }],
           contextSymbols: {
             securityContext: { cubeCloud: { groups } }
           },
@@ -1845,4 +1845,214 @@ views:
       });
     });
   });
+
+  describe('Conditional masking with row-level filters (memberMaskFilters)', () => {
+    it('generates CASE WHEN with row filter for masked members that have conditional full access', async () => {
+      const compilers = prepareYamlCompiler(`
+cubes:
+  - name: users
+    sql_table: public.users
+    dimensions:
+      - name: id
+        sql: id
+        type: number
+        primary_key: true
+      - name: city
+        sql: city
+        type: string
+      - name: data_region
+        sql: data_region
+        type: string
+    measures:
+      - name: count
+        type: count
+      `);
+
+      await compilers.compiler.compile();
+
+      const query = new PostgresQuery(compilers, {
+        measures: ['users.count'],
+        dimensions: ['users.city'],
+        maskedMembers: [{
+          member: 'users.city',
+          filter: {
+            member: 'users.data_region',
+            operator: 'equals',
+            values: ['RESEARCH', 'DEMO'],
+          }
+        }],
+      });
+      const [sql] = query.buildSqlAndParams();
+      expect(sql).toMatch(/CASE\s+WHEN/);
+      expect(sql).toMatch(/WHEN.*data_region.*THEN.*city.*ELSE.*NULL.*END/s);
+    });
+
+    it('generates CASE WHEN with AND row filter for multiple filter conditions', async () => {
+      const compilers = prepareYamlCompiler(`
+cubes:
+  - name: users
+    sql_table: public.users
+    dimensions:
+      - name: id
+        sql: id
+        type: number
+        primary_key: true
+      - name: city
+        sql: city
+        type: string
+      - name: data_region
+        sql: data_region
+        type: string
+      - name: region_lock
+        sql: region_lock
+        type: number
+    measures:
+      - name: count
+        type: count
+      `);
+
+      await compilers.compiler.compile();
+
+      const query = new PostgresQuery(compilers, {
+        measures: ['users.count'],
+        dimensions: ['users.city'],
+        maskedMembers: [{
+          member: 'users.city',
+          filter: {
+            and: [
+              {
+                member: 'users.data_region',
+                operator: 'equals',
+                values: ['RESEARCH'],
+              },
+              {
+                member: 'users.region_lock',
+                operator: 'equals',
+                values: ['0'],
+              }
+            ]
+          }
+        }],
+      });
+      const [sql] = query.buildSqlAndParams();
+      expect(sql).toMatch(/CASE\s+WHEN/);
+      expect(sql).toMatch(/WHEN.*AND.*THEN.*city.*ELSE.*NULL.*END/s);
+    });
+
+    it('uses mask.sql as the ELSE branch when dimension has a custom mask', async () => {
+      const compilers = prepareYamlCompiler(`
+cubes:
+  - name: users
+    sql_table: public.users
+    dimensions:
+      - name: id
+        sql: id
+        type: number
+        primary_key: true
+      - name: city
+        sql: city
+        type: string
+        mask:
+          sql: "'***MASKED***'"
+      - name: data_region
+        sql: data_region
+        type: string
+    measures:
+      - name: count
+        type: count
+      `);
+
+      await compilers.compiler.compile();
+
+      const query = new PostgresQuery(compilers, {
+        measures: ['users.count'],
+        dimensions: ['users.city'],
+        maskedMembers: [{
+          member: 'users.city',
+          filter: {
+            member: 'users.data_region',
+            operator: 'equals',
+            values: ['RESEARCH'],
+          }
+        }],
+      });
+      const [sql] = query.buildSqlAndParams();
+      expect(sql).toMatch(/CASE\s+WHEN/);
+      expect(sql).toMatch(/WHEN.*data_region.*THEN.*city.*ELSE.*MASKED.*END/s);
+    });
+
+    it('applies regular masking (no CASE WHEN) when no memberMaskFilters', async () => {
+      const compilers = prepareYamlCompiler(`
+cubes:
+  - name: users
+    sql_table: public.users
+    dimensions:
+      - name: id
+        sql: id
+        type: number
+        primary_key: true
+      - name: city
+        sql: city
+        type: string
+    measures:
+      - name: count
+        type: count
+      `);
+
+      await compilers.compiler.compile();
+
+      const query = new PostgresQuery(compilers, {
+        measures: ['users.count'],
+        dimensions: ['users.city'],
+        maskedMembers: [{ member: 'users.city' }],
+      });
+      const [sql] = query.buildSqlAndParams();
+      expect(sql).not.toMatch(/CASE\s+WHEN/);
+      expect(sql).toContain('NULL');
+    });
+
+    it('does not recurse when filter member is also masked', async () => {
+      const compilers = prepareYamlCompiler(`
+cubes:
+  - name: items
+    sql_table: public.items
+    dimensions:
+      - name: id
+        sql: id
+        type: number
+        primary_key: true
+      - name: product_id
+        sql: product_id
+        type: number
+      - name: price
+        sql: price
+        type: number
+        mask: -1
+    measures:
+      - name: count
+        type: count
+      `);
+
+      await compilers.compiler.compile();
+
+      const query = new PostgresQuery(compilers, {
+        measures: ['items.count'],
+        dimensions: ['items.product_id', 'items.price'],
+        maskedMembers: [
+          {
+            member: 'items.product_id',
+            filter: { member: 'items.product_id', operator: 'lte', values: ['3'] }
+          },
+          {
+            member: 'items.price',
+            filter: { member: 'items.product_id', operator: 'lte', values: ['3'] }
+          },
+        ],
+      });
+      const [sql] = query.buildSqlAndParams();
+      expect(sql).toMatch(/CASE\s+WHEN/);
+      expect(sql).toMatch(/product_id/);
+      expect(sql).not.toMatch(/Maximum call stack/);
+    });
+  });
 });