Incorrect "Max number of right hand side join partitions limit is hit" error on v1.6.28+

[Mhs-220]

Describe the bug
Starting from version 1.6.28, queries using rollup_join pre-aggregations that combine 5 rollups fail with a partition limit error.
After checking the code, I believe this is related to the new distribute_join_partitions() logic (introduced in commit 9da75cd) caps CUBESTORE_MAX_JOINED_PARTITIONS at 5 by default, and a rollup_join referencing 5 rollups hits this limit exactly.

Here is the exact error:

Error: Internal: Execution error: Max number of right hand side join partitions limit is hit. Max limit is 5. Query requires 5. Please consider reducing right hand side join partition count and dataset size.

Now I have some questions:

1. We have exactly 5 joins and this is a big misleading, shouldn't this prevent anything with more than 5 joins?
2. Is it OK to change CUBESTORE_MAX_JOINED_PARTITIONS and set it to a higher number? I'm not sure if it's OK because it's not even documented yet.
3. It works well when we are in the DEBUG mode, why is that so?

To Reproduce

1. Define a fact cube (orders) with 4 many_to_one joins to dimension cubes
2. Create individual rollup pre-aggregations for each cube (5 total)
3. Create a rollup_join pre-aggregation that references all 5 rollups
4. Run any query that hits the combined_rollup
5. Query fails with the partition limit error

Expected behavior
The query should execute successfully. A rollup_join with 5 constituent rollups should not exceed the default partition limit, as each dimension rollup is a single partition.

Minimally reproducible Cube Schema

I tried my best to create something reproducible but not sure if I was successful or not.

Schema

cube(`orders`, {
  sql: `
    SELECT 1 AS a, 'X1' AS b, 'Y1' AS c, 'Z1' AS d, 'W1' AS e, NOW()::Date AS f
    UNION ALL
    SELECT 2, 'X1', 'Y2', 'Z2', 'W2', NOW()::Date
    UNION ALL
    SELECT 3, 'X2', 'Y1', 'Z1', 'W1', NOW()::Date
  `,

  joins: {
    t1: {
      sql: `${CUBE.b} = ${t1.b}`,
      relationship: `many_to_one`,
    },
    t2: {
      sql: `${CUBE.c} = ${t2.c}`,
      relationship: `many_to_one`,
    },
    t3: {
      sql: `${CUBE.d} = ${t3.d}`,
      relationship: `many_to_one`,
    },
    t4: {
      sql: `${CUBE.e} = ${t4.e}`,
      relationship: `many_to_one`,
    },
  },

  dimensions: {
    a: { sql: `a`, type: `number`, primary_key: true },
    b: { sql: `b`, type: `string` },
    c: { sql: `c`, type: `string` },
    d: { sql: `d`, type: `string` },
    e: { sql: `e`, type: `string` },
    f: { sql: `f`, type: `time` },
    g: { sql: `${t1.g}`, type: `string` },
    h: { sql: `${t3.h}`, type: `string` },
  },

  measures: {
    count: { type: `count` },
  },

  pre_aggregations: {
    main_rollup: {
      type: `rollup`,
      time_dimension: CUBE.f,
      granularity: `day`,
      partition_granularity: `month`,
      dimensions: [CUBE.a, CUBE.b, CUBE.c, CUBE.d, CUBE.e, CUBE.f],
      measures: [CUBE.count],
      refresh_key: { every: `1 day` },
      indexes: {
        b_index: { columns: [CUBE.b] },
        c_index: { columns: [CUBE.c] },
        d_index: { columns: [CUBE.d] },
        e_index: { columns: [CUBE.e] },
      },
    },
    combined_rollup: {
      type: `rollup_join`,
      time_dimension: CUBE.f,
      granularity: `day`,
      partition_granularity: `day`,
      dimensions: [
        CUBE.f,
        t1.g,
        t2.c,
        t3.h,
        t4.e,
      ],
      measures: [CUBE.count],
      rollups: [
        CUBE.main_rollup,
        t1.t1_rollup,
        t2.t2_rollup,
        t3.t3_rollup,
        t4.t4_rollup,
      ],
    },
  },
});

cube(`t1`, {
  sql: `
    SELECT 'X1' AS b, 'S1' AS g
    UNION ALL
    SELECT 'X2', 'S2'
  `,
  dimensions: {
    b: { sql: `b`, type: `string`, primary_key: true },
    g: { sql: `g`, type: `string` },
  },
  pre_aggregations: {
    t1_rollup: {
      type: `rollup`,
      dimensions: [CUBE.b, CUBE.g],
      refresh_key: { every: `1 day` },
      indexes: { b_index: { columns: [CUBE.b] } },
    },
  },
});

cube(`t2`, {
  sql: `
    SELECT 'Y1' AS c
    UNION ALL
    SELECT 'Y2'
  `,
  dimensions: {
    c: { sql: `c`, type: `string`, primary_key: true },
  },
  pre_aggregations: {
    t2_rollup: {
      type: `rollup`,
      dimensions: [CUBE.c],
      refresh_key: { every: `1 day` },
      indexes: { c_index: { columns: [CUBE.c] } },
    },
  },
});

cube(`t3`, {
  sql: `
    SELECT 'Z1' AS d, 'Alpha' AS h
    UNION ALL
    SELECT 'Z2', 'Beta'
  `,
  dimensions: {
    d: { sql: `d`, type: `string`, primary_key: true },
    h: { sql: `h`, type: `string` },
  },
  pre_aggregations: {
    t3_rollup: {
      type: `rollup`,
      dimensions: [CUBE.d, CUBE.h],
      refresh_key: { every: `1 day` },
      indexes: { d_index: { columns: [CUBE.d] } },
    },
  },
});

cube(`t4`, {
  sql: `
    SELECT 'W1' AS e
    UNION ALL
    SELECT 'W2'
  `,
  dimensions: {
    e: { sql: `e`, type: `string`, primary_key: true },
  },
  pre_aggregations: {
    t4_rollup: {
      type: `rollup`,
      dimensions: [CUBE.e],
      refresh_key: { every: `1 day` },
      indexes: { e_index: { columns: [CUBE.e] } },
    },
  },
});

Query to reproduce

query CubeQuery {
  cube(
    where: {orders: {g: {equals: "S1"}, f: {inDateRange: "this week"}}}
  ) {
    orders {
      f { value }
      h
    }
  }
}

Version:
1.6.28+

Additional context
N/A

[Mhs-220]

The issue got solved after setting CUBESTORE_MAX_JOINED_PARTITIONS to 6 although the behavior is peculiar, but I'll close the ticket.