fix(data-fabric): route expansionLevel to URL query string on queryRecordsById (#531)

* fix(data-fabric): route expansionLevel to URL query string on queryRecordsById

DF record POST endpoints honour `expansionLevel` only as a URL query
param, not from the request body. The rest of the query options (filters,
sort, aggregates, etc.) are still sent in the POST body.

Adds a generic `urlParams` hook to PaginationHelpers.getAll that keeps
declared keys in the URL query string regardless of HTTP method — body on
POST, merged into params on GET — for both the paginated and non-paginated
flows. `queryRecordsById` wires `expansionLevel` through it so non-zero
levels actually expand reference fields again.

Also stops base.ts from folding caller `options.params` into the POST body
and nulling out the URL params, so URL query params survive on POST.

Adds two integration tests covering all expansion levels (0-3): one for
the system reference field `CreatedBy`, and one that builds an isolated
source→target schema with a custom RELATIONSHIP field to verify the
user-defined column inflates per level.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(data-fabric): skip RELATIONSHIP expansion test under PAT auth

entities.create() needs the entity.schema.write scope, which PAT tokens
cannot hold (returns insufficient_scope). The CreatedBy expansionLevel
test already validates the fix under PAT; keep this body for OAuth runs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(pagination): rename urlParams to queryParams

queryParams is the recognized HTTP nomenclature and the existing
convention in this codebase for query-string params passed via params:
(see process-instances.ts, buckets.ts). urlParams elsewhere refers to
URLSearchParams over window.location — a different concept.

Also re-skip the RELATIONSHIP expansion integration test (needs
entity.schema.write, unavailable to PAT) and refine the queryRecordsById
comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(data-fabric): use describe.skip for RELATIONSHIP expansion test

Match the file's convention for schema-write tests (describe.skip), which
need the entity.schema.write scope PAT tokens can't hold. it.skip on a
lone test was inconsistent with the surrounding describe.skip blocks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: Sarath1018

src/services/base.ts

@@ -172,17 +172,16 @@ export class BaseService {
     // Prepare request parameters based on pagination type
     const requestParams = this.preparePaginationRequestParams(paginationType, params, options.pagination);
 
-    // For POST requests, merge pagination params into body and set params to undefined; for GET, use query params
+    // Route pagination state to wherever the API expects it (body for POST, URL for GET).
+    // Caller-supplied options.body / options.params are respected as-is — the api-client
+    // already handles params (URL) and body (request body) independently for every method.
     if (method.toUpperCase() === 'POST') {
       const existingBody = (options.body && typeof options.body === 'object') ? options.body as Record<string, unknown> : {};
       options.body = {
         ...existingBody,
-        ...options.params,
         ...requestParams
       };
-      options.params = undefined;
     } else {
-      // Merge pagination parameters with existing parameters
       options.params = {
         ...options.params,
         ...requestParams

src/services/data-fabric/entities.ts

@@ -625,15 +625,15 @@ export class EntityService extends BaseService implements EntityServiceModel {
     id: string,
     options?: T
   ): Promise<T extends HasPaginationOptions<T> ? PaginatedResponse<EntityRecord> : NonPaginatedResponse<EntityRecord>> {
-    // folderKey is header-only — destructure it out so PaginationHelpers doesn't include
-    // it in the POST body alongside the query filters.
-    const { folderKey, ...rest } = options ?? {};
+    // folderKey is header-only; expansionLevel must be sent as a query param by PaginationHelpers.
+    const { folderKey, expansionLevel, ...rest } = options ?? {};
     const downstreamOptions = options === undefined ? undefined : (rest as T);
     return PaginationHelpers.getAll({
       serviceAccess: this.createPaginationServiceAccess(),
       getEndpoint: () => DATA_FABRIC_ENDPOINTS.ENTITY.QUERY_BY_ID(id),
       method: HTTP_METHODS.POST,
       headers: createHeaders({ [FOLDER_KEY]: folderKey }),
+      queryParams: createParams({ expansionLevel }),
       pagination: {
         paginationType: PaginationType.OFFSET,
         itemsField: ENTITY_PAGINATION.ITEMS_FIELD,
@@ -644,7 +644,7 @@ export class EntityService extends BaseService implements EntityServiceModel {
           countParam: ENTITY_OFFSET_PARAMS.COUNT_PARAM
         }
       },
-      excludeFromPrefix: ['expansionLevel', 'filterGroup', 'selectedFields', 'sortOptions', 'aggregates', 'groupBy']
+      excludeFromPrefix: ['filterGroup', 'selectedFields', 'sortOptions', 'aggregates', 'groupBy']
     }, downstreamOptions);
   }
 

src/utils/pagination/helpers.ts

@@ -178,6 +178,7 @@ export class PaginationHelpers {
       headers: providedHeaders,
       paginationParams,
       additionalParams,
+      queryParams,
       transformFn,
       method = HTTP_METHODS.GET,
       options = {}
@@ -186,13 +187,20 @@ export class PaginationHelpers {
     const endpoint = getEndpoint(folderId);
     const headers = providedHeaders ?? (folderId ? createHeaders({ [FOLDER_ID]: folderId }) : {});
 
+    // On POST, the caller's options go in the body; queryParams stays in the URL.
+    // On GET, everything is URL — queryParams merges with additionalParams.
+    const isPost = method === HTTP_METHODS.POST;
+    const requestSpec = isPost
+      ? { body: additionalParams, params: queryParams }
+      : { params: { ...additionalParams, ...queryParams } };
+
     const paginatedResponse = await serviceAccess.requestWithPagination<T>(
       method,
       endpoint,
       paginationParams,
       {
         headers,
-        params: additionalParams,
+        ...requestSpec,
         pagination: {
           paginationType: options.paginationType || PaginationType.OFFSET,
           itemsField: options.itemsField || DEFAULT_ITEMS_FIELD,
@@ -230,6 +238,7 @@ export class PaginationHelpers {
       folderId,
       headers: providedHeaders,
       additionalParams,
+      queryParams,
       transformFn,
       method = HTTP_METHODS.GET,
       options = {}
@@ -249,13 +258,13 @@ export class PaginationHelpers {
       response = await serviceAccess.post<any>(
         endpoint,
         additionalParams,
-        { headers }
+        { headers, params: queryParams }
       );
     } else {
       response = await serviceAccess.get<any>(
         endpoint,
         {
-          params: additionalParams,
+          params: { ...additionalParams, ...queryParams },
           headers
         }
       );
@@ -309,7 +318,7 @@ export class PaginationHelpers {
     const excludeKeys = config.excludeFromPrefix || [];
     const keysToPrefix = Object.keys(processedOptions).filter(k => !excludeKeys.includes(k));
     const prefixedOptions = addPrefixToKeys(processedOptions, ODATA_PREFIX, keysToPrefix);
-    
+
     // Default pagination options
     const paginationOptions = {
       paginationType: PaginationType.OFFSET,
@@ -327,6 +336,7 @@ export class PaginationHelpers {
         headers: config.headers,
         paginationParams: cursor ? { cursor, pageSize } : jumpToPage !== undefined ? { jumpToPage, pageSize } : { pageSize },
         additionalParams: prefixedOptions,
+        queryParams: config.queryParams,
         transformFn: config.transformFn,
         method: config.method,
         options: {
@@ -345,6 +355,7 @@ export class PaginationHelpers {
       folderId,
       headers: config.headers,
       additionalParams: prefixedOptions,
+      queryParams: config.queryParams,
       transformFn: config.transformFn,
       method: config.method,
       options: {

src/utils/pagination/internal-types.ts

@@ -74,6 +74,8 @@ export interface GetAllPaginatedParams<T, R = T> {
   headers?: Record<string, string>;
   paginationParams: PaginationOptions;
   additionalParams: Record<string, any>;
+  /** URL query params kept in the URL regardless of HTTP method. */
+  queryParams?: Record<string, string | number | boolean>;
   /**
    * Optional function to transform API response items.
    */
@@ -107,6 +109,8 @@ export interface GetAllNonPaginatedParams<T, R = T> {
   /** Pre-resolved request headers. Overrides the helper's auto-built folder header from `folderId`. */
   headers?: Record<string, string>;
   additionalParams: Record<string, any>;
+  /** URL query params kept in the URL regardless of HTTP method. */
+  queryParams?: Record<string, string | number | boolean>;
   /**
    * Optional function to transform API response items.
    */
@@ -236,9 +240,16 @@ export interface GetAllConfig<TRaw, TTransformed = TRaw> {
   /** Keys to exclude from ODATA prefix transformation */
   excludeFromPrefix?: string[];
 
+  /**
+   * Pre-built URL query parameters that always travel in the URL query string,
+   * regardless of HTTP method. Use `createParams({ ... })` to build this.
+   * Typical use: response-shape modifiers on POST endpoints (e.g. `expansionLevel`).
+   */
+  queryParams?: Record<string, string | number | boolean>;
+
   /** HTTP method to use for the request (default: 'GET') */
   method?: HttpMethodType;
 
   /** Pre-resolved request headers. Overrides the helper's auto-built folder header from `folderId`. */
   headers?: Record<string, string>;
-} 
+}

tests/integration/shared/data-fabric/entities.integration.test.ts

@@ -829,6 +829,135 @@ describe.each(modes)('Data Fabric Entities - Integration Tests [%s]', (mode) =>
       expect(typeof row.total).toBe('number');
       expect(row.total).toBeGreaterThanOrEqual(0);
     });
+
+    // Regression guard: DF reads `expansionLevel` only from the URL on POST record endpoints.
+    // If the SDK sends it in the body, DF silently ignores it and every level collapses to L0.
+    it('should expand reference fields at each expansionLevel (0-3)', async () => {
+      const { entities } = getServices();
+      const config = getTestConfig();
+      const entityId = config.dataFabricTestEntityId || testEntityId;
+      if (!entityId) {
+        throw new Error('No entity ID available for testing');
+      }
+
+      const levels = [0, 1, 2, 3] as const;
+      const responses = await Promise.all(
+        levels.map(level => entities.queryRecordsById(entityId, { expansionLevel: level, pageSize: 1 })),
+      );
+
+      responses.forEach((resp, i) => {
+        expect(resp, `expansionLevel=${levels[i]} returned no response`).toBeDefined();
+        expect(Array.isArray(resp.items), `expansionLevel=${levels[i]} items not an array`).toBe(true);
+      });
+
+      if (responses.some(r => r.items.length === 0)) {
+        throw new Error('Test entity has no records — expansionLevel diff cannot be verified. Insert at least one record into DATA_FABRIC_TEST_ENTITY_ID.');
+      }
+
+      const records = responses.map(r => r.items[0] as Record<string, any>);
+      const [l0Record, l1Record, l2Record, l3Record] = records;
+
+      // CreatedBy is a system reference field on every DF record.
+      // L0: raw GUID string. L1+: object envelope with Id.
+      expect(typeof l0Record.CreatedBy).toBe('string');
+
+      for (const [level, rec] of [[1, l1Record], [2, l2Record], [3, l3Record]] as const) {
+        expect(typeof rec.CreatedBy, `L${level} CreatedBy should be object`).toBe('object');
+        expect(rec.CreatedBy, `L${level} CreatedBy should not be null`).not.toBeNull();
+        expect(rec.CreatedBy, `L${level} CreatedBy should have Id`).toHaveProperty('Id');
+      }
+
+      // L2 inflates the L1 envelope into the referenced user's full record.
+      expect(Object.keys(l2Record.CreatedBy).length).toBeGreaterThan(
+        Object.keys(l1Record.CreatedBy).length,
+      );
+
+      // L3 must remain at least as expanded as L2 (deeper nesting is schema-dependent, but never shrinks).
+      expect(Object.keys(l3Record.CreatedBy).length).toBeGreaterThanOrEqual(
+        Object.keys(l2Record.CreatedBy).length,
+      );
+    });
+
+    // Custom user-defined RELATIONSHIP fields must follow the same expansion rules as
+    // system reference fields, but verifying this requires creating an isolated
+    // source→target schema at runtime. entities.create() needs the entity.schema.write
+    // scope, which PAT tokens cannot hold (insufficient_scope) — so this stays skipped,
+    // matching the other schema-write describe.skip blocks below. The CreatedBy test
+    // above already validates the expansionLevel-via-URL fix under PAT.
+    describe.skip('RELATIONSHIP field expansion (requires schema-write scope)', () => {
+      it('should expand a custom RELATIONSHIP field at each expansionLevel (0-3)', async () => {
+        const { entities } = getServices();
+        const stamp = generateRandomString(8).toLowerCase();
+
+        // 1. Target entity with a user-defined string field we can assert on at L2+.
+        const targetId = await entities.create(`sdk_target_${stamp}`, [
+          { fieldName: 'label', type: EntityFieldDataType.STRING, isRequired: true },
+        ]);
+        createdEntityIds.push(targetId);
+
+        const targetMeta = await entities.getById(targetId);
+        const targetPk = targetMeta.fields.find(f => f.isPrimaryKey);
+        if (!targetPk?.id) {
+          throw new Error(`Target entity ${targetId} has no primary-key field`);
+        }
+
+        // 2. Source entity with a RELATIONSHIP field bound to target.Id.
+        const sourceId = await entities.create(`sdk_source_${stamp}`, [
+          { fieldName: 'name', type: EntityFieldDataType.STRING },
+          {
+            fieldName: 'parent',
+            type: EntityFieldDataType.RELATIONSHIP,
+            referenceEntityId: targetId,
+            referenceFieldId: targetPk.id,
+          },
+        ]);
+        createdEntityIds.push(sourceId);
+
+        // 3. Insert a target record so the FK has something to resolve.
+        const labelValue = `Target_${stamp}`;
+        const targetInsert = await entities.insertRecordById(targetId, { label: labelValue });
+        const targetRecordId = targetInsert.Id;
+        registerResource('entityRecords', { entityId: targetId, recordIds: [targetRecordId] });
+
+        // 4. Insert the source record with the FK populated.
+        const sourceInsert = await entities.insertRecordById(sourceId, {
+          name: `Source_${stamp}`,
+          parent: targetRecordId,
+        });
+        registerResource('entityRecords', { entityId: sourceId, recordIds: [sourceInsert.Id] });
+
+        // 5. Query the source at every expansion level.
+        const levels = [0, 1, 2, 3] as const;
+        const responses = await Promise.all(
+          levels.map(level => entities.queryRecordsById(sourceId, { expansionLevel: level, pageSize: 10 })),
+        );
+
+        const sourceRecords = responses.map(r =>
+          (r.items as Record<string, any>[]).find(item => item.Id === sourceInsert.Id),
+        );
+        sourceRecords.forEach((rec, i) => {
+          expect(rec, `expansionLevel=${levels[i]} did not return the inserted source record`).toBeDefined();
+        });
+        const [l0, l1, l2, l3] = sourceRecords as Record<string, any>[];
+
+        // L0: FK is the raw target record Id string.
+        expect(typeof l0.parent).toBe('string');
+        expect(l0.parent).toBe(targetRecordId);
+
+        // L1+: FK inflates into an object envelope carrying the target Id.
+        for (const [level, rec] of [[1, l1], [2, l2], [3, l3]] as const) {
+          expect(typeof rec.parent, `L${level} parent should be object`).toBe('object');
+          expect(rec.parent, `L${level} parent should not be null`).not.toBeNull();
+          expect(rec.parent, `L${level} parent should carry target Id`).toHaveProperty('Id', targetRecordId);
+        }
+
+        // L2 surfaces the user-defined `label` field from the target record.
+        expect(l2.parent.label).toBe(labelValue);
+
+        // L3 cannot shrink relative to L2.
+        expect(Object.keys(l3.parent).length).toBeGreaterThanOrEqual(Object.keys(l2.parent).length);
+      });
+    });
   });
 
   describe('importRecordsById', () => {

tests/unit/services/base.test.ts

@@ -407,7 +407,7 @@ describe('BaseService Unit Tests', () => {
       expect(params).not.toHaveProperty(ODATA_OFFSET_PARAMS.COUNT_PARAM);
     });
 
-    it('should merge pagination params into the body for POST and clear query params', async () => {
+    it('should merge pagination params into body for POST while leaving caller params in the URL', async () => {
       mockApiClient.post.mockResolvedValue({ value: [TEST_RESPONSE], totalRecordCount: 1 });
 
       await service.exposedRequestWithPagination(
@@ -423,14 +423,14 @@ describe('BaseService Unit Tests', () => {
 
       const [calledPath, calledBody, calledOptions] = mockApiClient.post.mock.calls[0];
       expect(calledPath).toBe(TEST_PATH);
+      // Pagination state lands in body. Caller's body keys are preserved. No URL keys leak in.
       expect(calledBody).toEqual({
         filter: 'abc',
-        extra: 'query',
         [ODATA_OFFSET_PARAMS.PAGE_SIZE_PARAM]: 10,
         [ODATA_OFFSET_PARAMS.COUNT_PARAM]: true,
       });
-      expect(calledOptions.params).toBeUndefined();
-      expect(calledOptions.body).toEqual(calledBody);
+      // Caller's URL params are respected — no longer clobbered.
+      expect(calledOptions.params).toEqual({ extra: 'query' });
     });
 
     it('should fall back to itemsCount heuristic for hasNextPage when totalCount is missing', async () => {

tests/unit/services/data-fabric/entities.test.ts

@@ -1505,7 +1505,7 @@ describe("EntityService Unit Tests", () => {
             itemsField: "value",
             totalCountField: "totalRecordCount",
           }),
-          excludeFromPrefix: expect.arrayContaining(["expansionLevel", "filterGroup", "sortOptions"]),
+          excludeFromPrefix: expect.arrayContaining(["filterGroup", "sortOptions"]),
         }),
         undefined,
       );
@@ -1557,24 +1557,28 @@ describe("EntityService Unit Tests", () => {
       );
     });
 
-    it("should pass expansionLevel through excludeFromPrefix without ODATA prefix", async () => {
+    it("should route expansionLevel to queryParams and strip it from the body options", async () => {
       let capturedConfig: any;
-      vi.mocked(PaginationHelpers.getAll).mockImplementation(async (config) => {
+      let capturedDownstream: any;
+      vi.mocked(PaginationHelpers.getAll).mockImplementation(async (config, downstream) => {
         capturedConfig = config;
+        capturedDownstream = downstream;
         return { items: [], totalCount: 0 };
       });
 
       await entityService.queryRecordsById(ENTITY_TEST_CONSTANTS.ENTITY_ID, {
         expansionLevel: ENTITY_TEST_CONSTANTS.EXPANSION_LEVEL,
       });
 
-      // expansionLevel is in excludeFromPrefix — no URL manipulation
       expect(capturedConfig.getEndpoint()).toBe(
         DATA_FABRIC_ENDPOINTS.ENTITY.QUERY_BY_ID(ENTITY_TEST_CONSTANTS.ENTITY_ID),
       );
-      expect(capturedConfig.getEndpoint()).not.toContain("expansionLevel");
-      expect(capturedConfig.excludeFromPrefix).toContain("expansionLevel");
-      // no processParametersFn — same pattern as getAllRecords
+      // expansionLevel is hoisted into queryParams (URL) — same pattern as insertRecordById / updateRecordById.
+      expect(capturedConfig.queryParams).toEqual({
+        expansionLevel: ENTITY_TEST_CONSTANTS.EXPANSION_LEVEL,
+      });
+      // It must not also appear in the downstream (body) options.
+      expect(capturedDownstream).not.toHaveProperty("expansionLevel");
       expect(capturedConfig.processParametersFn).toBeUndefined();
     });