Skip to content

refactor: optimize az descendant queries BED-4867#2892

Merged
urangel merged 7 commits into
mainfrom
BED-4867
Jun 23, 2026
Merged

refactor: optimize az descendant queries BED-4867#2892
urangel merged 7 commits into
mainfrom
BED-4867

Conversation

@urangel

@urangel urangel commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Description

Refactors are made to the azure entity descendant relationship data retrieval query logic for improving efficiency and query duration.

Motivation and Context

Resolves BED-4867

Our current methods for obtaining entity descendant relationship data often result in timeout errors in modestly sized tenants. This issue is exacerbated when attempting to view this information through the entity panel in the UI as a series of requests are executed through a promise.all call that hinges on all of the requests succeeding and otherwise fails to show any information even if only one of the requests fails.

The logic has been updated to pivot based on source and target kinds in order to route queries more effectively. Our modeled AZContains relationships include those that are only ever 1 hop away (like AZTenant to AZDevice) and those that may be several hops away (AZTenant to AZLogicApp will at minimum cross an AZSubscription and AZResourceGroup).

We were previously traversing any AZContains relationships then returning paths that had the target kind in the path. This means that we would follow AZContains to deep terminals for any target kind when most if not all traversals could be eliminated up front.

The logic has been updated to first pull the set of terminals by kind and tenant id then traversing backwards to the source for relationships that are nested. Direct relationships simply do an edge filter rather than a traversal.

Finally, the UI is updated to use promise.allSettled over promise.all so that successful nested sections are still displayed and functional even if a sibling section fails.

How Has This Been Tested?

API integration tests have been added. The azure example data was uploaded and various descendant entity panels were inspected (the AZTenant descendants section was of particular focus).

Screenshots (optional):

Types of changes

  • Chore (a change that does not modify the application functionality)

Checklist:

Summary by CodeRabbit

Release Notes

  • New Features

    • Improved Azure “related entities” rendering to better respect source context across graph and list views.
  • Performance Improvements

    • Faster, tenant-scoped descendant traversal and more consistent descendant counting for entity detail panels.
  • Bug Fixes

    • Fixed graphed table count aggregation across multiple section requests, including partial failure scenarios.
    • Updated the related-entities API to return a generic error if display-kind lookup fails.
  • Tests

    • Expanded Azure integration tests for direct vs non-direct descendant paths and nested listings.

@urangel urangel self-assigned this Jun 15, 2026
@urangel urangel added user interface A pull request containing changes affecting the UI code. go Pull requests that update go code labels Jun 15, 2026
@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 51e97e73-d810-4f50-b3cf-bcabe5c681bf

📥 Commits

Reviewing files that changed from the base of the PR and between 8557a8e and a02ff87.

📒 Files selected for processing (1)
  • packages/go/analysis/azure/queries.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/go/analysis/azure/queries.go

📝 Walkthrough

Walkthrough

Introduces direct-vs-indirect descendant routing for Azure graph traversal: queries.go is reimplemented with new FetchDirect* helpers and updated FetchEntityDescendents/FetchEntityDescendentCounts signatures that remove pagination parameters. A sourceKind parameter threads from GetAZRelatedEntities through db_ops to the query layer to drive route selection. All caller sites update to the new signatures. EntityInfoDataTableGraphed gains a getCount helper and switches multi-section fetches to Promise.allSettled for graceful partial failure handling.

Changes

Azure Descendant Traversal Refactoring

Layer / File(s) Summary
New descendant query primitives and traversal logic
packages/go/analysis/azure/queries.go
Adds directRelationshipKinds mapping, isDirectDescendent, and nodeAzureTenantID helpers; introduces FetchDirectDescendentPaths, FetchDirectEntityDescendents, and FetchDescendentKindByTenantID; reimplements FetchEntityDescendentPaths with tenant-scoped terminal selection and inbound azure.Contains path traversal to the root; updates FetchEntityDescendents to remove skip/limit and accept a single descendentKind; updates FetchEntityDescendentCounts signature to remove skip/limit; adds FetchDirectDescendentCounts for direct-path count computation.
db_ops sourceKind dispatch routing
packages/go/analysis/azure/db_ops.go
ListEntityDescendentPaths and ListEntityDescendents gain a sourceKind graph.Kind parameter and branch on isDirectDescendent to route to direct or general fetch functions; skip/limit removed from downstream Fetch* call sites.
API layer options struct and sourceKind propagation
cmd/api/src/api/v2/azure.go, cmd/api/src/api/v2/azure_test.go
Introduces relatedEntityTypeOptions struct consolidating entity type, object ID, source kind, and pagination; refactors graphRelatedEntityType and listRelatedEntityType to consume options and precomputed primaryDisplayKinds; GetAZRelatedEntities gains sourceKind graph.Kind parameter, computes primaryDisplayKinds once, and routes both graph and list paths through updated helpers; GetAZEntity passes azKind as sourceKind; test error mapping expectation updated.
Caller sites updated to new query signatures
packages/go/analysis/azure/management_group.go, packages/go/analysis/azure/subscription.go, packages/go/analysis/azure/tenant.go, packages/go/analysis/azure/resource_group.go
Removes 0, 0 arguments from FetchEntityDescendentCounts calls in management group, subscription, and tenant; switches resource group to FetchDirectDescendentCounts.
Integration tests for descendant path traversal
packages/go/analysis/azure/azure_integration_test.go
Six new integration tests cover direct-path AZContains children to root, multi-hop traversal with intermediate nodes, empty results when no target-kind terminals exist, shared intermediate node between multiple terminals, terminal disconnected from root, and nested management-group-to-subscription listing via ListEntityDescendents.

EntityInfoDataTableGraphed Count Normalization

Layer / File(s) Summary
getCount helper and Promise.allSettled multi-section support
packages/javascript/bh-shared-ui/src/components/EntityInfoDataTableGraphed/EntityInfoDataTableGraphed.tsx, packages/javascript/bh-shared-ui/src/components/EntityInfoDataTableGraphed/EntityInfoDataTableGraphed.test.tsx
Imports PaginatedResponse; adds getCount helper that normalizes counts from a single PaginatedResponse or an array of settled promise results with optional countLabel filtering; switches multi-section fetch to Promise.allSettled for graceful partial failure; replaces inline count calculation with getCount; updates test to verify count sum on partial failure and confirm error icon is absent.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Suggested reviewers

  • mistahj67
  • elikmiller
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 51.85% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: refactoring Azure descendant queries for optimization, with the associated ticket reference.
Description check ✅ Passed The description includes all required template sections: clear description of changes, motivation and context with ticket resolution, testing methodology, and completed checklist items. All major requirements are addressed.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch BED-4867

Comment @coderabbitai help to get the list of available commands.

@urangel urangel changed the title Bed 4867 refactor: optimize az descendant queries BED-4867 Jun 15, 2026
@urangel urangel marked this pull request as draft June 15, 2026 22:23

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
packages/javascript/bh-shared-ui/src/components/EntityInfoDataTableGraphed/EntityInfoDataTableGraphed.tsx (1)

26-26: 💤 Low value

Type annotation does not match Promise.allSettled output.

Promise.allSettled returns PromiseSettledResult<T>[] (union of PromiseFulfilledResult and PromiseRejectedResult), but the parameter is typed as PromiseFulfilledResult<T>[]. The runtime filtering on line 30 handles both cases, but the type is inaccurate.

♻️ Suggested type correction
 function getCount<T>(
-    queryData: Array<PromiseFulfilledResult<PaginatedResponse<T>>> | PaginatedResponse<T> | undefined,
+    queryData: Array<PromiseSettledResult<PaginatedResponse<T>>> | PaginatedResponse<T> | undefined,
     countLabel: string | undefined
 ): number | undefined {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@packages/javascript/bh-shared-ui/src/components/EntityInfoDataTableGraphed/EntityInfoDataTableGraphed.tsx`
at line 26, The type annotation for the queryData parameter on line 26 in
EntityInfoDataTableGraphed.tsx does not accurately reflect the output of
Promise.allSettled, which returns a union of both PromiseFulfilledResult and
PromiseRejectedResult. Change the type annotation for queryData from
Array<PromiseFulfilledResult<PaginatedResponse<T>>> to
Array<PromiseSettledResult<PaginatedResponse<T>>> to correctly represent the
complete settled result type that includes both fulfilled and rejected promises.
packages/go/analysis/azure/azure_integration_test.go (1)

953-1046: ⚡ Quick win

Add a sibling-branch regression test for non-tenant roots.

These new tests only anchor the root at a tenant. Please add a case with a non-tenant root (e.g., one subscription under a tenant with a sibling subscription) and assert only the selected root’s subtree is returned.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/go/analysis/azure/azure_integration_test.go` around lines 953 -
1046, Add a new test function after
TestFetchEntityDescendentPaths_NoTerminalsReturnsEmpty to verify that
FetchEntityDescendentPaths correctly handles non-tenant root nodes. The test
should create a tenant with two sibling subscriptions, each containing an
application, then call FetchEntityDescendentPaths with one subscription as the
root and graphAzure.App as the target kind. Assert that the returned paths
contain only the nodes from the selected root's subtree (the chosen subscription
and its app), excluding the sibling subscription and its app. This ensures the
function properly isolates each branch when starting from a non-tenant node.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/go/analysis/azure/queries.go`:
- Around line 860-870: The FetchDescendentKindByTenantID function at lines
860-870 filters nodes only by tenant ID and kind, but does not ensure the
results are actual descendants of the provided root node. This allows sibling
branches in the same tenant to be included in results. Additionally, the code at
lines 897-915 applies this same flawed traversal logic. Fix this by adding a
constraint to the query that ensures returned nodes are actual descendants of
the root node parameter, not merely nodes within the same tenant. This
constraint should be applied in the query.And() call at line 862 where you build
the filter conditions, and any similar filtering logic at lines 897-915 should
also be updated to include this descendant relationship validation to properly
scope results to the requested root's subtree.

In
`@packages/javascript/bh-shared-ui/src/components/EntityInfoDataTableGraphed/EntityInfoDataTableGraphed.tsx`:
- Around line 32-35: The `forEach` loop with a `return` statement inside the
callback only exits the callback, not the outer `getCount` function, causing the
function to fall through and return `undefined` implicitly. Replace the
`forEach` call with the `find` method when filtering `fulfilledData` by
`countLabel`. The `find` method will properly return the matching `sectionData`
object, allowing you to extract and return its `count` property directly, fixing
the countLabel filtering feature so the correct count is displayed instead of
`undefined`.

---

Nitpick comments:
In `@packages/go/analysis/azure/azure_integration_test.go`:
- Around line 953-1046: Add a new test function after
TestFetchEntityDescendentPaths_NoTerminalsReturnsEmpty to verify that
FetchEntityDescendentPaths correctly handles non-tenant root nodes. The test
should create a tenant with two sibling subscriptions, each containing an
application, then call FetchEntityDescendentPaths with one subscription as the
root and graphAzure.App as the target kind. Assert that the returned paths
contain only the nodes from the selected root's subtree (the chosen subscription
and its app), excluding the sibling subscription and its app. This ensures the
function properly isolates each branch when starting from a non-tenant node.

In
`@packages/javascript/bh-shared-ui/src/components/EntityInfoDataTableGraphed/EntityInfoDataTableGraphed.tsx`:
- Line 26: The type annotation for the queryData parameter on line 26 in
EntityInfoDataTableGraphed.tsx does not accurately reflect the output of
Promise.allSettled, which returns a union of both PromiseFulfilledResult and
PromiseRejectedResult. Change the type annotation for queryData from
Array<PromiseFulfilledResult<PaginatedResponse<T>>> to
Array<PromiseSettledResult<PaginatedResponse<T>>> to correctly represent the
complete settled result type that includes both fulfilled and rejected promises.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: b184f80c-bbbe-467e-ba8d-6a6f7613c556

📥 Commits

Reviewing files that changed from the base of the PR and between 41c8ea3 and 395c7fb.

📒 Files selected for processing (9)
  • cmd/api/src/api/v2/azure.go
  • packages/go/analysis/azure/azure_integration_test.go
  • packages/go/analysis/azure/db_ops.go
  • packages/go/analysis/azure/management_group.go
  • packages/go/analysis/azure/queries.go
  • packages/go/analysis/azure/resource_group.go
  • packages/go/analysis/azure/subscription.go
  • packages/go/analysis/azure/tenant.go
  • packages/javascript/bh-shared-ui/src/components/EntityInfoDataTableGraphed/EntityInfoDataTableGraphed.tsx

Comment thread packages/go/analysis/azure/queries.go
@urangel urangel marked this pull request as ready for review June 16, 2026 16:26
@coderabbitai coderabbitai Bot added api A pull request containing changes affecting the API code. javascript Pull requests that update javascript code labels Jun 16, 2026
Comment thread packages/go/analysis/azure/queries.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/go/analysis/azure/azure_integration_test.go (1)

1-1122: 📐 Maintainability & Code Quality | 🟡 Minor

Update license header copyright year to 2026.

The file has the correct license header structure and build tag (//go:build integration) placed correctly underneath it (line 17). However, the copyright year in the license header (line 1) is 2023, while the LICENSE.header reference file specifies 2026. Per the coding guidelines, the license header must use the up-to-date version from LICENSE.header.

Update copyright year from 2023 to 2026:
-// Copyright 2023 Specter Ops, Inc.
+// Copyright 2026 Specter Ops, Inc.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/go/analysis/azure/azure_integration_test.go` around lines 1 - 1122,
The copyright year in the license header at the beginning of the file is
outdated and needs to be updated from 2023 to 2026 to match the current
LICENSE.header reference file and coding guidelines. Update the copyright year
in the first line of the license header comment block that appears before the
build tag.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/go/analysis/azure/azure_integration_test.go`:
- Around line 1124-1170: The test
TestListEntityDescendents_NestedManagementGroupToSubscription expects nestedSub
to be returned as a descendent of mgRoot through a 2-hop path (mgRoot → mgChild
→ nestedSub), but ListEntityDescendents routes to FetchDirectEntityDescendents
which only performs single-hop filtering on direct relationships. To fix this
mismatch, either remove the intermediate mgChildNode and create a direct
Contains relationship from mgRootNode to nestedSubNode to match the single-hop
behavior of FetchDirectEntityDescendents, or if multi-hop traversal is required,
modify the implementation to use FetchEntityDescendents instead of
FetchDirectEntityDescendents for this scenario. Choose the approach that aligns
with the intended Azure resource hierarchy behavior.

---

Outside diff comments:
In `@packages/go/analysis/azure/azure_integration_test.go`:
- Around line 1-1122: The copyright year in the license header at the beginning
of the file is outdated and needs to be updated from 2023 to 2026 to match the
current LICENSE.header reference file and coding guidelines. Update the
copyright year in the first line of the license header comment block that
appears before the build tag.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: d629cc23-8aac-4b3f-9015-2751e77bd2d5

📥 Commits

Reviewing files that changed from the base of the PR and between ef8c299 and 8557a8e.

📒 Files selected for processing (1)
  • packages/go/analysis/azure/azure_integration_test.go

Comment thread packages/go/analysis/azure/azure_integration_test.go

@AD7ZJ AD7ZJ left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, working nicely on my local setup.

@urangel urangel merged commit 3220cda into main Jun 23, 2026
13 checks passed
@urangel urangel deleted the BED-4867 branch June 23, 2026 19:47
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 23, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api A pull request containing changes affecting the API code. go Pull requests that update go code javascript Pull requests that update javascript code user interface A pull request containing changes affecting the UI code.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants