Bitsight risk findings ccf#14442
Conversation
This README.md provides comprehensive documentation for the BitSight Risk Findings Data Connector, including its purpose, deployment instructions, and operational details.
…iskFindingsLogs_ccf/test
|
Hello Team, I confirm this is a CCF (pull / RestApiPoller) connector. I've restructured the solution to the V3 layout: the connector is now split into the four source files under Data Connectors/BitSightRiskFindingsLogs_ccf/ (_ConnectorDefinition, _PollerConfig, _DCR, _Table), with the Data/Solution_BitSightRiskFindingsCCF.json data file added, SolutionMetadata.json at the root, and content moved to the standard Hunting Queries/ and Analytic Rules/ folders. Repackaged with createSolutionV3.ps1 at version 3.0.0. Live-tested in a Sentinel workspace and connect succeeds and data ingests into BitsightRiskFindings_CL. The only remaining arm-ttk failures are the standard CCF ones (IDs Should Be Derived From ResourceIDs and Template Should Not Contain Blanks). Also the workbook will follow in a subsequent revision. Its re-open from my previous mistake to close the PR14361! |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a new reduced-scope BitSight Risk Findings Microsoft Sentinel solution that ingests BitSight findings via Codeless Connector Framework (RestApiPoller + DCR transform), plus workbook/hunting/detection content.
Changes:
- Added solution metadata, documentation, and release notes for the new BitSight Risk Findings CCF solution.
- Added CCF connector artifacts (connector definition, poller config, DCR transform, custom table schema) and packaged ARM templates.
- Added 5 hunting queries and 1 scheduled analytic rule targeting WARN/BAD findings.
Reviewed changes
Copilot reviewed 19 out of 21 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| Solutions/BitSight Risk Findings CCF/SolutionMetadata.json | Defines marketplace/solution metadata (publisher/offer/support/categories). |
| Solutions/BitSight Risk Findings CCF/ReleaseNotes.md | Adds the solution’s release notes entry for initial package. |
| Solutions/BitSight Risk Findings CCF/README.md | Documents deployment, prerequisites, and validation for the CCF connector. |
| Solutions/BitSight Risk Findings CCF/Package/testParameters.json | Provides ARM test parameters for ARM-TTK validation. |
| Solutions/BitSight Risk Findings CCF/Package/mainTemplate.json | Packages the solution, connector, DCR, table, hunting queries, and analytic rule. |
| Solutions/BitSight Risk Findings CCF/Package/createUiDefinition.json | Packages the UI definition for solution installation flow. |
| Solutions/BitSight Risk Findings CCF/Hunting Queries/NewlyObservedBadBitSightFindingsLast24Hours.yaml | Adds hunting query to find newly observed BAD findings in last 24h. |
| Solutions/BitSight Risk Findings CCF/Hunting Queries/LatestWarnBadBitSightFindings.yaml | Adds hunting query for latest WARN/BAD findings. |
| Solutions/BitSight Risk Findings CCF/Hunting Queries/CompanyRiskPostureSummary.yaml | Adds summary hunting query by company/risk vector/state. |
| Solutions/BitSight Risk Findings CCF/Hunting Queries/BitSightFindingTrendByRiskVector.yaml | Adds trend hunting query by risk vector over time. |
| Solutions/BitSight Risk Findings CCF/Hunting Queries/ActiveBadBitSightFindingsByCompany.yaml | Adds hunting query summarizing active BAD findings by company. |
| Solutions/BitSight Risk Findings CCF/Data/Solution_BitSightRiskFindingsCCF.json | Declares solution content inventory and packaging inputs. |
| Solutions/BitSight Risk Findings CCF/Data Connectors/BitSightRiskFindingsLogs_ccf/BitsightRiskFindings.kql | Provides sample KQL queries for validation/monitoring. |
| Solutions/BitSight Risk Findings CCF/Data Connectors/BitSightRiskFindingsLogs_ccf/BitSightRiskFindings_Table.json | Defines the custom table schema for BitsightRiskFindings_CL. |
| Solutions/BitSight Risk Findings CCF/Data Connectors/BitSightRiskFindingsLogs_ccf/BitSightRiskFindings_PollerConfig.json | Defines RestApiPoller settings for BitSight API polling and paging. |
| Solutions/BitSight Risk Findings CCF/Data Connectors/BitSightRiskFindingsLogs_ccf/BitSightRiskFindings_DCR.json | Defines DCR stream/transformKql for normalization and filtering. |
| Solutions/BitSight Risk Findings CCF/Data Connectors/BitSightRiskFindingsLogs_ccf/BitSightRiskFindings_ConnectorDefinition.json | Defines the connector UI/permissions/instructions. |
| Solutions/BitSight Risk Findings CCF/Analytic Rules/NewBadBitSightRiskFinding.yaml | Adds scheduled analytic rule for newly observed BAD findings. |
| .script/tests/KqlvalidationsTests/BitsightRiskFindings_CL.json | Adds schema for KQL validation tests for the new custom table. |
Updated author information, logo format, and description for the BitSight Risk Findings CCF solution. Changed base path to relative format.
KanenasCS
left a comment
KanenasCS
left a comment
There was a problem hiding this comment.
Dear Team,
I have reviewed and addressed the Copilot review comments.
The required metadata updates have been completed, including the approved community publisherId, the required support email, and the aligned README publisher statement.
I also updated the solution data file conventions, including the author contact format, repository-relative BasePath, and repository-hosted logo reference. The README and validation KQL snippets were also reviewed and corrected where required.
For the ARM expression comments related to [[... ] syntax in the packaged template, I reviewed them and confirmed that the escaping is intentional for nested/embedded ARM template evaluation during connector deployment. Therefore, those comments were resolved without changing the expression syntax.
The package has been validated based on the current PR scope.
|
Dear Team, I have reviewed and addressed the Copilot review comments. The required metadata updates have been completed, including the approved community publisherId, the required support email, and the aligned README publisher statement. I also updated the solution data file conventions, including the author contact format, repository-relative BasePath, and repository-hosted logo reference. The README and validation KQL snippets were also reviewed and corrected where required. For the ARM expression comments related to [[... ] syntax in the packaged template, I reviewed them and confirmed that the escaping is intentional for nested/embedded ARM template evaluation during connector deployment. Therefore, those comments were resolved without changing the expression syntax. The package has been validated based on the current PR scope. |
|
Hi @KanenasCS, thanks for the update. I'll review the PR and provide update if anything required. Thanks! |
Placed in wrong place thats why failed the KQLValidation test.
Place the Custom Table in to the right path-folder thats why the KQLValidation test fails.
|
Hello Team, I apologize. I placed the custom table in the wrong folder, which caused the KqlValidations test to fail. |
|
Hi @KanenasCS, could you add the solutions logo to the location below, update the logo path in the data file, and repackage the solution. Thanks! |
The sole arm-ttk failure is "IDs Should Be Derived From ResourceIDs". It fires on the content-metadata `id` and `contentProductId` properties (and, on older arm-ttk builds, on the CCF `dataCollectionEndpointId` / `dataCollectionRuleImmutableId` properties, which current arm-ttk already exempts as CCP/CCF connector exceptions). These property values are emitted by the packaging tool's generation logic, not by solution source content, and follow the standard concat(take(...), uniqueString(...)) pattern. They cannot be changed without editing the generated package post-build, which the tool overwrites on every run. Per the V3 packaging README, this is explicitly safe to ignore: https://github.com/Azure/Azure-Sentinel/blob/master/Tools/Create-Azure-Sentinel-Solution/V3/README.md#arm-ttk-failue-for-contentproductid-id-issues Result: 48 pass / 1 fail (the documented false positive). The connector, analytic rule, and hunting queries deploy and function correctly.
|
Hello @v-shukore i apologize for the delay. I have rebuild via V3 tool. All other 48 checks pass. No functional issue, the CCF connector deploys correctly. |
|
Hi @KanenasCS, to resolve solution validation please update |
…ata file to include the required fields and correct values.
|
Thank you for you guidance @v-shukore . Its fixed |
…gests full findings set; ensure Community support tier; regenerate package"
|
Dear @v-shukore , The package passes 49 of 50 checks. The single failure is "IDs Should Be These values are produced by the V3 packaging tool itself (createSolutionV3 → The solution was deployed and validated against a live workspace: the CCF Happy to make any changes you'd like. |
Updated publisher and author information, clarified Data Collection Endpoint (DCE) requirements, and improved descriptions for authentication and polling processes.
|
Hi @KanenasCS, could you please confirm that this solution is supported as |
|
Hi @v-shukore , |
Change(s):
Added a new reduced-scope BitSight Microsoft Sentinel solution under Solutions/BitSightRiskFindings/.
Added a BitSight Risk Findings data connector built on the Codeless Connector Framework (CCF), using a managed RestApiPoller + DCR ingestion model (no Azure Function App), to support targeted monitoring of the following BitSight risk vectors: Botnet Infections, Spam Propagation, Malware Servers, Unsolicited Communications, Potentially Exploited, TLS/SSL Certificates, Patching Cadence, Mobile Software, Open Ports, File Sharing.
The connector authenticates to the BitSight API (token sent as Basic-equivalent auth), follows links.next pagination, and performs risk-state normalization and field flattening in the DCR transform before writing to BitsightRiskFindings_CL.
Added solution content for the reduced connector: 1 workbook, 5 hunting queries, 1 scheduled analytic rule.
Added supporting solution documentation and release notes.
Reason for Change(s):
The previous BitSight connector path relied on the Microsoft Sentinel HTTP Data Collector API, which has been retired.
The interim DCE + DCR + Function App model has been superseded by the Codeless Connector Framework, Microsoft's current supported, no-infrastructure SaaS ingestion approach — it provides built-in health monitoring and removes the Function App, storage account, and App Service plan that the duo model required.
Migrating to CCF provides a supportable Microsoft Sentinel integration for the agreed reduced-scope monitoring use case; the API polling, authentication, and field-normalization logic that previously ran in the Function App now run in the CCF poller and the DCR transform.
The integration remains focused on WARN and BAD findings only, aligned to the selected BitSight risk vectors and intended operational use in Microsoft Sentinel.
This update also provides out-of-the-box monitoring value through workbook, hunting, and detection content.
Version Updated: Yes
Updated the analytic rule template version field to 1.0.0.
Testing Completed: Yes
Deployed the CCF solution template into a Microsoft Sentinel environment and connected a test BitSight company.
Validated creation of the DCR and the custom table BitsightRiskFindings_CL at connect time, and successful ingestion of findings.
Validated RestApiPoller authentication and links.next pagination against the BitSight findings endpoint.
Validated the reduced solution scope for WARN and BAD findings.
Reviewed the included workbook, hunting queries, and analytic rule content against the ingested schema.
Checked that the validations are passing and have addressed any issues that are present: Yes