Skip to content

feat: add workload filter support (RHINENG-27093)#2417

Open
adarshdubey-star wants to merge 3 commits into
RedHatInsights:masterfrom
adarshdubey-star:workload_filter
Open

feat: add workload filter support (RHINENG-27093)#2417
adarshdubey-star wants to merge 3 commits into
RedHatInsights:masterfrom
adarshdubey-star:workload_filter

Conversation

@adarshdubey-star

@adarshdubey-star adarshdubey-star commented Jul 14, 2026

Copy link
Copy Markdown

JIra: https://redhat.atlassian.net/browse/RHINENG-27093

Summary

  • Add 5 new workload types (crowdstrike, ibm_db2, intersystems, oracle_db, rhel_ai) to the Vulnerability systems API
  • Unify all workload filters (existing + new) into a single filter function with OR logic, matching Inventory's behavior
  • Ingest new workload data from Kafka inventory events

What changed

  • Database: New migration (166) adds 5 boolean columns to system_platform
  • Listener: Extracts new workloads from system_profile.workloads in Kafka messages
  • Filter logic: Replaced 3 separate filter functions (_filter_system_by_sap, _filter_system_by_ansible, _filter_system_by_mssql) with a single _filter_system_by_workloads that ORs all workload params together
  • API: 5 new boolean query parameters on /systems, /cves/{id}/affected_systems, /vulnerabilities/cves, /dashboard, /dashbar
  • Tests: Filter tests for each new workload type + OR logic verification

Why

  • The Global Filter is being deprecated (RHINENG-25895). Each application needs its own workload filter. This aligns the Vulnerability API with Inventory's workload filtering behavior, where selecting multiple workloads returns systems matching any of them (OR logic).

Summary by Sourcery

Add support for filtering vulnerability-related system data by multiple workload types using unified OR-based logic across handlers and APIs.

New Features:

  • Expose new workload query parameters (crowdstrike, ibm_db2, intersystems, oracle_db, rhel_ai) on systems and CVE-related endpoints to filter systems by these workloads.
  • Persist new workload attributes from inventory Kafka events into the system_platform model for use in vulnerability queries.

Enhancements:

  • Replace separate SAP, Ansible, and MSSQL workload filters with a single workload filter that applies OR logic across all supported workload types.
  • Extend dashboard and dashbar handlers to accept and apply the unified workload filters while keeping cacheability rules aligned.

Build:

  • Add a database migration to introduce new workload columns on system_platform and update development seed data accordingly.

Tests:

  • Add system handler tests to cover filtering by each new workload type and to verify OR logic across combinations of workloads.

@sourcery-ai

sourcery-ai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds workload filter support across the Vulnerability API by introducing new workload columns, ingesting them from inventory events, and unifying all workload-related filtering into a single OR-based workload filter used consistently by systems, CVEs, dashboard, vulnerabilities, and dashbar endpoints.

Sequence diagram for unified workload filtering in Vulnerability API

sequenceDiagram
    actor Client
    participant SystemHandler
    participant Filters as apply_filters
    participant WorkloadFilter as _filter_system_by_workloads
    participant DB as SystemPlatform

    Client->>SystemHandler: GET /systems?sap_system&ansible&crowdstrike
    SystemHandler->>Filters: apply_filters(query, args, [SYSTEM_WORKLOADS])
    Filters->>WorkloadFilter: _filter_system_by_workloads(query, args)
    WorkloadFilter->>WorkloadFilter: build exprs from WORKLOAD_PARAMS
    WorkloadFilter->>DB: query.where(reduce(or_, exprs))
    DB-->>SystemHandler: matching systems
    SystemHandler-->>Client: filtered systems response
Loading

File-Level Changes

Change Details Files
Introduce unified workload filter that applies OR logic across all workload-related query parameters, replacing separate SAP, Ansible, and MSSQL filters.
  • Define WORKLOAD_PARAMS mapping for sap_system, ansible, mssql, crowdstrike, ibm_db2, intersystems, oracle_db, and rhel_ai workload flags
  • Implement _filter_system_by_workloads to accumulate filter expressions for present workload params and apply a combined OR condition
  • Update filter_types enum to use SYSTEM_WORKLOADS instead of distinct SYSTEM_SAP, SYSTEM_AAP, and SYSTEM_MSSQL filter variants
  • Wire SYSTEM_WORKLOADS into system, CVE, vulnerabilities, dashboard, and dashbar handlers in place of the removed workload-specific filters
manager/filters.py
manager/system_handler.py
manager/cve_handler.py
manager/vulnerabilities_handler.py
manager/dashboard_handler.py
manager/dashbar_handler.py
Extend system platform schema and ingestion to store new workload flags from Kafka inventory messages.
  • Add crowdstrike, ibm_db2, intersystems, oracle_db, and rhel_ai boolean fields to SystemPlatform peewee model and PostgreSQL schema
  • Create migration script to add new workload boolean columns to the system_platform table
  • Update dev seed data to set appropriate workload flag combinations for selected inventory_ids
  • Modify inventory_processor to extract workload flags from system_profile.workloads and persist them on insert and update
common/peewee_model.py
database/schema/ve_db_postgresql.sql
database/schema/upgrade_scripts/166-add-workload-columns.sql
database/schema/ve_db_dev_data.sql
listener/inventory_processor.py
Expose new workload filters in the public API contracts and request argument parsing across various endpoints.
  • Add crowdstrike, ibm_db2, intersystems, oracle_db, and rhel_ai query parameters to /systems, /cves/{id}/affected_systems, /vulnerabilities/cves, /dashboard, and /dashbar paths in the OpenAPI spec
  • Define component parameter schemas (type boolean, descriptions, examples) for each new workload type
  • Extend argument parsing descriptors in handlers to accept the new workload query parameters
  • Update cacheability logic to consider the new workload params as cache-busting filters
manager.spec.yaml
manager/system_handler.py
manager/cve_handler.py
manager/vulnerabilities_handler.py
manager/dashboard_handler.py
manager/dashbar_handler.py
manager/base.py
Add tests and schema expectations for new workload filters and OR logic behavior.
  • Extend request schema expectations to include new workload boolean parameters in multiple test schemas
  • Seed dev data with workload flags tied to specific inventory IDs used in tests
  • Add tests verifying positive/negative filtering for each new workload type on /systems
  • Add tests ensuring OR behavior when multiple workload filters are combined, including across legacy and new workload types
  • Expand CVE filter check tests to exercise new workload query params
tests/manager_tests/schemas.py
tests/manager_tests/test_system_handler.py
tests/manager_tests/test_cve_handler.py
database/schema/ve_db_dev_data.sql
Minor documentation and comment clean-up in filters module.
  • Fix repeated typo in filter function docstrings from 'arguemnts' to 'arguments'
manager/filters.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues, and left some high level feedback:

  • The new OR-based _filter_system_by_workloads changes semantics for combinations of true and false workload flags (e.g. ansible=false&mssql=false now matches ~ansible OR ~mssql instead of ~ansible AND ~mssql); consider confirming this matches intended API behavior or restricting OR logic to true selections only.
  • The repeated fn.COALESCE(field, False) patterns in WORKLOAD_PARAMS could be centralized into a small helper to reduce duplication and make future workload additions less error-prone.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The new OR-based `_filter_system_by_workloads` changes semantics for combinations of `true` and `false` workload flags (e.g. `ansible=false&mssql=false` now matches `~ansible OR ~mssql` instead of `~ansible AND ~mssql`); consider confirming this matches intended API behavior or restricting OR logic to `true` selections only.
- The repeated `fn.COALESCE(field, False)` patterns in `WORKLOAD_PARAMS` could be centralized into a small helper to reduce duplication and make future workload additions less error-prone.

## Individual Comments

### Comment 1
<location path="manager/filters.py" line_range="450-456" />
<code_context>
-        query = query.where(expr)
+    exprs = []
+    for param, expr_fns in WORKLOAD_PARAMS.items():
+        if param in args and args[param] is not None:
+            if args[param]:
+                exprs.append(expr_fns["positive"]())
+            else:
+                exprs.append(expr_fns["negative"]())
+    if exprs:
+        query = query.where(reduce(or_, exprs))
     return query

</code_context>
<issue_to_address>
**issue (bug_risk):** Using OR semantics for negative workload flags (`False` values) leads to broader-than-expected result sets compared to the previous implementation.

Previously, negative workload flags were combined with AND semantics: e.g. `ansible=False` and `mssql=False` yielded `~ansible AND ~mssql` (systems that were neither ansible- nor mssql-managed). With the new `reduce(or_, exprs)` approach, these become `~ansible OR ~mssql`, which also matches systems that are only ansible-managed or only mssql-managed. This changes the exclusion behavior and may surprise callers. If OR is desired for positive flags, consider handling negative flags differently (e.g., combine them with AND or restrict to a single negative flag) to avoid overly broad matches.
</issue_to_address>

### Comment 2
<location path="tests/manager_tests/test_system_handler.py" line_range="627-625" />
<code_context>
+        expected_ids = set(ANSIBLE_SYSTEMS) | set(MSSQL_SYSTEMS)
+        assert returned_ids == expected_ids
+
+    def test_system_workload_or_logic_new_types(self):
+        """Test OR logic across new workload types"""
+        response = self.vfetch("systems?crowdstrike=true&ibm_db2=true").check_response()
+        returned_ids = {s.attributes.inventory_id for s in response.body.data}
+        expected_ids = set(CROWDSTRIKE_SYSTEMS) | set(IBM_DB2_SYSTEMS)
+        assert returned_ids == expected_ids
+
     def test_system_cves_known_exploit(self):
</code_context>
<issue_to_address>
**suggestion (testing):** Add tests for mixed true/false workload flags to validate correct OR behavior when some flags are false

Current tests only cover cases where all workload flags are `true`. Please add scenarios with mixed `true`/`false` flags (e.g. `systems?crowdstrike=true&ibm_db2=false`, `systems?ansible=false&mssql=true`) to verify that only active (`true`) filters participate in the OR and that `false` flags are enforced as exclusion conditions rather than incorrectly included in the OR expression.

Suggested implementation:

```python
        for system in response.body.data:
            assert system.attributes.inventory_id not in RHEL_AI_SYSTEMS

    def test_system_workload_or_logic(self):
        """Test that multiple workload filters use OR logic"""
        response = self.vfetch("systems?ansible=true&mssql=true").check_response()
        returned_ids = {s.attributes.inventory_id for s in response.body.data}
        expected_ids = set(ANSIBLE_SYSTEMS) | set(MSSQL_SYSTEMS)
        assert returned_ids == expected_ids

    def test_system_workload_or_logic_new_types(self):
        """Test OR logic across new workload types"""
        response = self.vfetch("systems?crowdstrike=true&ibm_db2=true").check_response()
        returned_ids = {s.attributes.inventory_id for s in response.body.data}
        expected_ids = set(CROWDSTRIKE_SYSTEMS) | set(IBM_DB2_SYSTEMS)
        assert returned_ids == expected_ids

    def test_system_workload_or_logic_mixed_flags(self):
        """Mixed true/false flags: only true workloads participate in OR, false workloads are excluded"""
        # ansible=false&mssql=true => only MSSQL systems returned, no Ansible systems
        response = self.vfetch("systems?ansible=false&mssql=true").check_response()
        returned_ids = {s.attributes.inventory_id for s in response.body.data}

        expected_ids = set(MSSQL_SYSTEMS)
        excluded_ids = set(ANSIBLE_SYSTEMS)

        assert returned_ids == expected_ids
        assert returned_ids.isdisjoint(excluded_ids)

    def test_system_workload_or_logic_mixed_flags_new_types(self):
        """Mixed true/false flags across new workload types"""
        # crowdstrike=true&ibm_db2=false => only CrowdStrike systems returned, no IBM DB2 systems
        response = self.vfetch("systems?crowdstrike=true&ibm_db2=false").check_response()
        returned_ids = {s.attributes.inventory_id for s in response.body.data}

        expected_ids = set(CROWDSTRIKE_SYSTEMS)
        excluded_ids = set(IBM_DB2_SYSTEMS)

        assert returned_ids == expected_ids
        assert returned_ids.isdisjoint(excluded_ids)

    def test_system_cves_known_exploit(self):
        def count_known_exploits(data):
            cnt = 0

```

These tests assume that `ANSIBLE_SYSTEMS`, `MSSQL_SYSTEMS`, `CROWDSTRIKE_SYSTEMS`, and `IBM_DB2_SYSTEMS` are already defined in this test module and that the API correctly interprets `workload=false` as an exclusion filter. If the underlying handler supports additional workload flags, you may want to mirror these patterns to cover those as well. Also ensure that the test class name and any setup fixtures (e.g. `vfetch`, system populations) are consistent with the rest of the file so the new tests run in the same context as the existing workload tests.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread manager/filters.py
Comment thread tests/manager_tests/test_system_handler.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant