Skip to content

verification report need type#660

Open
FScholPer wants to merge 1 commit into
mainfrom
verification_needs
Open

verification report need type#660
FScholPer wants to merge 1 commit into
mainfrom
verification_needs

Conversation

@FScholPer

Copy link
Copy Markdown
Contributor

Relates to #611

Split out of #612 (verification part). The inspection-record part is in a separate PR.

📌 Description

Adds a machine-readable module verification report need type to the metamodel.

  • metamodel.yaml: new mod_ver_report need type (prefix: mod_vrep__) classifying a report by safety, security, status and verification_method, with optional coverage/verdict/version attributes; links belongs_to a mod and optionally contains/covers/evidence/realizes. Adds the evidence extra link (evidence / evidence_for) it depends on.
  • requirements.rst: new "Verification Evidence" section with tool_req__docs_verification_report_need, plus the Verif row in the requirements overview table.
  • Tests: rst/options/test_options_verification_report.rst covering a valid report and an invalid verdict value.

🚨 Impact Analysis

  • This change does not violate any tool requirements and is covered by existing tool requirements
  • This change does not violate any design decisions
  • Otherwise I have created a ticket for new tool qualification

✅ Checklist

  • Added/updated documentation for new or changed features
  • Added/updated tests to cover the changes
  • Followed project coding standards and guidelines

Frank Scholter Peres frank.scholter_peres@mercedes-benz.com, Mercedes-Benz Tech Innovation GmbH
Provider Information

Adds the mod_ver_report need type to the metamodel with an evidence extra
link, documents the tool requirement, and adds option tests.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run --lockfile_mode=error //src:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.6.0) and connecting to it...
INFO: Invocation ID: 5adfd3b9-0347-4163-864a-c94fe1c3022a
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: src
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=ppc64le-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=riscv64-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=s390x-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=x86_64-unknown-linux-musl. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
Loading: 0 packages loaded
    currently loading: src
Loading: 0 packages loaded
    currently loading: src
WARNING: Target pattern parsing failed.
ERROR: Skipping '//src:license-check': no such target '//src:license-check': target 'license-check' not declared in package 'src' defined by /home/runner/work/docs-as-code/docs-as-code/src/BUILD
ERROR: no such target '//src:license-check': target 'license-check' not declared in package 'src' defined by /home/runner/work/docs-as-code/docs-as-code/src/BUILD
INFO: Elapsed time: 7.747s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

@github-actions

Copy link
Copy Markdown

The created documentation from the pull request is available at: docu-html

@FScholPer FScholPer changed the title Add machine-readable module verification report need type verification report need type Jul 23, 2026

@aschemmel-tech aschemmel-tech left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some remarks, not crucial for the PoC

# req-Id: tool_req__docs_verification_report_need
verification_method: ^.*$
optional_options:
requirements_coverage_percent: ^(100|[1-9]?[0-9])$

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not clear what can be seen here. maybe call it "requirements_tests_completely_passed_percent"

verification_method: ^.*$
optional_options:
requirements_coverage_percent: ^(100|[1-9]?[0-9])$
structural_coverage_percent: ^(100|[1-9]?[0-9])$

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

branch coverage is also structural coverage, so maybe call "line/block_coverage_percent" (depending what our selected tool supports)

requirements_coverage_percent: ^(100|[1-9]?[0-9])$
structural_coverage_percent: ^(100|[1-9]?[0-9])$
branch_coverage_percent: ^(100|[1-9]?[0-9])$
verdict: ^(pass|fail|open)$

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

giving a verdict is not part of the process, who would give it? based on what ...? I think we had this more in the release note.

structural_coverage_percent: ^(100|[1-9]?[0-9])$
branch_coverage_percent: ^(100|[1-9]?[0-9])$
verdict: ^(pass|fail|open)$
report_version: ^.*$

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

every needs element has a version attribute. What is special about this?

branch_coverage_percent: ^(100|[1-9]?[0-9])$
verdict: ^(pass|fail|open)$
report_version: ^.*$
release_baseline: ^.*$

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still prefer: "The verification report is valid for the module version tagged together with the report" as written in the template.

verification_method: ^.*$
optional_options:
requirements_coverage_percent: ^(100|[1-9]?[0-9])$
structural_coverage_percent: ^(100|[1-9]?[0-9])$

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An overall value I do not think helpful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants