Skip to content

Commit f9ada3f

Browse files
committed
[rules score] fix tests
- execute Rules Score Tests from Test Module in CI - Fix Test Module dependencies - Fix Failing Tests
1 parent c310490 commit f9ada3f

7 files changed

Lines changed: 205 additions & 36 deletions

File tree

.bazelignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,5 @@
1313

1414
python_basics/integration_tests
1515
starpls/integration_tests
16+
# Should be run from its own directory (separate Module)
17+
bazel/rules/rules_score/test

.github/copilot-instructions.md

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
<!--
2+
Copyright (c) 2026 Contributors to the Eclipse Foundation
3+
4+
SPDX-License-Identifier: Apache-2.0
5+
-->
6+
7+
# Score Tooling — Agent Instructions
8+
9+
**Score Tooling** is a unified Bazel module (`score_tooling`) providing development tools for building, testing, and maintaining code quality. All public macros are re-exported from [defs.bzl](../defs.bzl).
10+
11+
## Modules
12+
13+
| Module | Language | Purpose | Docs |
14+
|--------|----------|---------|------|
15+
| `python_basics` | Starlark/Python | `score_py_pytest` + `score_virtualenv` macros | [README](../python_basics/README.md) |
16+
| `cr_checker` | Python | Copyright header checker/fixer | [README](../cr_checker/README.md) |
17+
| `dash` | Tool | DASH license compliance | [README](../dash/README.md) |
18+
| `format_checker` | Starlark | `use_format_targets()``:format.fix` / `:format.check` | [README](../format_checker/README.md) |
19+
| `cli_helper` | Starlark | CLI helper Bazel rule | [README](../cli_helper/README.md) |
20+
| `starpls` | Config | Starlark language server | [README](../starpls/README.md) |
21+
| `coverage` | Shell+Bazel | Combined Rust+Python coverage → HTML | [README](../coverage/README.md) |
22+
| `lobster_bazel` | Python | Extracts traceability tags for LOBSTER | [README](../lobster_bazel/README.md) |
23+
| `manual_analysis` | Python+Starlark | Interactive manual verification workflow | [README](../manual_analysis/README.md) |
24+
| `plantuml` | Rust | PlantUML FTA-metamodel parser | [README](../plantuml/parser/README.md) |
25+
| `validation` | Rust+Python | Bazel-graph vs. PlantUML validation + AI checker | [README](../validation/ai_checker/README.md) |
26+
| `tools` | Config | Multitool lockfile: ruff, shellcheck, actionlint, yamlfmt | [README](../tools/README.md) |
27+
28+
## Build & Test Commands
29+
30+
```bash
31+
bazel test //... # all tests
32+
bazel test //:format.check # check formatting (ruff, buildifier, rustfmt, yamlfmt)
33+
bazel run //:format.fix # auto-fix formatting
34+
bazel run //:copyright.check # check copyright headers
35+
36+
# Rust-specific
37+
bazel build //plantuml/... --config=clippy
38+
bazel build //validation/... --config=clippy
39+
40+
# Coverage
41+
bazel run //coverage:combined_report
42+
43+
# IDE virtualenv
44+
bazel run //:ide_support # creates .venv
45+
46+
# AI checker (runs at test time; inherits Copilot credentials from your shell)
47+
bazel test //path/to:my_ai_check
48+
```
49+
50+
**Integration tests** have their own `MODULE.bazel` — run from their subdirectory:
51+
52+
```bash
53+
cd python_basics/integration_tests && bazel test //...
54+
cd starpls/integration_tests && bazel test //...
55+
```
56+
57+
## Critical Conventions
58+
59+
### Copyright header — mandatory on every file
60+
Every `.py`, `.bzl`, `.rs`, `.sh`, `.cpp` file must begin with:
61+
```
62+
# Copyright (c) <year> Contributors to the Eclipse Foundation
63+
#
64+
# SPDX-License-Identifier: Apache-2.0
65+
```
66+
The `:copyright.check` target enforces this and will fail CI if missing.
67+
68+
### Python
69+
- Version: **3.12** (pinned in `MODULE.bazel` + `python_basics/pyproject.toml`)
70+
- Linter: **ruff** (`E, F, I, B, C90, UP, SIM, RET` rules)
71+
- Type checker: **basedpyright** (`standard` mode)
72+
- All Python tests must use `score_py_pytest`, not raw `py_test`
73+
- `--incompatible_default_to_explicit_init_py` is active — no automatic `__init__.py`
74+
75+
### Bazel/Starlark
76+
- Formatting: **buildifier**
77+
- Bzlmod (`MODULE.bazel`) is primary; `deps.bzl` is legacy compat only
78+
- Root `BUILD` registers `:copyright.check`, `:format.check`, `:format.fix` targets
79+
80+
### Rust
81+
- Edition 2021; rustfmt + clippy via `--config=clippy`
82+
83+
## Dependency / Requirements Management
84+
85+
Each sub-package has its own pip hub with `requirements.in``requirements.txt`:
86+
87+
| Hub | Lock file |
88+
|-----|-----------|
89+
| `pip_tooling` | `python_basics/requirements.txt` |
90+
| `pip_lobster_bazel` | `lobster_bazel/requirements.txt` |
91+
| `manual_analysis_deps` | `manual_analysis/requirements.txt` |
92+
| `pip_validation_ai_checker` | `validation/ai_checker/requirements.txt` |
93+
94+
After editing `requirements.in`, regenerate with:
95+
```bash
96+
bazel run //<pkg>:requirements.update
97+
```
98+
Both `requirements.in` and `requirements.txt` are committed to VCS.
99+
100+
## Common Pitfalls
101+
102+
- **AI checker tests need `tags = ["manual"]`** — otherwise `bazel test //...` attempts to run them without Copilot credentials.
103+
- **`lobster_bazel` traceability tags** must be at the start of a line (not inline).
104+
- **Format check fails silently** if buildifier or ruff versions are out of sync with the multitool lockfile in `tools/`.
105+
- **`manual_analysis` lock files are committed** — always run `{target}.update` after changing analysis YAML, then commit the updated lock.

.github/workflows/tests.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,8 @@ jobs:
4141
bazel test //coverage/tests:all
4242
- name: Run rules_score tests
4343
run: |
44-
bazel test //bazel/rules/rules_score/...
44+
cd bazel/rules/rules_score/test
45+
bazel test //...
4546
- name: Run Plantuml Tooling clippy
4647
run: |
4748
bazel build //plantuml/... --config=clippy

bazel/rules/rules_score/test/BUILD

Lines changed: 45 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
load("@aspect_rules_py//py:defs.bzl", "py_binary")
2+
13
# *******************************************************************************
24
# Copyright (c) 2025 Contributors to the Eclipse Foundation
35
#
@@ -26,6 +28,7 @@ load(
2628
"unit",
2729
"unit_design",
2830
)
31+
load("@score_tooling//bazel/rules/rules_score:sphinx_toolchain.bzl", "sphinx_toolchain")
2932
load("@trlc//:trlc.bzl", "trlc_requirements", "trlc_requirements_test")
3033
load(
3134
":html_generation_test.bzl",
@@ -93,6 +96,38 @@ load(
9396

9497
package(default_visibility = ["//visibility:public"])
9598

99+
# ============================================================================
100+
# Sphinx Toolchain — uses score_docs_as_code so score_metamodel is importable
101+
# ============================================================================
102+
103+
exports_files(["template/conf.template.py"])
104+
105+
py_binary(
106+
name = "score_build",
107+
srcs = ["@score_tooling//bazel/rules/rules_score:src/sphinx_wrapper.py"],
108+
main = "@score_tooling//bazel/rules/rules_score:src/sphinx_wrapper.py",
109+
package_collisions = "warning",
110+
visibility = ["//visibility:public"],
111+
deps = [
112+
"@score_docs_as_code//src:plantuml_for_python",
113+
"@score_docs_as_code//src/extensions/score_sphinx_bundle",
114+
"@score_tooling//bazel/rules/rules_score:bazel_sphinx_needs",
115+
"@score_tooling//bazel/rules/rules_score:sphinx_module_ext",
116+
],
117+
)
118+
119+
sphinx_toolchain(
120+
name = "score_sphinx_toolchain",
121+
conf_template = "//:template/conf.template.py",
122+
sphinx = ":score_build",
123+
)
124+
125+
toolchain(
126+
name = "score_toolchain",
127+
toolchain = ":score_sphinx_toolchain",
128+
toolchain_type = "@score_tooling//bazel/rules/rules_score:toolchain_type",
129+
)
130+
96131
# ============================================================================
97132
# Test Fixtures - Module Definitions
98133
# ============================================================================
@@ -274,7 +309,7 @@ unit_design(
274309
unit(
275310
name = "test_unit",
276311
testonly = True,
277-
scope = ["//bazel/rules/rules_score/test:mock_lib1"],
312+
scope = ["//:mock_lib1"],
278313
tests = [":test_unit_tests"],
279314
unit_design = [":test_unit_design"],
280315
implementation = [
@@ -700,26 +735,11 @@ aous_rst_sphinx_test(
700735
requirements_rst_test_suite(name = "requirements_rst_tests")
701736

702737
# ============================================================================
703-
# Image srcs Tests
738+
# Image srcs Tests (defined in sub-package to avoid workspace-root edge case)
704739
# ============================================================================
705-
706-
# Fixture: feature_requirements with image_srcs
707-
feature_requirements(
708-
name = "feat_req_with_image",
709-
srcs = ["fixtures/seooc_test/feature_requirements.trlc"],
710-
image_srcs = [
711-
"fixtures/image_srcs/diagrams/arch.svg",
712-
"fixtures/image_srcs/diagrams/arch.png",
713-
],
714-
deps = [":asr_trlc"],
715-
)
716-
717-
image_srcs_sphinx_sources_test(
718-
name = "image_srcs_sphinx_sources_test",
719-
target_under_test = ":feat_req_with_image",
720-
)
721-
722-
requirements_image_test_suite(name = "requirements_image_tests")
740+
# feat_req_with_image and image_srcs_sphinx_sources_test live in
741+
# //fixtures/image_srcs so that ctx.label.package is non-empty and
742+
# subrule_trlc_image_stage can strip the prefix correctly.
723743

724744
# ============================================================================
725745
# Combined Test Suite
@@ -736,7 +756,7 @@ requirements_image_test_suite(name = "requirements_image_tests")
736756
trlc_requirements(
737757
name = "safety_measures_fixtures",
738758
srcs = ["fixtures/seooc_test/safety_measures_fixtures.trlc"],
739-
spec = ["//bazel/rules/rules_score/trlc/config:score_requirements_model"],
759+
spec = ["@score_tooling//bazel/rules/rules_score/trlc/config:score_requirements_model"],
740760
)
741761

742762
trlc_requirements_test(
@@ -748,22 +768,22 @@ py_test(
748768
name = "test_safety_analysis_tools",
749769
size = "small",
750770
srcs = ["test_safety_analysis_tools.py"],
751-
deps = ["//bazel/rules/rules_score:safety_analysis_tools"],
771+
deps = ["@score_tooling//bazel/rules/rules_score:safety_analysis_tools"],
752772
)
753773

754774
py_test(
755775
name = "test_aou_forwarding_to_lobster",
756776
size = "small",
757777
srcs = ["test_aou_forwarding_to_lobster.py"],
758-
deps = ["//bazel/rules/rules_score:aou_forwarding_to_lobster"],
778+
deps = ["@score_tooling//bazel/rules/rules_score:aou_forwarding_to_lobster"],
759779
)
760780

761781
py_test(
762782
name = "test_rst_to_trlc",
763783
size = "small",
764784
srcs = ["rst_to_trlc_test.py"],
765785
main = "rst_to_trlc_test.py",
766-
deps = ["//bazel/rules/rules_score:rst_to_trlc_lib"],
786+
deps = ["@score_tooling//bazel/rules/rules_score:rst_to_trlc_lib"],
767787
)
768788

769789
py_test(
@@ -782,7 +802,6 @@ py_test(
782802
test_suite(
783803
name = "all_tests",
784804
tests = [
785-
":requirements_image_tests",
786805
":requirements_rst_tests",
787806
":seooc_tests",
788807
":sphinx_module_tests",
@@ -791,5 +810,6 @@ test_suite(
791810
":test_safety_analysis_tools",
792811
":test_trlc_rst_image_rendering",
793812
":unit_component_tests",
813+
"//fixtures/image_srcs:requirements_image_tests",
794814
],
795815
)

bazel/rules/rules_score/test/MODULE.bazel

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,8 @@ pip.parse(
5555
use_repo(pip, "pip_tooling_test")
5656

5757
bazel_dep(name = "score_docs_as_code", version = "3.0.1", dev_dependency = True)
58+
59+
bazel_dep(name = "googletest", version = "1.17.0.bcr.2")
5860
# bazel_dep(name = "score_platform", version = "0.5.0")
5961
# bazel_dep(name = "score_process", version = "1.3.2")
6062

@@ -71,13 +73,13 @@ register_toolchains(
7173
bazel_dep(name = "trlc", version = "0.0.0")
7274
git_override(
7375
module_name = "trlc",
74-
commit = "7f06d0396fd0e5f02b657d25700775af5113b7e0",
76+
commit = "c4c531b9d667085daa09dfc1590edacc314bfda4",
7577
remote = "https://github.com/bmw-software-engineering/trlc.git",
7678
)
7779

7880
bazel_dep(name = "lobster", version = "0.0.0")
7981
git_override(
8082
module_name = "lobster",
81-
commit = "56881461f9d3fde2918d1731aa5937aaf64cd67c",
83+
commit = "2792e2daee2cf524fdc7b1545fd3537791ebc36c",
8284
remote = "https://github.com/bmw-software-engineering/lobster.git",
8385
)
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# *******************************************************************************
2+
# Copyright (c) 2026 Contributors to the Eclipse Foundation
3+
#
4+
# See the NOTICE file(s) distributed with this work for additional
5+
# information regarding copyright ownership.
6+
#
7+
# This program and the accompanying materials are made available under the
8+
# terms of the Apache License Version 2.0 which is available at
9+
# https://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# SPDX-License-Identifier: Apache-2.0
12+
# *******************************************************************************
13+
14+
# This sub-package exists so that ctx.label.package == "fixtures/image_srcs"
15+
# when feature_requirements stages image files. subrule_trlc_image_stage
16+
# strips package_prefix from short_path to produce package-relative paths
17+
# (e.g. "diagrams/arch.png"); this only works when the package is not the
18+
# workspace root ("").
19+
20+
load(
21+
"@score_tooling//bazel/rules/rules_score:rules_score.bzl",
22+
"feature_requirements",
23+
)
24+
load(
25+
"//:requirements_image_test.bzl",
26+
"image_srcs_sphinx_sources_test",
27+
"requirements_image_test_suite",
28+
)
29+
30+
package(default_visibility = ["//visibility:public"])
31+
32+
feature_requirements(
33+
name = "feat_req_with_image",
34+
srcs = ["//:fixtures/seooc_test/feature_requirements.trlc"],
35+
image_srcs = [
36+
"diagrams/arch.svg",
37+
"diagrams/arch.png",
38+
],
39+
deps = ["//:asr_trlc"],
40+
)
41+
42+
image_srcs_sphinx_sources_test(
43+
name = "image_srcs_sphinx_sources_test",
44+
target_under_test = ":feat_req_with_image",
45+
)
46+
47+
requirements_image_test_suite(name = "requirements_image_tests")

bazel/rules/rules_score/test/trlc_rst_image_rendering_test.py

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,19 +48,11 @@ def setUp(self):
4848
self._tmpdir = tempfile.mkdtemp()
4949
self._out = os.path.join(self._tmpdir, "output.rst")
5050
self._fixture_trlc = _runfile(
51-
"bazel",
52-
"rules",
53-
"rules_score",
54-
"test",
5551
"fixtures",
5652
"image_requirements",
5753
"image_requirements.trlc",
5854
)
5955
self._fixture_rsl = _runfile(
60-
"bazel",
61-
"rules",
62-
"rules_score",
63-
"test",
6456
"fixtures",
6557
"image_requirements",
6658
"schema.rsl",

0 commit comments

Comments
 (0)