Skip to content

Commit dcd4447

Browse files
Ibraheem OlasupoIbraheem Olasupo
Ibraheem Olasupo
authored and
Ibraheem Olasupo
committed
Expose final Schemathesis CI diagnostics
1 parent c1f48a4 commit dcd4447

61 files changed

Lines changed: 717 additions & 263 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/api-security.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,24 @@ jobs:
4141
- name: Schemathesis local API validation
4242
run: make schemathesis-test
4343

44+
- name: Print dynamic diagnostics on failure
45+
if: failure()
46+
run: make dynamic-diagnostics
47+
48+
- name: Upload dynamic diagnostics
49+
if: always()
50+
# actions/upload-artifact v4.6.2
51+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
52+
with:
53+
name: dynamic-security-diagnostics
54+
path: |
55+
outputs/security/dynamic/raw/
56+
outputs/security/dynamic/*.json
57+
reports/security/dynamic-security-report.md
58+
reports/security/zap-report.md
59+
if-no-files-found: warn
60+
retention-days: 14
61+
4462
- name: OWASP ZAP local baseline
4563
run: make zap-baseline
4664

‎Makefile‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.PHONY: help setup install format format-check lint type-check test test-coverage auth-test api-security-test terraform-fmt terraform-fmt-check terraform-init terraform-validate terraform-test infrastructure-test infrastructure-evidence verify-infrastructure-evidence infrastructure-report security-tools secrets-scan sast sast-semgrep sast-bandit semgrep-test sca dependency-audit sbom verify-sbom iac-scan checkov-scan container-build-security container-scan appsec-fast appsec-full appsec-evidence verify-appsec-evidence appsec-report dynamic-tools dynamic-server-start dynamic-server-wait dynamic-server-stop schemathesis-test api-schema-security-test zap-baseline zap-api-scan auth-boundary-test authorisation-boundary-test object-access-test input-mutation-test security-header-test cors-test resource-consumption-test audit-dynamic-test dast dynamic-evidence verify-dynamic-evidence dynamic-report dynamic-pytest dynamic-fast dynamic-full findings-normalise findings-deduplicate findings-enrich findings-validate findings-evidence verify-findings-evidence findings-report findings-full release-policy-validate release-gate-evaluate release-gate-enforce release-evidence verify-release-evidence release-report release-full lifecycle-policy-validate lifecycle-initialise lifecycle-validate lifecycle-expiry lifecycle-evidence verify-lifecycle-evidence lifecycle-report lifecycle-full evidence-source-validate evidence-aggregate evidence-generate verify-consolidated-evidence evidence-report evidence-full integration-policy-validate integration-export integration-validate verify-integration-evidence integration-report integration-full portfolio-evidence verify-portfolio-evidence portfolio-report portfolio-full final-validation security-assurance-full security-doctor developer-docs-validate developer-enablement-evidence verify-developer-enablement-evidence developer-enablement-report developer-enablement-full champions-policy-validate champions-metrics champions-evidence verify-champions-evidence champions-report champions-full pre-commit-install pre-commit-run quality run docker-build docker-run threat-model-validate threat-model-evidence verify-threat-model-evidence threat-model-report api-security-evidence verify-api-security-evidence api-security-report dev-token-researcher dev-token-approver dev-token-auditor clean
1+
.PHONY: help setup install format format-check lint type-check test test-coverage auth-test api-security-test terraform-fmt terraform-fmt-check terraform-init terraform-validate terraform-test infrastructure-test infrastructure-evidence verify-infrastructure-evidence infrastructure-report security-tools secrets-scan sast sast-semgrep sast-bandit semgrep-test sca dependency-audit sbom verify-sbom iac-scan checkov-scan container-build-security container-scan appsec-fast appsec-full appsec-evidence verify-appsec-evidence appsec-report dynamic-tools dynamic-server-start dynamic-server-wait dynamic-server-stop dynamic-diagnostics schemathesis-test api-schema-security-test zap-baseline zap-api-scan auth-boundary-test authorisation-boundary-test object-access-test input-mutation-test security-header-test cors-test resource-consumption-test audit-dynamic-test dast dynamic-evidence verify-dynamic-evidence dynamic-report dynamic-pytest dynamic-fast dynamic-full findings-normalise findings-deduplicate findings-enrich findings-validate findings-evidence verify-findings-evidence findings-report findings-full release-policy-validate release-gate-evaluate release-gate-enforce release-evidence verify-release-evidence release-report release-full lifecycle-policy-validate lifecycle-initialise lifecycle-validate lifecycle-expiry lifecycle-evidence verify-lifecycle-evidence lifecycle-report lifecycle-full evidence-source-validate evidence-aggregate evidence-generate verify-consolidated-evidence evidence-report evidence-full integration-policy-validate integration-export integration-validate verify-integration-evidence integration-report integration-full portfolio-evidence verify-portfolio-evidence portfolio-report portfolio-full final-validation security-assurance-full security-doctor developer-docs-validate developer-enablement-evidence verify-developer-enablement-evidence developer-enablement-report developer-enablement-full champions-policy-validate champions-metrics champions-evidence verify-champions-evidence champions-report champions-full pre-commit-install pre-commit-run quality run docker-build docker-run threat-model-validate threat-model-evidence verify-threat-model-evidence threat-model-report api-security-evidence verify-api-security-evidence api-security-report dev-token-researcher dev-token-approver dev-token-auditor clean
22

33
PYTHON ?= python3
44
VENV ?= .venv
@@ -137,6 +137,9 @@ dynamic-server-wait:
137137
dynamic-server-stop:
138138
PYTHONPATH=src $(PYTHON) scripts/dynamic_security_tools.py server-stop
139139

140+
dynamic-diagnostics:
141+
PYTHONPATH=src $(PYTHON) scripts/dynamic_security_tools.py diagnostics
142+
140143
schemathesis-test:
141144
PYTHONPATH=src $(PYTHON) scripts/dynamic_security_tools.py schemathesis
142145

‎outputs/portfolio/README 2.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Portfolio Package
2+
3+
Deterministic portfolio summary package generated from committed security evidence. No deployment credentials, raw scanner secrets, or external service outputs are included.

‎outputs/portfolio/evidence-index 2.json‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
{
2+
"index_type": "evidence",
3+
"items": [
4+
{
5+
"path": "outputs/security/threat-model/evidence-manifest.json",
6+
"sha256": "b2d3277fd3d5a10c27929ada31c5855b084993eab09fd06d3f8f44a4d3939082",
7+
"size_bytes": 3465,
8+
"type": "evidence"
9+
},
10+
{
11+
"path": "outputs/security/api-security/evidence-manifest.json",
12+
"sha256": "9220e28a605114c19f2838e8a503815ca13a68992446125f35b2287278171474",
13+
"size_bytes": 2510,
14+
"type": "evidence"
15+
},
16+
{
17+
"path": "outputs/security/infrastructure/evidence-manifest.json",
18+
"sha256": "8eea5e1dd1bc011c8c3d6e7f65b43c24c67b53d602f671427bbc06f947e03846",
19+
"size_bytes": 9754,
20+
"type": "evidence"
21+
},
22+
{
23+
"path": "outputs/security/appsec/evidence-manifest.json",
24+
"sha256": "79d0c867bd242c8a5645960b8835f429c4e83798cd926849a3e74897c7bb5be8",
25+
"size_bytes": 3564,
26+
"type": "evidence"
27+
},
28+
{
29+
"path": "outputs/security/dynamic/evidence-manifest.json",
30+
"sha256": "db68f7706fb686aa8e42f39619848ecb015f3dc580394f575bb6cfc2d667148c",
31+
"size_bytes": 2099,
32+
"type": "evidence"
33+
},
34+
{
35+
"path": "outputs/security/findings/evidence-manifest.json",
36+
"sha256": "6d4219d7e2705c03cd10ce801983317e760c015306ab6e8bced66181f666e3a3",
37+
"size_bytes": 7093,
38+
"type": "evidence"
39+
},
40+
{
41+
"path": "outputs/security/release/evidence-manifest.json",
42+
"sha256": "f93f8fedd774163a080c89f18c1f83a0851b460bd525f26bc3e10d3bff3a1959",
43+
"size_bytes": 3296,
44+
"type": "evidence"
45+
},
46+
{
47+
"path": "outputs/security/lifecycle/evidence-manifest.json",
48+
"sha256": "21ab61da61298d25fdfed0020cc743305a972767ea5c259c74e8bde205036b6d",
49+
"size_bytes": 5829,
50+
"type": "evidence"
51+
},
52+
{
53+
"path": "outputs/security/evidence/evidence-manifest.json",
54+
"sha256": "bc178b3c9844284039c20c3b7b49d1f8653288c7192d189f91a1d1f902f323a9",
55+
"size_bytes": 6303,
56+
"type": "evidence"
57+
},
58+
{
59+
"path": "outputs/security/champions/evidence-manifest.json",
60+
"sha256": "9eb28068ad08dceb8a56598d8038ddd10614e826a63b37a3b754d7365a1c5fad",
61+
"size_bytes": 1762,
62+
"type": "evidence"
63+
},
64+
{
65+
"path": "outputs/security/integration/integration-manifest.json",
66+
"sha256": "3da478995e4dc7452de8dce8cf9aa232d283ac7208acf4281c20eef6e12c60c0",
67+
"size_bytes": 4620,
68+
"type": "evidence"
69+
}
70+
],
71+
"missing_paths": [],
72+
"schema_version": "1.0",
73+
"valid": true
74+
}

‎outputs/portfolio/evidence-index.json‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,25 +33,25 @@
3333
},
3434
{
3535
"path": "outputs/security/findings/evidence-manifest.json",
36-
"sha256": "221acd33f02653815ff261e9cc7fdfc650b65b916790da890989a7b11e69d6f9",
36+
"sha256": "6d4219d7e2705c03cd10ce801983317e760c015306ab6e8bced66181f666e3a3",
3737
"size_bytes": 7093,
3838
"type": "evidence"
3939
},
4040
{
4141
"path": "outputs/security/release/evidence-manifest.json",
42-
"sha256": "5cc5173f3437afce86571e502dedb82814d3cc347ab5b63123ab7d40da0dde39",
42+
"sha256": "f93f8fedd774163a080c89f18c1f83a0851b460bd525f26bc3e10d3bff3a1959",
4343
"size_bytes": 3296,
4444
"type": "evidence"
4545
},
4646
{
4747
"path": "outputs/security/lifecycle/evidence-manifest.json",
48-
"sha256": "72073c1ca9259c58c7b7c70dcaaa629141d8c542a2f430d4df7b5fb92e53b6ef",
48+
"sha256": "a7b842b3292c1a4078bb7c52c6fb5881bd7df3d52799b5234015b48578ffc0bd",
4949
"size_bytes": 5829,
5050
"type": "evidence"
5151
},
5252
{
5353
"path": "outputs/security/evidence/evidence-manifest.json",
54-
"sha256": "3e26d6836bf2c4efc067ada9ea52ec28b4d60cb6b53e1f790037a7fd97f4889e",
54+
"sha256": "14e116b19747cb53a9fd1bc6aa5ef2d44da7f4635500632f3c8943d306d6bcf4",
5555
"size_bytes": 6303,
5656
"type": "evidence"
5757
},
@@ -63,7 +63,7 @@
6363
},
6464
{
6565
"path": "outputs/security/integration/integration-manifest.json",
66-
"sha256": "33a53f44625901e5053c5afb375b2186df6cad6c8ae11ac4889b092e60f3480b",
66+
"sha256": "1b2f2c08cafe97e12dfc0d8b9773874887a670e064dae3b8cda26c6574ad3d31",
6767
"size_bytes": 4620,
6868
"type": "evidence"
6969
}

‎outputs/portfolio/portfolio-manifest 2.json‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
{
2+
"archive_limitation": "No archive is created to avoid platform-specific tar metadata drift.",
3+
"as_of_date": "2026-01-01",
4+
"controlled_timestamp": "2026-01-01T00:00:00Z",
5+
"deterministic_archive_created": false,
6+
"generated_artifacts": [
7+
"evidence-index.json",
8+
"milestone-completion.json",
9+
"portfolio-metrics.json",
10+
"portfolio-readiness.json",
11+
"portfolio-summary.json",
12+
"report-index.json",
13+
"security-capability-matrix.json"
14+
],
15+
"git": {
16+
"branch": "main",
17+
"commit": "c1f48a46b68d",
18+
"dirty": "true"
19+
},
20+
"input_files": {
21+
"config/portfolio/capability-mapping.yaml": {
22+
"path": "config/portfolio/capability-mapping.yaml",
23+
"sha256": "ebe0adc53faea7b0d502824e33651ef08529f7804df36b3b1b984fb4ab5f6e1c"
24+
},
25+
"config/portfolio/portfolio-policy.yaml": {
26+
"path": "config/portfolio/portfolio-policy.yaml",
27+
"sha256": "aaa8b4f2c9ab6202af5d926f172f97922dff8dc345fc8ef0879768c38c1ce585"
28+
},
29+
"config/portfolio/readiness-criteria.yaml": {
30+
"path": "config/portfolio/readiness-criteria.yaml",
31+
"sha256": "4454afb483124032818e4bb669fca918bae5773d2b0fb39c214de84f4b954feb"
32+
},
33+
"config/portfolio/report-policy.yaml": {
34+
"path": "config/portfolio/report-policy.yaml",
35+
"sha256": "b1964f92add7c453bc7611e8a4372412a568a6b3442a7a4a8c08291cb171828a"
36+
}
37+
},
38+
"output_files": {
39+
"evidence-index.json": {
40+
"path": "outputs/security/portfolio/evidence-index.json",
41+
"sha256": "eb5e328506ba981fe78ca9998e7bea6782d1fc73d3c1bf0aa80c1c18dac704b8",
42+
"size_bytes": 2487
43+
},
44+
"milestone-completion.json": {
45+
"path": "outputs/security/portfolio/milestone-completion.json",
46+
"sha256": "86db7066859f0415e3d70aed56edfd788f0e16a54de1e956d5bc7db2d0a5de52",
47+
"size_bytes": 1749
48+
},
49+
"portfolio-metrics.json": {
50+
"path": "outputs/security/portfolio/portfolio-metrics.json",
51+
"sha256": "65b76ddc01fc6bb0942fbc6cb2aaaf9eecb15cd63ad46ee66198fbcbb70df0ff",
52+
"size_bytes": 664
53+
},
54+
"portfolio-readiness.json": {
55+
"path": "outputs/security/portfolio/portfolio-readiness.json",
56+
"sha256": "26a4208670a19af674d180190ba820b4e2c9198c483cdc9ecd24384bc938f946",
57+
"size_bytes": 490
58+
},
59+
"portfolio-summary.json": {
60+
"path": "outputs/security/portfolio/portfolio-summary.json",
61+
"sha256": "22fb77ec0b4fa7a9d5e87d2ec10f6e0f46f78eb3dc60e0a37cde6d3f70eb8a8d",
62+
"size_bytes": 930
63+
},
64+
"report-index.json": {
65+
"path": "outputs/security/portfolio/report-index.json",
66+
"sha256": "0c55e3f4f6613423f104bd7dd070d2ebdd23fd110555cf21a3da5747eb5545d9",
67+
"size_bytes": 3239
68+
},
69+
"security-capability-matrix.json": {
70+
"path": "outputs/security/portfolio/security-capability-matrix.json",
71+
"sha256": "8f0519fca1710abfd3a234eaedcb9ab8844f67f59f5f2b7cffb96cb21ff66a0a",
72+
"size_bytes": 7933
73+
}
74+
},
75+
"portfolio_id": "PF-a338b8a8e6209a13",
76+
"schema_version": "1.0"
77+
}

‎outputs/portfolio/portfolio-manifest.json‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
],
1515
"git": {
1616
"branch": "main",
17-
"commit": "107814de9314",
17+
"commit": "c1f48a46b68d",
1818
"dirty": "true"
1919
},
2020
"input_files": {
@@ -38,7 +38,7 @@
3838
"output_files": {
3939
"evidence-index.json": {
4040
"path": "outputs/security/portfolio/evidence-index.json",
41-
"sha256": "2cdf10c904402419e012d751c30e76adf79a7d2eb79c01bd8c63b714c6b953f5",
41+
"sha256": "ac514a9f06c8d680739f3a3f86a5599acb77d4559e212c14f50e0ac687508b3e",
4242
"size_bytes": 2487
4343
},
4444
"milestone-completion.json": {
@@ -58,12 +58,12 @@
5858
},
5959
"portfolio-summary.json": {
6060
"path": "outputs/security/portfolio/portfolio-summary.json",
61-
"sha256": "417b0a06f955045d2263795956eae7b943a797b704b377cfb3bd14bca98f9691",
61+
"sha256": "d56ef981def8a4b203a959d718aa6f42ff8cdce2acbeb1ca2aaa1a53d611a0df",
6262
"size_bytes": 930
6363
},
6464
"report-index.json": {
6565
"path": "outputs/security/portfolio/report-index.json",
66-
"sha256": "80d06b29d81fca03ee18b469e382bd5bd4988de6c81d426b477a889fecd87ccf",
66+
"sha256": "7479f270113eaae6844a1b0b6db9969c7824caaa9110dc5fafe525f0cb899346",
6767
"size_bytes": 3239
6868
},
6969
"security-capability-matrix.json": {
@@ -72,6 +72,6 @@
7272
"size_bytes": 7933
7373
}
7474
},
75-
"portfolio_id": "PF-7064eb6c6bfc28c7",
75+
"portfolio_id": "PF-2c8115399fb79924",
7676
"schema_version": "1.0"
7777
}

‎outputs/portfolio/portfolio-summary 2.json‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
"as_of_date": "2026-01-01",
3+
"canonical_findings": 44,
4+
"capability_count": 25,
5+
"controlled_timestamp": "2026-01-01T00:00:00Z",
6+
"deployment_status": "not_deployed",
7+
"evidence_sources": 11,
8+
"milestones_completed": 14,
9+
"portfolio_id": "PF-a338b8a8e6209a13",
10+
"portfolio_package": "outputs/portfolio",
11+
"project_name": "DevSecOps Application Security Pipeline",
12+
"readiness_limitations": [
13+
"Application security evidence is locally validated and the platform is not deployed.",
14+
"AWS architecture is a Terraform reference implementation only.",
15+
"Repository 5 integration is contract-only and does not write to another repository.",
16+
"Release decision remains conditional_pass because known non-blocking findings are governed."
17+
],
18+
"readiness_status": "ready_with_limitations",
19+
"release_decision": "conditional_pass",
20+
"reports_indexed": 15,
21+
"schema_version": "1.0",
22+
"source_findings": 46
23+
}

‎outputs/portfolio/portfolio-summary.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
"deployment_status": "not_deployed",
77
"evidence_sources": 11,
88
"milestones_completed": 14,
9-
"portfolio_id": "PF-7064eb6c6bfc28c7",
9+
"portfolio_id": "PF-2c8115399fb79924",
1010
"portfolio_package": "outputs/portfolio",
1111
"project_name": "DevSecOps Application Security Pipeline",
1212
"readiness_limitations": [

0 commit comments

Comments
 (0)