Skip to content

Commit 620739e

Browse files
Copilot304NotModified
Copilot
and
304NotModified
authored
feat: Add SARIF 2.1.0 report output with shared reporting layer (#56)
* Initial plan * feat: add SARIF report output support with shared RuleSeverity/RuleMetadata layer - Rename SonarRuleSeverity → RuleSeverity in Core/Reporting/ - Add RuleMetadata record and RuleMetadataProvider static class - Add SarifReporter with SARIF 2.1.0 DTOs (no external NuGet deps) - Add --sarif-report-file CLI option and wire into ValidatorRunner - Update all SonarRuleSeverity references to RuleSeverity - Add SarifReporterTests with snapshot tests - Update README with SARIF docs and GitHub Code Scanning example Agent-Logs-Url: https://github.com/304NotModified/SLNX-validator/sessions/5d6c6f75-d638-45f6-a4ec-d404d306a102 Co-authored-by: 304NotModified <5808377+304NotModified@users.noreply.github.com> * refactor: address PR review comments - Split SarifModels.cs into individual files per class - Add XML doc comments to RuleSeverity enum with SARIF/SonarQube mapping - Move --sarif-report-file section before --continue-on-error in README - Remove unclear .sarif.json tip line from README - Update severity override flags table to include SARIF level column - Add short SARIF explanation to GitHub Code Scanning section Agent-Logs-Url: https://github.com/304NotModified/SLNX-validator/sessions/c8bac85a-7f8b-43c1-911f-a7724715655c Co-authored-by: 304NotModified <5808377+304NotModified@users.noreply.github.com> * refactor: address second round of PR review comments - Rename RuleMetadata -> Rule, RuleMetadataProvider -> RuleProvider - Convert switch expression in RuleProvider to private dictionary - Remove <remarks> block from RuleSeverity (keep per-value <summary>) - README: expand --sarif-report-file section with SARIF intro, benefits, links, viewers - README: add internal links in severity mapping section - README: update severity override description to mention console + SQ + SARIF Agent-Logs-Url: https://github.com/304NotModified/SLNX-validator/sessions/24076cbf-30ba-4777-8367-61481287936c Co-authored-by: 304NotModified <5808377+304NotModified@users.noreply.github.com> * refactor: address third round of PR review comments - RuleProvider: add Create() helper with default MAJOR severity to eliminate repeated code.ToCode() and explicit MAJOR - README: replace VS Code -> Visual Studio Code, VS -> Visual Studio throughout - README: fix out-of-the-box claim (Visual Studio built-in, VS Code/Azure DevOps need extensions) - README: fix Visual Studio link to SARIF viewer docs (was Roslyn analyzers) - README: fix Azure DevOps link to SARIF SAST Scans Tab extension (was code coverage docs) Agent-Logs-Url: https://github.com/304NotModified/SLNX-validator/sessions/f238c504-01a4-4790-a858-136add863e89 Co-authored-by: 304NotModified <5808377+304NotModified@users.noreply.github.com> * refactor: address fourth round of PR review comments - RuleProvider: dedup error codes by having Create() return (Key, Rule) tuple, build dictionary via .ToDictionary() so each ValidationErrorCode only appears once - RuleProvider: rename rules: "Input file not found" -> "SLNX file not found", "Invalid file extension" -> "Invalid solution file extension", "File is not a text file" -> "SLNX file is not a text file" - Update snapshot tests to match new rule names - README: link "Visual Studio" in benefits bullet to official SARIF viewer docs - README: viewers section - Visual Studio/Code/Azure DevOps no longer hyperlinks; only the extension/docs links are clickable, no duplicate links Agent-Logs-Url: https://github.com/304NotModified/SLNX-validator/sessions/e6c871f9-f7c6-4137-9c1d-716db710bb21 Co-authored-by: 304NotModified <5808377+304NotModified@users.noreply.github.com> * fix: correct Visual Studio SARIF viewer - requires extension, not built-in The previously linked docs URL returned 404. Visual Studio does not have a built-in SARIF viewer; it requires the Microsoft SARIF Viewer extension. - Benefits bullet: removed "built-in" claim, now says "via extensions for Visual Studio, Visual Studio Code, and Azure DevOps" - Viewers list: Visual Studio entry now links to the correct Marketplace extension (WDGIS.MicrosoftSarifViewer2022) Agent-Logs-Url: https://github.com/304NotModified/SLNX-validator/sessions/273158ae-504a-46bd-9190-2e9f6a642aa7 Co-authored-by: 304NotModified <5808377+304NotModified@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: 304NotModified <5808377+304NotModified@users.noreply.github.com>
1 parent 9a3c14c commit 620739e

35 files changed

Lines changed: 1359 additions & 128 deletions

‎README.md‎

Lines changed: 122 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -61,9 +61,44 @@ Writes a [SonarQube generic issue report](https://docs.sonarsource.com/sonarqube
6161
slnx-validator MySolution.slnx --sonarqube-report-file sonar-issues.json --continue-on-error
6262
```
6363

64+
### `--sarif-report-file <file>`
65+
66+
[SARIF](https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html) (Static Analysis Results Interchange Format) is an open OASIS standard for static analysis tool output. It enables interoperability between analysis tools and result viewers, so the same report can be consumed by GitHub Code Scanning, Azure DevOps, Visual Studio, Visual Studio Code, and other tools without any conversion.
67+
68+
**Benefits of SARIF:**
69+
- Native integration with [GitHub Code Scanning](https://docs.github.com/en/code-security/code-scanning) — issues appear as alerts in the **Security → Code Scanning** tab
70+
- Supported via extensions for Visual Studio, Visual Studio Code, and Azure DevOps
71+
- Rich result format: rule metadata, severity, file paths, and line numbers in a single file
72+
- Widely adopted standard — see [SARIF tutorials](https://github.com/microsoft/sarif-tutorials) and the [SARIF web viewer](https://sarifweb.azurewebsites.net/)
73+
74+
**Usage:**
75+
76+
```powershell
77+
slnx-validator MySolution.slnx --sarif-report-file results.sarif --continue-on-error
78+
```
79+
80+
Severity mapping from `RuleSeverity` to SARIF levels (see also the [GitHub Code Scanning integration example](#github-code-scanning-integration-example)):
81+
82+
| Severity | SARIF level |
83+
|----------|-------------|
84+
| `BLOCKER`, `CRITICAL`, `MAJOR` | `error` |
85+
| `MINOR` | `warning` |
86+
| `INFO` | `note` |
87+
88+
Severity overrides (via `--minor`, `--info`, `--ignore`, etc.) are reflected in the SARIF output. See [Severity override flags](#severity-override-flags) for details.
89+
90+
**Viewers and reporting:**
91+
- [GitHub Code Scanning](https://docs.github.com/en/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github) — upload via `github/codeql-action/upload-sarif@v3` (see [example below](#github-code-scanning-integration-example))
92+
- Visual Studio — requires the [Microsoft SARIF Viewer extension](https://marketplace.visualstudio.com/items?itemName=WDGIS.MicrosoftSarifViewer2022)
93+
- Visual Studio Code — requires the [SARIF Viewer extension](https://marketplace.visualstudio.com/items?itemName=MS-SarifVSCode.sarif-viewer)
94+
- Azure DevOps — requires the [SARIF SAST Scans Tab extension](https://marketplace.visualstudio.com/items?itemName=sariftools.scans)
95+
- [SARIF web viewer](https://sarifweb.azurewebsites.net/) — online viewer for quick inspection
96+
97+
Further reading: [SARIF tutorials](https://github.com/microsoft/sarif-tutorials) · [Why SARIF?](https://github.com/microsoft/sarif-tutorials/blob/main/docs/1-Introduction.md#why-sarif) · [SonarSource SARIF overview](https://www.sonarsource.com/resources/library/sarif/)
98+
6499
### `--continue-on-error`
65100

66-
Always exits with code `0`, even when validation errors are found. Useful in CI pipelines where SonarQube handles the failure decision. Default: `false`.
101+
Always exits with code `0`, even when validation errors are found. Useful in CI pipelines where SonarQube or GitHub Code Scanning handles the failure decision. Default: `false`.
67102

68103
### `--required-files`
69104

@@ -120,16 +155,16 @@ slnx-validator MySolution.slnx --required-files "appsettings.json;docs/"
120155

121156
### Severity override flags
122157

123-
Override the severity of specific validation codes, or suppress them entirely. This controls both the exit code behaviour and the severity written to the SonarQube JSON report.
158+
Override the severity of specific validation codes, or suppress them entirely. This controls the exit code behaviour, the console output label, and the severity written to SonarQube and SARIF reports.
124159

125-
| Flag | Severity | Causes exit code `1`? |
126-
|------|----------|-----------------------|
127-
| `--blocker <codes>` | `BLOCKER` | ✅ yes |
128-
| `--critical <codes>` | `CRITICAL` | ✅ yes |
129-
| `--major <codes>` | `MAJOR` | ✅ yes (default for all codes) |
130-
| `--minor <codes>` | `MINOR` | ❌ no — shown with `(warning)` label |
131-
| `--info <codes>` | `INFO` | ❌ no — shown with `(info)` label |
132-
| `--ignore <codes>` | *(suppressed)* | ❌ no — not shown at all, not in SonarQube report |
160+
| Flag | Severity | Causes exit code `1`? | SARIF level |
161+
|------|----------|-----------------------|-------------|
162+
| `--blocker <codes>` | `BLOCKER` | ✅ yes | `error` |
163+
| `--critical <codes>` | `CRITICAL` | ✅ yes | `error` |
164+
| `--major <codes>` | `MAJOR` | ✅ yes (default for all codes) | `error` |
165+
| `--minor <codes>` | `MINOR` | ❌ no — shown with `(warning)` label | `warning` |
166+
| `--info <codes>` | `INFO` | ❌ no — shown with `(info)` label | `note` |
167+
| `--ignore <codes>` | *(suppressed)* | ❌ no — not shown at all, not in SonarQube or SARIF report | *(omitted)* |
133168

134169
Each flag accepts a **comma-separated list of codes** or the **wildcard `*`** to match all codes:
135170

@@ -156,9 +191,9 @@ When `*` is combined with specific code flags, **specific codes always win over
156191
slnx-validator MySolution.slnx --info * --major SLNX011
157192
```
158193

159-
**Effect on SonarQube report**
194+
**Effect on reports**
160195

161-
Severity overrides are reflected in the generated rule definition in the JSON report:
196+
Severity overrides are reflected in the generated rule definition in both SonarQube and SARIF reports:
162197

163198
```json
164199
{
@@ -168,7 +203,7 @@ Severity overrides are reflected in the generated rule definition in the JSON re
168203
}
169204
```
170205

171-
Codes set to `--ignore` are excluded from both the `rules` and `issues` arrays entirely.
206+
Codes set to `--ignore` are excluded from both the `rules` and `issues`/`results` arrays entirely.
172207

173208
## SonarQube integration example
174209

@@ -220,6 +255,80 @@ sonar.sources=${{ parameters.slnPath }}
220255
sonar.xml.file.suffixes=.xml,.xsd,.xsl,.slnx
221256
```
222257

258+
## GitHub Code Scanning integration example
259+
260+
Use `--sarif-report-file` to generate a SARIF 2.1.0 file and upload it to GitHub Code Scanning:
261+
262+
```powershell
263+
slnx-validator MySolution.slnx --sarif-report-file results.sarif --continue-on-error
264+
```
265+
266+
```yaml
267+
- name: Validate .slnx files
268+
run: slnx-validator MySolution.slnx --sarif-report-file results.sarif --continue-on-error
269+
270+
- name: Upload SARIF to GitHub Code Scanning
271+
uses: github/codeql-action/upload-sarif@v3
272+
with:
273+
sarif_file: results.sarif
274+
```
275+
276+
This uploads the validation results to the **Security → Code Scanning** tab of your repository. Issues appear as alerts with rule descriptions, file locations, and links back to the relevant lines.
277+
278+
Example SARIF output:
279+
280+
```json
281+
{
282+
"$schema": "https://json.schemastore.org/sarif-2.1.0.json",
283+
"version": "2.1.0",
284+
"runs": [
285+
{
286+
"tool": {
287+
"driver": {
288+
"name": "slnx-validator",
289+
"informationUri": "https://github.com/304NotModified/SLNX-validator",
290+
"rules": [
291+
{
292+
"id": "SLNX011",
293+
"shortDescription": {
294+
"text": "Referenced file not found"
295+
},
296+
"fullDescription": {
297+
"text": "A file referenced in a <File Path=\"...\"> element does not exist on disk."
298+
},
299+
"defaultConfiguration": {
300+
"level": "error"
301+
}
302+
}
303+
]
304+
}
305+
},
306+
"results": [
307+
{
308+
"ruleId": "SLNX011",
309+
"level": "error",
310+
"message": {
311+
"text": "File not found: docs\\CONTRIBUTING.md"
312+
},
313+
"locations": [
314+
{
315+
"physicalLocation": {
316+
"artifactLocation": {
317+
"uri": "MySolution.slnx"
318+
},
319+
"region": {
320+
"startLine": 4
321+
}
322+
}
323+
}
324+
]
325+
}
326+
]
327+
}
328+
]
329+
}
330+
```
331+
223332
## Example output
224333

225334
### All valid ✅

‎src/SLNX-validator.Core/Reporting/Rule.cs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
namespace JulianVerdurmen.SlnxValidator.Core.Reporting;
2+
3+
public sealed record Rule(
4+
string Id,
5+
string Name,
6+
string Description,
7+
RuleSeverity DefaultSeverity);

‎src/SLNX-validator.Core/Reporting/RuleProvider.cs‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
using JulianVerdurmen.SlnxValidator.Core.ValidationResults;
2+
3+
namespace JulianVerdurmen.SlnxValidator.Core.Reporting;
4+
5+
public static class RuleProvider
6+
{
7+
private static (ValidationErrorCode Key, Rule Rule) Create(
8+
ValidationErrorCode code, string name, string description,
9+
RuleSeverity severity = RuleSeverity.MAJOR) =>
10+
(code, new Rule(code.ToCode(), name, description, severity));
11+
12+
private static readonly Dictionary<ValidationErrorCode, Rule> Rules =
13+
new (ValidationErrorCode Key, Rule Rule)[]
14+
{
15+
Create(ValidationErrorCode.FileNotFound,
16+
"SLNX file not found",
17+
"The specified .slnx file does not exist."),
18+
19+
Create(ValidationErrorCode.InvalidExtension,
20+
"Invalid solution file extension",
21+
"The input file does not have a .slnx extension.",
22+
RuleSeverity.MINOR),
23+
24+
Create(ValidationErrorCode.NotATextFile,
25+
"SLNX file is not a text file",
26+
"The file is binary and cannot be parsed as XML."),
27+
28+
Create(ValidationErrorCode.InvalidXml,
29+
"Invalid XML",
30+
"The .slnx file is not valid XML."),
31+
32+
Create(ValidationErrorCode.ReferencedFileNotFound,
33+
"Referenced file not found",
34+
"A file referenced in a <File Path=\"...\"> element does not exist on disk."),
35+
36+
Create(ValidationErrorCode.InvalidWildcardUsage,
37+
"Invalid wildcard usage",
38+
"A <File Path=\"...\"> element contains a wildcard pattern, which is not supported.",
39+
RuleSeverity.MINOR),
40+
41+
Create(ValidationErrorCode.XsdViolation,
42+
"XSD schema violation",
43+
"The XML structure violates the .slnx schema."),
44+
45+
Create(ValidationErrorCode.RequiredFileDoesntExistOnSystem,
46+
"Required file does not exist on the system",
47+
"A file required by '--required-files' does not exist on the file system."),
48+
49+
Create(ValidationErrorCode.RequiredFileNotReferencedInSolution,
50+
"Required file not referenced in solution",
51+
"A file required by '--required-files' exists on the file system but is not referenced as a <File> element in the solution."),
52+
}.ToDictionary(e => e.Key, e => e.Rule);
53+
54+
public static Rule Get(ValidationErrorCode code)
55+
{
56+
if (Rules.TryGetValue(code, out var rule))
57+
return rule;
58+
throw new ArgumentOutOfRangeException(nameof(code), code, null);
59+
}
60+
}

‎src/SLNX-validator.Core/Reporting/RuleSeverity.cs‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
namespace JulianVerdurmen.SlnxValidator.Core.Reporting;
2+
3+
/// <summary>Universal severity level used across the entire validator pipeline.</summary>
4+
public enum RuleSeverity
5+
{
6+
/// <summary>SARIF: <c>error</c> — causes exit code 1.</summary>
7+
BLOCKER,
8+
/// <summary>SARIF: <c>error</c> — causes exit code 1.</summary>
9+
CRITICAL,
10+
/// <summary>SARIF: <c>error</c> — causes exit code 1. Default severity for most rules.</summary>
11+
MAJOR,
12+
/// <summary>SARIF: <c>warning</c> — does not cause exit code 1.</summary>
13+
MINOR,
14+
/// <summary>SARIF: <c>note</c> — does not cause exit code 1.</summary>
15+
INFO
16+
}

‎src/SLNX-validator.Core/SarifReporting/ISarifReporter.cs‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
using JulianVerdurmen.SlnxValidator.Core.Reporting;
2+
using JulianVerdurmen.SlnxValidator.Core.ValidationResults;
3+
4+
namespace JulianVerdurmen.SlnxValidator.Core.SarifReporting;
5+
6+
public interface ISarifReporter
7+
{
8+
Task WriteReportAsync(IReadOnlyList<FileValidationResult> results, string outputPath,
9+
IReadOnlyDictionary<ValidationErrorCode, RuleSeverity?>? severityOverrides = null);
10+
11+
Task WriteReportAsync(IReadOnlyList<FileValidationResult> results, Stream outputStream,
12+
IReadOnlyDictionary<ValidationErrorCode, RuleSeverity?>? severityOverrides = null);
13+
}

‎src/SLNX-validator.Core/SarifReporting/SarifArtifactLocation.cs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
namespace JulianVerdurmen.SlnxValidator.Core.SarifReporting;
2+
3+
internal sealed record SarifArtifactLocation
4+
{
5+
public required string Uri { get; init; }
6+
}

‎src/SLNX-validator.Core/SarifReporting/SarifDefaultConfiguration.cs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
namespace JulianVerdurmen.SlnxValidator.Core.SarifReporting;
2+
3+
internal sealed record SarifDefaultConfiguration
4+
{
5+
public required string Level { get; init; }
6+
}

‎src/SLNX-validator.Core/SarifReporting/SarifLocation.cs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
namespace JulianVerdurmen.SlnxValidator.Core.SarifReporting;
2+
3+
internal sealed record SarifLocation
4+
{
5+
public required SarifPhysicalLocation PhysicalLocation { get; init; }
6+
}

‎src/SLNX-validator.Core/SarifReporting/SarifLog.cs‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
using System.Text.Json.Serialization;
2+
3+
namespace JulianVerdurmen.SlnxValidator.Core.SarifReporting;
4+
5+
internal sealed record SarifLog
6+
{
7+
[JsonPropertyName("$schema")]
8+
public required string Schema { get; init; }
9+
10+
public required string Version { get; init; }
11+
12+
public required List<SarifRun> Runs { get; init; }
13+
}

‎src/SLNX-validator.Core/SarifReporting/SarifMessage.cs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
namespace JulianVerdurmen.SlnxValidator.Core.SarifReporting;
2+
3+
internal sealed record SarifMessage
4+
{
5+
public required string Text { get; init; }
6+
}

0 commit comments

Comments
 (0)