Skip to content

Commit 11dc019

Browse files
sferiksferik
committed
Publish a JSON Schema for coverage.json
1 parent 6fcec99 commit 11dc019

12 files changed

Lines changed: 385 additions & 1 deletion

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ Unreleased
1313
* `# :nocov:` toggle comments (and the configurable `SimpleCov.nocov_token` / `SimpleCov.skip_token`) are deprecated in favor of the new `# simplecov:disable` / `# simplecov:enable` directives. Each file that still uses `# :nocov:` emits a one-time deprecation warning to stderr at load time pointing at the recommended replacement, and any call to `SimpleCov.nocov_token` or `SimpleCov.skip_token` (getter or setter) likewise warns. The directive will be removed in a future release.
1414

1515
## Enhancements
16+
* JSON formatter: `coverage.json` now carries a `meta.schema_version` field (`"major.minor"`, currently `"1.0"`) describing which version of the schema it conforms to. A formal JSON Schema (draft-07) is published at `schemas/coverage.schema.json` so downstream tools can validate inputs, generate types, or pin to a known shape. The schema version is independent of the gem version: additive changes bump minor, removals or shape changes bump major.
1617
* Added `# simplecov:disable` / `# simplecov:enable` directive comments for selectively skipping `line`, `branch`, and `method` coverage. Block form (own line) opens a region until the matching `# simplecov:enable`; inline form (trailing a code line) skips just that line. Categories may be combined (`# simplecov:disable line, branch`); omitting categories targets all three. Any trailing text is treated as a free-form reason and discarded (e.g. `# simplecov:disable line legacy adapter`). Directive markers inside string literals or heredocs are ignored.
1718
* JSON formatter: `meta.timestamp` is now emitted with millisecond precision (`iso8601(3)`) so the concurrent-overwrite warning can distinguish writes within the same wall-clock second
1819
* JSON formatter: added `total` section with aggregate coverage statistics (covered, missed, total, percent, strength) for line, branch, and method coverage. Line stats additionally include `omitted` (count of blank/comment lines, i.e. lines that cannot be covered)

‎Gemfile‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ group :development do
77
gem "capybara"
88
gem "cucumber"
99
gem "cuprite"
10+
gem "json_schemer"
1011
gem "nokogiri"
1112
gem "rackup"
1213
gem "rake"

‎Gemfile.lock‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ GEM
7373
websocket-driver (~> 0.7)
7474
ffi (1.17.4)
7575
ffi (1.17.4-java)
76+
hana (1.3.7)
7677
io-console (0.8.2)
7778
io-console (0.8.2-java)
7879
irb (1.18.0)
@@ -83,6 +84,11 @@ GEM
8384
jar-dependencies (0.5.7)
8485
json (2.19.5)
8586
json (2.19.5-java)
87+
json_schemer (2.5.0)
88+
bigdecimal
89+
hana (~> 1.3)
90+
regexp_parser (~> 2.0)
91+
simpleidn (~> 0.2)
8692
language_server-protocol (3.17.0.5)
8793
lint_roller (1.1.0)
8894
logger (1.7.0)
@@ -169,6 +175,7 @@ GEM
169175
lint_roller (~> 1.1)
170176
rubocop (~> 1.81)
171177
ruby-progressbar (1.13.0)
178+
simpleidn (0.2.3)
172179
stringio (3.2.0)
173180
sys-uname (1.5.1)
174181
ffi (~> 1.1)
@@ -203,6 +210,7 @@ DEPENDENCIES
203210
capybara
204211
cucumber
205212
cuprite
213+
json_schemer
206214
nokogiri
207215
rackup
208216
rake

‎README.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -931,6 +931,29 @@ SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter
931931

932932
> The JSON formatter was originally a separate gem called [simplecov_json_formatter](https://github.com/codeclimate-community/simplecov_json_formatter). It is now built in and loaded by default. Existing code that does `require "simplecov_json_formatter"` will continue to work.
933933
934+
### `coverage.json` schema
935+
936+
The `coverage.json` payload is treated as a public contract. Every emitted file carries a `meta.schema_version` (`"major.minor"`) describing which version of the schema it conforms to. A JSON Schema (draft-07) lives in the repo at [`schemas/coverage.schema.json`](schemas/coverage.schema.json); downstream tools can use it to validate inputs, generate types, or pin themselves to a known shape.
937+
938+
The schema version is independent of the gem version:
939+
940+
- Additive changes (new fields) bump the **minor** segment. Existing consumers keep working.
941+
- Removals or shape changes bump the **major** segment.
942+
943+
The current version is **1.0**. Top-level structure:
944+
945+
```jsonc
946+
{
947+
"meta": { /* schema_version, simplecov_version, command_name, project_name, timestamp, root, branch_coverage, method_coverage */ },
948+
"total": { /* aggregate stats for lines (and branches / methods when enabled) */ },
949+
"coverage": { "<project-relative path>": { /* per-file lines, source, branches, methods, etc. */ } },
950+
"groups": { "<group name>": { /* per-group stats + files */ } },
951+
"errors": { /* minimum_coverage, minimum_coverage_by_file, minimum_coverage_by_group, maximum_coverage_drop violations */ }
952+
}
953+
```
954+
955+
The `.resultset.json` file is **not** schema'd — it's SimpleCov-internal and may change shape across releases. Build integrations on top of `coverage.json`.
956+
934957
## Available formatters, editor integrations and hosted services
935958

936959
* [Open Source formatter and integration plugins for SimpleCov](doc/alternate-formatters.md)

‎lib/simplecov/formatter/json_formatter/result_hash_formatter.rb‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,16 @@ def formatted_result
9393
@formatted_result ||= {meta: format_meta, total: {}, coverage: {}, groups: {}, errors: {}}
9494
end
9595

96+
# Bump SCHEMA_VERSION when the JSON shape changes. Additive
97+
# changes bump the minor segment; removals or shape changes bump
98+
# major. See schemas/coverage.schema.json for the contract this
99+
# version describes.
100+
SCHEMA_VERSION = "1.0"
101+
private_constant :SCHEMA_VERSION
102+
96103
def format_meta
97104
{
105+
schema_version: SCHEMA_VERSION,
98106
simplecov_version: SimpleCov::VERSION,
99107
command_name: @result.command_name,
100108
project_name: SimpleCov.project_name,

‎schemas/coverage.schema.json‎

Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"$id": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage.schema.json",
4+
"title": "SimpleCov coverage.json",
5+
"description": "Schema for the coverage.json file emitted by SimpleCov's JSONFormatter. Versioned independently of the gem; non-breaking additions bump the minor segment of schema_version, breaking changes bump the major segment.",
6+
"type": "object",
7+
"required": ["meta", "total", "coverage", "groups", "errors"],
8+
"additionalProperties": false,
9+
"properties": {
10+
"meta": {
11+
"type": "object",
12+
"required": [
13+
"schema_version",
14+
"simplecov_version",
15+
"command_name",
16+
"project_name",
17+
"timestamp",
18+
"root",
19+
"branch_coverage",
20+
"method_coverage"
21+
],
22+
"additionalProperties": false,
23+
"properties": {
24+
"schema_version": {
25+
"type": "string",
26+
"pattern": "^\\d+\\.\\d+$",
27+
"description": "Schema major.minor. Additive changes bump minor; breaking changes bump major."
28+
},
29+
"simplecov_version": {
30+
"type": "string",
31+
"description": "The version of the SimpleCov gem that produced this file."
32+
},
33+
"command_name": {"type": "string"},
34+
"project_name": {"type": "string"},
35+
"timestamp": {
36+
"type": "string",
37+
"format": "date-time",
38+
"description": "ISO 8601 timestamp with millisecond precision."
39+
},
40+
"root": {
41+
"type": "string",
42+
"description": "Absolute path to SimpleCov.root at the time of write."
43+
},
44+
"branch_coverage": {"type": "boolean"},
45+
"method_coverage": {"type": "boolean"}
46+
}
47+
},
48+
"total": {"$ref": "#/definitions/totals"},
49+
"coverage": {
50+
"type": "object",
51+
"description": "Map of project-relative file paths to per-file coverage data.",
52+
"additionalProperties": {"$ref": "#/definitions/source_file"}
53+
},
54+
"groups": {
55+
"type": "object",
56+
"description": "Map of group names to per-group totals plus the list of files in the group.",
57+
"additionalProperties": {"$ref": "#/definitions/group"}
58+
},
59+
"errors": {
60+
"type": "object",
61+
"description": "Threshold violations from minimum_coverage, minimum_coverage_by_file, minimum_coverage_by_group, and maximum_coverage_drop. Empty object when no thresholds were violated.",
62+
"additionalProperties": false,
63+
"properties": {
64+
"minimum_coverage": {
65+
"type": "object",
66+
"description": "Keyed by criterion: lines, branches, methods.",
67+
"additionalProperties": {"$ref": "#/definitions/expected_actual"}
68+
},
69+
"minimum_coverage_by_file": {
70+
"type": "object",
71+
"description": "Keyed by criterion, then by project-relative filename.",
72+
"additionalProperties": {
73+
"type": "object",
74+
"additionalProperties": {"$ref": "#/definitions/expected_actual"}
75+
}
76+
},
77+
"minimum_coverage_by_group": {
78+
"type": "object",
79+
"description": "Keyed by group name, then by criterion.",
80+
"additionalProperties": {
81+
"type": "object",
82+
"additionalProperties": {"$ref": "#/definitions/expected_actual"}
83+
}
84+
},
85+
"maximum_coverage_drop": {
86+
"type": "object",
87+
"description": "Keyed by criterion: lines, branches, methods.",
88+
"additionalProperties": {"$ref": "#/definitions/maximum_actual"}
89+
}
90+
}
91+
}
92+
},
93+
"definitions": {
94+
"totals": {
95+
"type": "object",
96+
"required": ["lines"],
97+
"additionalProperties": false,
98+
"properties": {
99+
"lines": {"$ref": "#/definitions/line_statistic"},
100+
"branches": {"$ref": "#/definitions/coverage_statistic"},
101+
"methods": {"$ref": "#/definitions/coverage_statistic"}
102+
}
103+
},
104+
"source_file": {
105+
"type": "object",
106+
"required": ["lines", "source", "lines_covered_percent", "covered_lines", "missed_lines", "total_lines"],
107+
"additionalProperties": false,
108+
"properties": {
109+
"lines": {
110+
"type": "array",
111+
"description": "Per-source-line coverage. Element index N corresponds to source line N+1. Integer hit-count, null for non-relevant (blank/comment) lines, or the string \"ignored\" for lines inside a simplecov:disable / :nocov: region.",
112+
"items": {"$ref": "#/definitions/line_coverage"}
113+
},
114+
"source": {
115+
"type": "array",
116+
"description": "Source lines, in order. Same length as the lines array.",
117+
"items": {"type": "string"}
118+
},
119+
"lines_covered_percent": {"type": "number"},
120+
"covered_lines": {"type": "integer", "minimum": 0},
121+
"missed_lines": {"type": "integer", "minimum": 0},
122+
"total_lines": {"type": "integer", "minimum": 0},
123+
"branches": {
124+
"type": "array",
125+
"items": {"$ref": "#/definitions/branch"}
126+
},
127+
"branches_covered_percent": {"type": "number"},
128+
"covered_branches": {"type": "integer", "minimum": 0},
129+
"missed_branches": {"type": "integer", "minimum": 0},
130+
"total_branches": {"type": "integer", "minimum": 0},
131+
"methods": {
132+
"type": "array",
133+
"items": {"$ref": "#/definitions/method"}
134+
},
135+
"methods_covered_percent": {"type": "number"},
136+
"covered_methods": {"type": "integer", "minimum": 0},
137+
"missed_methods": {"type": "integer", "minimum": 0},
138+
"total_methods": {"type": "integer", "minimum": 0}
139+
}
140+
},
141+
"branch": {
142+
"type": "object",
143+
"required": ["type", "start_line", "end_line", "coverage", "inline", "report_line"],
144+
"additionalProperties": false,
145+
"properties": {
146+
"type": {
147+
"type": "string",
148+
"description": "Branch kind from Ruby's Coverage library (e.g. \"then\", \"else\", \"when\")."
149+
},
150+
"start_line": {"type": "integer", "minimum": 1},
151+
"end_line": {"type": "integer", "minimum": 1},
152+
"coverage": {"$ref": "#/definitions/line_coverage"},
153+
"inline": {"type": "boolean"},
154+
"report_line": {"type": "integer", "minimum": 1}
155+
}
156+
},
157+
"method": {
158+
"type": "object",
159+
"required": ["name", "start_line", "end_line", "coverage"],
160+
"additionalProperties": false,
161+
"properties": {
162+
"name": {
163+
"type": "string",
164+
"description": "Qualified method name, e.g. \"Foo#bar\", \"Foo.bar\", or \"Foo::Bar#baz\"."
165+
},
166+
"start_line": {"type": "integer", "minimum": 1},
167+
"end_line": {"type": "integer", "minimum": 1},
168+
"coverage": {"$ref": "#/definitions/line_coverage"}
169+
}
170+
},
171+
"group": {
172+
"type": "object",
173+
"required": ["lines", "files"],
174+
"additionalProperties": false,
175+
"properties": {
176+
"lines": {"$ref": "#/definitions/line_statistic"},
177+
"branches": {"$ref": "#/definitions/coverage_statistic"},
178+
"methods": {"$ref": "#/definitions/coverage_statistic"},
179+
"files": {
180+
"type": "array",
181+
"description": "Project-relative paths of the files that fell into this group.",
182+
"items": {"type": "string"}
183+
}
184+
}
185+
},
186+
"line_statistic": {
187+
"type": "object",
188+
"required": ["covered", "missed", "omitted", "total", "percent", "strength"],
189+
"additionalProperties": false,
190+
"properties": {
191+
"covered": {"type": "integer", "minimum": 0},
192+
"missed": {"type": "integer", "minimum": 0},
193+
"omitted": {
194+
"type": "integer",
195+
"minimum": 0,
196+
"description": "Lines that cannot be covered (blank, comment, etc.). Only present on line stats."
197+
},
198+
"total": {"type": "integer", "minimum": 0},
199+
"percent": {"type": "number"},
200+
"strength": {"type": "number"}
201+
}
202+
},
203+
"coverage_statistic": {
204+
"type": "object",
205+
"required": ["covered", "missed", "total", "percent", "strength"],
206+
"additionalProperties": false,
207+
"properties": {
208+
"covered": {"type": "integer", "minimum": 0},
209+
"missed": {"type": "integer", "minimum": 0},
210+
"total": {"type": "integer", "minimum": 0},
211+
"percent": {"type": "number"},
212+
"strength": {"type": "number"}
213+
}
214+
},
215+
"line_coverage": {
216+
"description": "Integer hit-count for executable lines/branches/methods, null for non-relevant lines, or the literal string \"ignored\" for code inside a simplecov:disable region.",
217+
"oneOf": [
218+
{"type": "integer", "minimum": 0},
219+
{"type": "null"},
220+
{"type": "string", "const": "ignored"}
221+
]
222+
},
223+
"expected_actual": {
224+
"type": "object",
225+
"required": ["expected", "actual"],
226+
"additionalProperties": false,
227+
"properties": {
228+
"expected": {"type": "number"},
229+
"actual": {"type": "number"}
230+
}
231+
},
232+
"maximum_actual": {
233+
"type": "object",
234+
"required": ["maximum", "actual"],
235+
"additionalProperties": false,
236+
"properties": {
237+
"maximum": {"type": "number"},
238+
"actual": {"type": "number"}
239+
}
240+
}
241+
}
242+
}

‎simplecov.gemspec‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,6 @@ Gem::Specification.new do |gem|
3636

3737
gem.required_ruby_version = ">= 3.1"
3838

39-
gem.files = Dir["{lib}/**/*.*", "bin/*", "LICENSE", "CHANGELOG.md", "README.md", "doc/*"]
39+
gem.files = Dir["{lib,schemas}/**/*.*", "bin/*", "LICENSE", "CHANGELOG.md", "README.md", "doc/*"]
4040
gem.require_paths = ["lib"]
4141
end

‎spec/fixtures/json/sample.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"meta": {
3+
"schema_version": "1.0",
34
"simplecov_version": "0.22.0",
45
"command_name": "STUB_COMMAND_NAME",
56
"project_name": "STUB_PROJECT_NAME",

‎spec/fixtures/json/sample_groups.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"meta": {
3+
"schema_version": "1.0",
34
"simplecov_version": "0.22.0",
45
"command_name": "STUB_COMMAND_NAME",
56
"project_name": "STUB_PROJECT_NAME",

‎spec/fixtures/json/sample_with_branch.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"meta": {
3+
"schema_version": "1.0",
34
"simplecov_version": "0.22.0",
45
"command_name": "STUB_COMMAND_NAME",
56
"project_name": "STUB_PROJECT_NAME",

0 commit comments

Comments
 (0)