Skip to content

Commit 1aaf166

Browse files
sferiksferik
committed
Publish a JSON Schema for coverage.json
1 parent ad30894 commit 1aaf166

12 files changed

Lines changed: 445 additions & 1 deletion

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ Unreleased
2525
* `# :nocov:` toggle comments (and the configurable `SimpleCov.nocov_token` / `SimpleCov.skip_token`) are deprecated in favor of the new `# simplecov:disable` / `# simplecov:enable` directives. Each file that still uses `# :nocov:` emits a one-time deprecation warning to stderr at load time pointing at the recommended replacement, and any call to `SimpleCov.nocov_token` or `SimpleCov.skip_token` (getter or setter) likewise warns. The directive will be removed in a future release.
2626

2727
## Enhancements
28+
* JSON formatter: `coverage.json` now carries a `meta.schema_version` field (`"major.minor"`, currently `"1.0"`) describing which version of the schema it conforms to. A formal JSON Schema (draft-07) is published at `schemas/coverage.schema.json` so downstream tools can validate inputs, generate types, or pin to a known shape. The schema version is independent of the gem version: additive changes bump minor, removals or shape changes bump major.
2829
* Added `SimpleCov::ParallelAdapters` — a pluggable adapter interface for parallel test runners. SimpleCov's coordination with parallel test runners (deciding which worker does final-result work, waiting for siblings, knowing how many resultsets to expect) now routes through an adapter chain rather than hard-coding the `parallel_tests` gem's API. Two adapters ship: `ParallelTestsAdapter` wraps the historical grosser/parallel_tests gem (precise, gem-API-based); `GenericAdapter` handles any runner that follows the `TEST_ENV_NUMBER` / `PARALLEL_TEST_GROUPS` env-var convention without shipping a Ruby API. The practical impact: **parallel_rspec (and any similar env-var-only runner) now works out of the box** — previously every worker thought it was the "final" one and they clobbered each other's resultsets. Custom runners can register their own adapter via `SimpleCov::ParallelAdapters.register MyAdapter`, where `MyAdapter` subclasses `SimpleCov::ParallelAdapters::Base` and overrides the four contract methods (`active?`, `first_worker?`, `wait_for_siblings`, `expected_worker_count`). See #1065.
2930
* Added `SimpleCov.ignore_branches` for opting out of synthetic `:else` branches that Ruby's `Coverage` library reports for constructs with no literal `else` keyword — exhaustive `case/in` pattern matches, `case/when` without `else`, `||=` / `&&=`, and `if` / `unless` without `else`. Variadic; only `:implicit_else` is supported today, with room for future synthetic branch types. Calling it without (or before) `enable_coverage :branch` is harmless — the setting is stored and applies once branch coverage is enabled. Explicit `else` arms still count. See #1033.
3031
* Added `SimpleCov.cover` for declaring a positive coverage scope (the long-requested allowlist counterpart to `add_filter`). Accepts string globs, Regexps, blocks, or arrays of those; multiple calls union. When any `cover` matcher is configured the report drops every source file that doesn't match at least one of them, and string-glob matchers also expand on disk so files that exist but were never required during the run still appear in the report (at 0% coverage). Resolves the long-standing requests in #696 and #869. The companion `SimpleCov.no_default_skips` opts out of the filters that `SimpleCov.start` installs (hidden files, `vendor/bundle/`, test directories) so users who want to opt out wholesale don't have to call `clear_filters` themselves.

‎Gemfile‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ group :development do
77
gem "capybara"
88
gem "cucumber"
99
gem "cuprite"
10+
gem "json_schemer"
1011
gem "nokogiri"
1112
gem "rackup"
1213
gem "rake"

‎Gemfile.lock‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ GEM
7373
websocket-driver (~> 0.7)
7474
ffi (1.17.4)
7575
ffi (1.17.4-java)
76+
hana (1.3.7)
7677
io-console (0.8.2)
7778
io-console (0.8.2-java)
7879
irb (1.18.0)
@@ -83,6 +84,11 @@ GEM
8384
jar-dependencies (0.5.7)
8485
json (2.19.5)
8586
json (2.19.5-java)
87+
json_schemer (2.5.0)
88+
bigdecimal
89+
hana (~> 1.3)
90+
regexp_parser (~> 2.0)
91+
simpleidn (~> 0.2)
8692
language_server-protocol (3.17.0.5)
8793
lint_roller (1.1.0)
8894
logger (1.7.0)
@@ -169,6 +175,7 @@ GEM
169175
lint_roller (~> 1.1)
170176
rubocop (~> 1.81)
171177
ruby-progressbar (1.13.0)
178+
simpleidn (0.2.3)
172179
stringio (3.2.0)
173180
sys-uname (1.5.1)
174181
ffi (~> 1.1)
@@ -203,6 +210,7 @@ DEPENDENCIES
203210
capybara
204211
cucumber
205212
cuprite
213+
json_schemer
206214
nokogiri
207215
rackup
208216
rake

‎README.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1151,6 +1151,29 @@ SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter
11511151

11521152
> The JSON formatter was originally a separate gem called [simplecov_json_formatter](https://github.com/codeclimate-community/simplecov_json_formatter). It is now built in and loaded by default. Existing code that does `require "simplecov_json_formatter"` will continue to work.
11531153
1154+
### `coverage.json` schema
1155+
1156+
The `coverage.json` payload is treated as a public contract. Every emitted file carries a `meta.schema_version` (`"major.minor"`) describing which version of the schema it conforms to. A JSON Schema (draft-07) lives in the repo at [`schemas/coverage.schema.json`](schemas/coverage.schema.json); downstream tools can use it to validate inputs, generate types, or pin themselves to a known shape.
1157+
1158+
The schema version is independent of the gem version:
1159+
1160+
- Additive changes (new fields) bump the **minor** segment. Existing consumers keep working.
1161+
- Removals or shape changes bump the **major** segment.
1162+
1163+
The current version is **1.0**. Top-level structure:
1164+
1165+
```jsonc
1166+
{
1167+
"meta": { /* schema_version, simplecov_version, command_name, project_name, timestamp, root, branch_coverage, method_coverage */ },
1168+
"total": { /* aggregate stats for lines (and branches / methods when enabled) */ },
1169+
"coverage": { "<project-relative path>": { /* per-file lines, source, branches, methods, etc. */ } },
1170+
"groups": { "<group name>": { /* per-group stats + files */ } },
1171+
"errors": { /* minimum_coverage, minimum_coverage_by_file, minimum_coverage_by_group, maximum_coverage_drop violations */ }
1172+
}
1173+
```
1174+
1175+
The `.resultset.json` file is **not** schema'd — it's SimpleCov-internal and may change shape across releases. Build integrations on top of `coverage.json`.
1176+
11541177
## Running a suite from the command line
11551178

11561179
If your project has no `test_helper.rb` hook that calls `SimpleCov.start`

‎lib/simplecov/formatter/json_formatter/result_hash_formatter.rb‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,16 @@ def format_groups
3737
end
3838
end
3939

40+
# Bump SCHEMA_VERSION when the JSON shape changes. Additive
41+
# changes bump the minor segment; removals or shape changes bump
42+
# major. See schemas/coverage.schema.json for the contract this
43+
# version describes.
44+
SCHEMA_VERSION = "1.0"
45+
private_constant :SCHEMA_VERSION
46+
4047
def format_meta
4148
{
49+
schema_version: SCHEMA_VERSION,
4250
simplecov_version: SimpleCov::VERSION,
4351
command_name: @result.command_name,
4452
project_name: SimpleCov.project_name,

‎schemas/coverage.schema.json‎

Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"$id": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage.schema.json",
4+
"title": "SimpleCov coverage.json",
5+
"description": "Schema for the coverage.json file emitted by SimpleCov's JSONFormatter. Versioned independently of the gem; non-breaking additions bump the minor segment of schema_version, breaking changes bump the major segment.",
6+
"type": "object",
7+
"required": ["meta", "total", "coverage", "groups", "errors"],
8+
"additionalProperties": false,
9+
"properties": {
10+
"meta": {
11+
"type": "object",
12+
"required": [
13+
"schema_version",
14+
"simplecov_version",
15+
"command_name",
16+
"project_name",
17+
"timestamp",
18+
"root",
19+
"branch_coverage",
20+
"method_coverage"
21+
],
22+
"additionalProperties": false,
23+
"properties": {
24+
"schema_version": {
25+
"type": "string",
26+
"const": "1.0",
27+
"description": "Schema major.minor. Additive changes bump minor; breaking changes bump major. Update this `const` whenever the schema version is bumped so documents claiming a different version don't quietly validate against this contract."
28+
},
29+
"simplecov_version": {
30+
"type": "string",
31+
"description": "The version of the SimpleCov gem that produced this file."
32+
},
33+
"command_name": {"type": "string"},
34+
"project_name": {"type": "string"},
35+
"timestamp": {
36+
"type": "string",
37+
"format": "date-time",
38+
"description": "ISO 8601 timestamp with millisecond precision."
39+
},
40+
"root": {
41+
"type": "string",
42+
"description": "Absolute path to SimpleCov.root at the time of write."
43+
},
44+
"branch_coverage": {"type": "boolean"},
45+
"method_coverage": {"type": "boolean"}
46+
}
47+
},
48+
"total": {"$ref": "#/definitions/totals"},
49+
"coverage": {
50+
"type": "object",
51+
"description": "Map of project-relative file paths to per-file coverage data.",
52+
"additionalProperties": {"$ref": "#/definitions/source_file"}
53+
},
54+
"groups": {
55+
"type": "object",
56+
"description": "Map of group names to per-group totals plus the list of files in the group.",
57+
"additionalProperties": {"$ref": "#/definitions/group"}
58+
},
59+
"errors": {
60+
"type": "object",
61+
"description": "Threshold violations from minimum_coverage, minimum_coverage_by_file, minimum_coverage_by_group, and maximum_coverage_drop. Empty object when no thresholds were violated.",
62+
"additionalProperties": false,
63+
"properties": {
64+
"minimum_coverage": {
65+
"type": "object",
66+
"description": "Keyed by criterion: lines, branches, methods.",
67+
"additionalProperties": {"$ref": "#/definitions/expected_actual"}
68+
},
69+
"minimum_coverage_by_file": {
70+
"type": "object",
71+
"description": "Keyed by criterion, then by project-relative filename.",
72+
"additionalProperties": {
73+
"type": "object",
74+
"additionalProperties": {"$ref": "#/definitions/expected_actual"}
75+
}
76+
},
77+
"minimum_coverage_by_group": {
78+
"type": "object",
79+
"description": "Keyed by group name, then by criterion.",
80+
"additionalProperties": {
81+
"type": "object",
82+
"additionalProperties": {"$ref": "#/definitions/expected_actual"}
83+
}
84+
},
85+
"maximum_coverage_drop": {
86+
"type": "object",
87+
"description": "Keyed by criterion: lines, branches, methods.",
88+
"additionalProperties": {"$ref": "#/definitions/maximum_actual"}
89+
}
90+
}
91+
}
92+
},
93+
"definitions": {
94+
"totals": {
95+
"type": "object",
96+
"required": ["lines"],
97+
"additionalProperties": false,
98+
"properties": {
99+
"lines": {"$ref": "#/definitions/line_statistic"},
100+
"branches": {"$ref": "#/definitions/coverage_statistic"},
101+
"methods": {"$ref": "#/definitions/coverage_statistic"}
102+
}
103+
},
104+
"source_file": {
105+
"type": "object",
106+
"required": ["lines", "source", "lines_covered_percent", "covered_lines", "missed_lines", "total_lines"],
107+
"additionalProperties": false,
108+
"properties": {
109+
"lines": {
110+
"type": "array",
111+
"description": "Per-source-line coverage. Element index N corresponds to source line N+1. Integer hit-count, null for non-relevant (blank/comment) lines, or the string \"ignored\" for lines inside a simplecov:disable / :nocov: region.",
112+
"items": {"$ref": "#/definitions/line_coverage"}
113+
},
114+
"source": {
115+
"type": "array",
116+
"description": "Source lines, in order. Same length as the lines array.",
117+
"items": {"type": "string"}
118+
},
119+
"lines_covered_percent": {"type": "number"},
120+
"covered_lines": {"type": "integer", "minimum": 0},
121+
"missed_lines": {"type": "integer", "minimum": 0},
122+
"total_lines": {"type": "integer", "minimum": 0},
123+
"branches": {
124+
"type": "array",
125+
"items": {"$ref": "#/definitions/branch"}
126+
},
127+
"branches_covered_percent": {"type": "number"},
128+
"covered_branches": {"type": "integer", "minimum": 0},
129+
"missed_branches": {"type": "integer", "minimum": 0},
130+
"total_branches": {"type": "integer", "minimum": 0},
131+
"methods": {
132+
"type": "array",
133+
"items": {"$ref": "#/definitions/method"}
134+
},
135+
"methods_covered_percent": {"type": "number"},
136+
"covered_methods": {"type": "integer", "minimum": 0},
137+
"missed_methods": {"type": "integer", "minimum": 0},
138+
"total_methods": {"type": "integer", "minimum": 0}
139+
}
140+
},
141+
"branch": {
142+
"type": "object",
143+
"required": ["type", "start_line", "end_line", "coverage", "inline", "report_line"],
144+
"additionalProperties": false,
145+
"properties": {
146+
"type": {
147+
"type": "string",
148+
"description": "Branch kind from Ruby's Coverage library (e.g. \"then\", \"else\", \"when\")."
149+
},
150+
"start_line": {"type": "integer", "minimum": 1},
151+
"end_line": {"type": "integer", "minimum": 1},
152+
"coverage": {"$ref": "#/definitions/line_coverage"},
153+
"inline": {"type": "boolean"},
154+
"report_line": {"type": "integer", "minimum": 1}
155+
}
156+
},
157+
"method": {
158+
"type": "object",
159+
"required": ["name", "start_line", "end_line", "coverage"],
160+
"additionalProperties": false,
161+
"properties": {
162+
"name": {
163+
"type": "string",
164+
"description": "Qualified method name, e.g. \"Foo#bar\", \"Foo.bar\", or \"Foo::Bar#baz\"."
165+
},
166+
"start_line": {"type": "integer", "minimum": 1},
167+
"end_line": {"type": "integer", "minimum": 1},
168+
"coverage": {"$ref": "#/definitions/line_coverage"}
169+
}
170+
},
171+
"group": {
172+
"type": "object",
173+
"required": ["lines", "files"],
174+
"additionalProperties": false,
175+
"properties": {
176+
"lines": {"$ref": "#/definitions/line_statistic"},
177+
"branches": {"$ref": "#/definitions/coverage_statistic"},
178+
"methods": {"$ref": "#/definitions/coverage_statistic"},
179+
"files": {
180+
"type": "array",
181+
"description": "Project-relative paths of the files that fell into this group.",
182+
"items": {"type": "string"}
183+
}
184+
}
185+
},
186+
"line_statistic": {
187+
"type": "object",
188+
"required": ["covered", "missed", "omitted", "total", "percent", "strength"],
189+
"additionalProperties": false,
190+
"properties": {
191+
"covered": {"type": "integer", "minimum": 0},
192+
"missed": {"type": "integer", "minimum": 0},
193+
"omitted": {
194+
"type": "integer",
195+
"minimum": 0,
196+
"description": "Lines that cannot be covered (blank, comment, etc.). Only present on line stats."
197+
},
198+
"total": {"type": "integer", "minimum": 0},
199+
"percent": {"type": "number"},
200+
"strength": {"type": "number"}
201+
}
202+
},
203+
"coverage_statistic": {
204+
"type": "object",
205+
"required": ["covered", "missed", "total", "percent", "strength"],
206+
"additionalProperties": false,
207+
"properties": {
208+
"covered": {"type": "integer", "minimum": 0},
209+
"missed": {"type": "integer", "minimum": 0},
210+
"total": {"type": "integer", "minimum": 0},
211+
"percent": {"type": "number"},
212+
"strength": {"type": "number"}
213+
}
214+
},
215+
"line_coverage": {
216+
"description": "Integer hit-count for executable lines/branches/methods, null for non-relevant lines, or the literal string \"ignored\" for code inside a simplecov:disable region.",
217+
"oneOf": [
218+
{"type": "integer", "minimum": 0},
219+
{"type": "null"},
220+
{"type": "string", "const": "ignored"}
221+
]
222+
},
223+
"expected_actual": {
224+
"type": "object",
225+
"required": ["expected", "actual"],
226+
"additionalProperties": false,
227+
"properties": {
228+
"expected": {"type": "number"},
229+
"actual": {"type": "number"}
230+
}
231+
},
232+
"maximum_actual": {
233+
"type": "object",
234+
"required": ["maximum", "actual"],
235+
"additionalProperties": false,
236+
"properties": {
237+
"maximum": {"type": "number"},
238+
"actual": {"type": "number"}
239+
}
240+
}
241+
}
242+
}

‎simplecov.gemspec‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ Gem::Specification.new do |gem|
3636

3737
gem.required_ruby_version = ">= 3.1"
3838

39-
gem.files = Dir["lib/**/*.*", "exe/*", "LICENSE", "CHANGELOG.md", "README.md", "doc/*"]
39+
gem.files = Dir["{lib,schemas}/**/*.*", "exe/*", "LICENSE", "CHANGELOG.md", "README.md", "doc/*"]
4040
gem.bindir = "exe"
4141
gem.executables = ["simplecov"]
4242
gem.require_paths = ["lib"]

‎spec/fixtures/json/sample.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"meta": {
3+
"schema_version": "1.0",
34
"simplecov_version": "0.22.0",
45
"command_name": "STUB_COMMAND_NAME",
56
"project_name": "STUB_PROJECT_NAME",

‎spec/fixtures/json/sample_groups.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"meta": {
3+
"schema_version": "1.0",
34
"simplecov_version": "0.22.0",
45
"command_name": "STUB_COMMAND_NAME",
56
"project_name": "STUB_PROJECT_NAME",

‎spec/fixtures/json/sample_with_branch.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"meta": {
3+
"schema_version": "1.0",
34
"simplecov_version": "0.22.0",
45
"command_name": "STUB_COMMAND_NAME",
56
"project_name": "STUB_PROJECT_NAME",

0 commit comments

Comments
 (0)