Skip to content

Commit 9be47f4

Browse files
norberttechnorberttech
committed
fix(flow-php/etl): SchemaValidationException no longer lists columns accepted by the validator and renders the nullable marker on expected types (#2496
- SchemaValidator::isValid() replaced by validate() returning ValidationContext with missing/mismatched/unexpected definitions - exception message rendered from validator findings via ValidationContext::toString(), FROM_NULL columns no longer reported - schema_validate() DSL returns ValidationContext instead of bool - upgrading.md entry for the SchemaValidator API break - allow brick/math ^0.18
1 parent 4fb29ec commit 9be47f4

21 files changed

Lines changed: 682 additions & 263 deletions

‎composer.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
"ext-zlib": "*",
2323
"composer-runtime-api": "^2.2",
2424
"async-aws/s3": "^2.6 || ^3.0",
25-
"brick/math": "^0.12 || ^0.13 || ^0.14 || ^0.15 || ^0.16 || ^0.17",
25+
"brick/math": "^0.12 || ^0.13 || ^0.14 || ^0.15 || ^0.16 || ^0.17 || ^0.18",
2626
"cmsig/seal": "^0.12",
2727
"coduo/php-humanizer": "^5.0",
2828
"cuyz/valinor": "^2.4",

‎composer.lock‎

Lines changed: 7 additions & 7 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎documentation/upgrading.md‎

Lines changed: 23 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -226,25 +226,38 @@ Applies to the HttpKernel server span and to the `http_client`/`psr18_client` cl
226226

227227
### 15) `flow-php/symfony-telemetry-bundle` - `runtime_mode` removed; terminate flushes, process end shuts down
228228

229-
| Before | After |
230-
|------------------------------------------------------------------------------|---------------------------------------------|
231-
| `flow_telemetry.runtime_mode: auto`/`classic`/`worker` | removed |
232-
| `Flow\Bridge\Symfony\TelemetryBundle\Runtime\RuntimeModeResolver` | removed |
233-
| `Flow\Bridge\Symfony\TelemetryBundle\Runtime\RuntimeMode` | removed |
234-
| `Flow\Bridge\Symfony\TelemetryBundle\Runtime\WorkerModeDetector` | removed |
235-
| `Flow\Bridge\Symfony\TelemetryBundle\Runtime\EnvironmentWorkerModeDetector` | removed |
236-
| shutdown on `kernel.terminate`/`console.terminate` (classic mode) | flush on terminate; shutdown at process end |
229+
| Before | After |
230+
|-----------------------------------------------------------------------------|---------------------------------------------|
231+
| `flow_telemetry.runtime_mode: auto`/`classic`/`worker` | removed |
232+
| `Flow\Bridge\Symfony\TelemetryBundle\Runtime\RuntimeModeResolver` | removed |
233+
| `Flow\Bridge\Symfony\TelemetryBundle\Runtime\RuntimeMode` | removed |
234+
| `Flow\Bridge\Symfony\TelemetryBundle\Runtime\WorkerModeDetector` | removed |
235+
| `Flow\Bridge\Symfony\TelemetryBundle\Runtime\EnvironmentWorkerModeDetector` | removed |
236+
| shutdown on `kernel.terminate`/`console.terminate` (classic mode) | flush on terminate; shutdown at process end |
237237

238238
Drop the `runtime_mode` key from `flow_telemetry` config and remove any `WorkerModeDetector` service overrides.
239239

240240
### 16) `flow-php/telemetry` - `Telemetry::registerShutdownFunction()` holds a weak reference
241241

242-
| Before | After |
243-
|--------------------------------------------------|----------------------------------------------------------------|
242+
| Before | After |
243+
|--------------------------------------------------|---------------------------------------------------------------|
244244
| strong reference; instance kept alive until exit | weak reference; garbage-collected instances are not shut down |
245245

246246
Keep the registered `Telemetry` instance referenced for as long as it should be shut down at process end.
247247

248+
### 17) `flow-php/etl` - `SchemaValidator::isValid()` replaced by `validate(): ValidationContext`
249+
250+
| Before | After |
251+
|-----------------------------------------------|------------------------------------------------------------------|
252+
| `SchemaValidator::isValid(...): bool` | `SchemaValidator::validate(...): ValidationContext` |
253+
| `$validator->isValid($expected, $given)` | `$validator->validate($expected, $given)->isValid()` |
254+
| `schema_validate(...): bool` | `schema_validate(...): ValidationContext` |
255+
| `new SchemaValidationException($exp, $given)` | `new SchemaValidationException($exp, $given, ValidationContext)` |
256+
| — | `SchemaValidationException::context(): ValidationContext` |
257+
258+
Custom `SchemaValidator` implementations must return a `Flow\ETL\Schema\Validator\ValidationContext`
259+
built from the missing, mismatched (`MismatchedDefinition`), and unexpected definitions they reject.
260+
248261
---
249262

250263
## Upgrading from 0.40.x to 0.41.x

‎src/core/etl/composer.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
"php": "~8.3.0 || ~8.4.0 || ~8.5.0",
1313
"ext-json": "*",
1414
"psr/clock": "^1.0",
15-
"brick/math": "^0.12 || ^0.13 || ^0.14 || ^0.15 || ^0.16 || ^0.17",
15+
"brick/math": "^0.12 || ^0.13 || ^0.14 || ^0.15 || ^0.16 || ^0.17 || ^0.18",
1616
"composer-runtime-api": "^2.2",
1717
"flow-php/telemetry": "self.version",
1818
"flow-php/types": "self.version",

‎src/core/etl/src/Flow/ETL/DSL/functions.php‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,7 @@
217217
use Flow\ETL\Schema\Validator\EvolvingValidator;
218218
use Flow\ETL\Schema\Validator\SelectiveValidator;
219219
use Flow\ETL\Schema\Validator\StrictValidator;
220+
use Flow\ETL\Schema\Validator\ValidationContext;
220221
use Flow\ETL\SchemaValidator;
221222
use Flow\ETL\String\StringStyles;
222223
use Flow\ETL\Time\Duration;
@@ -1900,9 +1901,12 @@ function schema_to_ascii(Schema $schema, ?SchemaFormatter $formatter = null): st
19001901
* @param Schema $given
19011902
*/
19021903
#[DocumentationDSL(module: Module::CORE, type: DSLType::HELPER)]
1903-
function schema_validate(Schema $expected, Schema $given, SchemaValidator $validator = new StrictValidator()): bool
1904-
{
1905-
return $validator->isValid($expected, $given);
1904+
function schema_validate(
1905+
Schema $expected,
1906+
Schema $given,
1907+
SchemaValidator $validator = new StrictValidator(),
1908+
): ValidationContext {
1909+
return $validator->validate($expected, $given);
19061910
}
19071911

19081912
#[DocumentationDSL(module: Module::CORE, type: DSLType::HELPER)]

‎src/core/etl/src/Flow/ETL/Exception/SchemaValidationException.php‎

Lines changed: 7 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -5,98 +5,21 @@
55
namespace Flow\ETL\Exception;
66

77
use Flow\ETL\Schema;
8-
9-
use function count;
8+
use Flow\ETL\Schema\Validator\ValidationContext;
109

1110
final class SchemaValidationException extends RuntimeException
1211
{
1312
public function __construct(
1413
private readonly Schema $expected,
1514
private readonly Schema $given,
15+
private readonly ValidationContext $context,
1616
) {
17-
/**
18-
* @var array<string> $missingDefinitions
19-
*/
20-
$missingDefinitions = [];
21-
22-
/**
23-
* @var array<string> $mismatchedDefinitions
24-
*/
25-
$mismatchedDefinitions = [];
26-
27-
/**
28-
* @var array<string> $unexpectedDefinitions
29-
*/
30-
$unexpectedDefinitions = [];
31-
32-
foreach ($this->expected->definitions() as $expectedDefinition) {
33-
$givenDefinition = $this->given->findDefinition($expectedDefinition->entry());
34-
35-
if ($givenDefinition === null) {
36-
$missingDefinitions[] =
37-
$expectedDefinition->entry()->name()
38-
. '<'
39-
. ($expectedDefinition->isNullable() ? '?' : '')
40-
. $expectedDefinition->type()->toString()
41-
. '>';
42-
43-
continue;
44-
}
45-
46-
if (!$expectedDefinition->isCompatible($givenDefinition)) {
47-
$mismatchedDefinitions[] =
48-
'expected: '
49-
. $expectedDefinition->entry()->name()
50-
. '<'
51-
. $expectedDefinition->type()->toString()
52-
. '>, '
53-
. 'given: '
54-
. $givenDefinition->entry()->name()
55-
. '<'
56-
. ($givenDefinition->isNullable() ? '?' : '')
57-
. $givenDefinition->type()->toString()
58-
. '>';
59-
}
60-
}
61-
62-
foreach ($this->given->definitions() as $givenDefinition) {
63-
if ($this->expected->findDefinition($givenDefinition->entry()) === null) {
64-
$unexpectedDefinitions[] =
65-
$givenDefinition->entry()->name()
66-
. '<'
67-
. ($givenDefinition->isNullable() ? '?' : '')
68-
. $givenDefinition->type()->toString()
69-
. '>';
70-
}
71-
}
72-
73-
$message = '';
74-
75-
if (count($missingDefinitions)) {
76-
$message .= " Missing Definitions: \n";
77-
78-
foreach ($missingDefinitions as $missingDefinition) {
79-
$message .= ' |-- ' . $missingDefinition . "\n";
80-
}
81-
}
82-
83-
if (count($mismatchedDefinitions)) {
84-
$message .= " Mismatched Definitions: \n";
85-
86-
foreach ($mismatchedDefinitions as $mismatchedDefinition) {
87-
$message .= ' |-- ' . $mismatchedDefinition . "\n";
88-
}
89-
}
90-
91-
if (count($unexpectedDefinitions)) {
92-
$message .= " Unexpected Definitions: \n";
93-
94-
foreach ($unexpectedDefinitions as $unexpectedDefinition) {
95-
$message .= ' |-- ' . $unexpectedDefinition . "\n";
96-
}
97-
}
17+
parent::__construct("Schema validation failed: \n" . $this->context->toString());
18+
}
9819

99-
parent::__construct("Schema validation failed: \n" . $message);
20+
public function context(): ValidationContext
21+
{
22+
return $this->context;
10023
}
10124

10225
/**

‎src/core/etl/src/Flow/ETL/Loader/SchemaValidationLoader.php‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,10 @@ public function load(Rows $rows, FlowContext $context): void
2727
try {
2828
$given = $rows->schema();
2929

30-
if (!$this->validator->isValid($this->expected, $given)) {
31-
throw new SchemaValidationException($this->expected, $given);
30+
$validation = $this->validator->validate($this->expected, $given);
31+
32+
if (!$validation->isValid()) {
33+
throw new SchemaValidationException($this->expected, $given, $validation);
3234
}
3335

3436
$context->telemetry()->loadingCompleted($this, [TelemetryAttributes::ATTR_LOADING_ROWS => $rows->count()]);

‎src/core/etl/src/Flow/ETL/Schema/Validator/EvolvingValidator.php‎

Lines changed: 14 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -20,38 +20,31 @@
2020
*/
2121
final class EvolvingValidator implements SchemaValidator
2222
{
23-
/**
24-
* @param Schema $expected
25-
* @param Schema $given
26-
*/
27-
public function isValid(Schema $expected, Schema $given): bool
23+
public function validate(Schema $expected, Schema $given): ValidationContext
2824
{
29-
if ($given->count() < $expected->count()) {
30-
return false;
31-
}
25+
$missingDefinitions = [];
26+
$mismatchedDefinitions = [];
3227

33-
foreach ($expected->definitions() as $definition) {
34-
if ($given->findDefinition($definition->entry()) === null) {
35-
return false;
36-
}
37-
}
28+
foreach ($expected->definitions() as $expectedDefinition) {
29+
$givenDefinition = $given->findDefinition($expectedDefinition->entry());
3830

39-
foreach ($given->definitions() as $rightDefinition) {
40-
$leftDefinition = $expected->findDefinition($rightDefinition->entry());
31+
if ($givenDefinition === null) {
32+
$missingDefinitions[] = $expectedDefinition;
4133

42-
if ($leftDefinition === null) {
4334
continue;
4435
}
4536

46-
if (!$rightDefinition->isNullable() && $leftDefinition->isNullable()) {
47-
return false;
37+
if (!$givenDefinition->isNullable() && $expectedDefinition->isNullable()) {
38+
$mismatchedDefinitions[] = new MismatchedDefinition($expectedDefinition, $givenDefinition);
39+
40+
continue;
4841
}
4942

50-
if (!type_equals($rightDefinition->type(), $leftDefinition->type())) {
51-
return false;
43+
if (!type_equals($givenDefinition->type(), $expectedDefinition->type())) {
44+
$mismatchedDefinitions[] = new MismatchedDefinition($expectedDefinition, $givenDefinition);
5245
}
5346
}
5447

55-
return true;
48+
return new ValidationContext($missingDefinitions, $mismatchedDefinitions);
5649
}
5750
}

‎src/core/etl/src/Flow/ETL/Schema/Validator/MismatchedDefinition.php‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
<?php
2+
3+
declare(strict_types=1);
4+
5+
namespace Flow\ETL\Schema\Validator;
6+
7+
use Flow\ETL\Schema\Definition;
8+
9+
final readonly class MismatchedDefinition
10+
{
11+
/**
12+
* @param Definition<mixed> $expected
13+
* @param Definition<mixed> $given
14+
*/
15+
public function __construct(
16+
private Definition $expected,
17+
private Definition $given,
18+
) {}
19+
20+
/**
21+
* @return Definition<mixed>
22+
*/
23+
public function expected(): Definition
24+
{
25+
return $this->expected;
26+
}
27+
28+
/**
29+
* @return Definition<mixed>
30+
*/
31+
public function given(): Definition
32+
{
33+
return $this->given;
34+
}
35+
}

‎src/core/etl/src/Flow/ETL/Schema/Validator/SelectiveValidator.php‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,18 @@
1616
*/
1717
final class SelectiveValidator implements SchemaValidator
1818
{
19-
public function isValid(Schema $expected, Schema $given): bool
19+
public function validate(Schema $expected, Schema $given): ValidationContext
2020
{
21+
$missingDefinitions = [];
22+
$mismatchedDefinitions = [];
23+
2124
foreach ($expected->definitions() as $expectedDefinition) {
2225
$givenDefinition = $given->findDefinition($expectedDefinition->entry());
2326

2427
if ($givenDefinition === null) {
25-
return false;
28+
$missingDefinitions[] = $expectedDefinition;
29+
30+
continue;
2631
}
2732

2833
if (
@@ -34,10 +39,10 @@ public function isValid(Schema $expected, Schema $given): bool
3439
}
3540

3641
if (!$expectedDefinition->isCompatible($givenDefinition)) {
37-
return false;
42+
$mismatchedDefinitions[] = new MismatchedDefinition($expectedDefinition, $givenDefinition);
3843
}
3944
}
4045

41-
return true;
46+
return new ValidationContext($missingDefinitions, $mismatchedDefinitions);
4247
}
4348
}

0 commit comments

Comments
 (0)