`generator.valueType` false positives: oversized-array generalization produces types that reject their own contributors

[gnutix]

Filing this at @VincentLanglet's request on phpstan/phpstan-src#5568.

Summary

PHPStan 2.1.52+ produces generator.valueType false positives where a closure's inferred Generator value type rejects the very yields it was synthesised from. The oversized-array generalization in TypeCombinator::optimizeConstantArrays produces a type that is not a super-type of the variants it was derived from.

The error is self-inconsistent: PHPStan complains that each yielded value (with a perfectly valid concrete shape) does not match the Generator value type — but that inferred value type is synthesized from those very same yields. A function cannot legitimately yield a value its own inferred return type rejects; the oversized-array generalization step is producing a type that fails to be a super-type of its own contributors.

Symptom signature: the inferred type contains &oversized-array markers on shapes that are themselves empty (array{}&oversized-array) or fully-generalized array<intKey, V>&accessories mixed in the same union as preserved array{...}&oversized-array records.

Reproducer

https://phpstan.org/r/5b811780-35b4-4ebf-b2a6-cecb90f02b2b

(level: max, bleedingEdge: true, treatPhpDocTypesAsCertain: true.)

A standalone version with non-numeric-string yield keys is included in the PR's test data file: bug-yield-oversized-self-rejection.php.

Affected versions

Version	Result
2.1.51	OK
2.1.52	9 errors (regression)
2.1.53	9 errors
2.1.54	9 errors (still broken)
with #5568	0 errors

Introduced by 2f66c45222 ("Preserve constant array when assigning a union of scalars") in 2.1.52. That commit is itself a correct precision improvement; it exposed two latent bugs in TypeCombinator downstream.

[phpstan-bot]

@gnutix After the latest push in 2.2.x, PHPStan now reports different result with your code snippet:

@@ @@
-17: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}}}, lookup: array{}, targets: array{array{1, '2022-08-04'}}} given.
-23: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{1, '2022-08-04', array{array{'08:00', '12:00'}, array{'14:00', '18:00'}}}, y?: array{1, '2022-08-04', array{array{'08:00', '12:00'}, array{'14:00', '18:00'}}}}}, lookup: array{}, targets: array{array{1, '2022-08-04'}}} given.
-29: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{1, '2022-08-04', array{array{'00:00', '00:00'}}}, y?: array{1, '2022-08-04', array{array{'00:00', '00:00'}}}}}, lookup: array{}, targets: array{array{1, '2022-08-04'}}} given.
-35: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{1, '2022-08-04', array{array{'22:00', '04:00'}}}, y?: array{1, '2022-08-04', array{array{'22:00', '04:00'}}}}}, lookup: array{}, targets: array{array{1, '2022-08-04/2022-08-05'}}} given.
-41: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}, y?: array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}}}, lookup: array{42: array{1, '2022-08-05T00:05/2022-08-05T02:00'}}, targets: array{array{1, '2022-08-04/2022-08-05'}}} given.
-47: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}}, non-empty-array{x?: array{1, '2022-08-10', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-10', array{array{'08:00', '12:00'}}}}}, lookup: array{}, targets: array{array{1, '2022-08-04'}, array{1, '2022-08-10'}}} given.
-56: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}}, non-empty-array{x?: array{1, '2022-08-05', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-05', array{array{'08:00', '12:00'}}}}, non-empty-array{x?: array{1, '2022-08-06', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-06', array{array{'08:00', '12:00'}}}}}, lookup: array{}, targets: array{array{1, '2022-08-04/2022-08-06'}}} given.
-66: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}}, non-empty-array{x?: array{2, '2022-08-05', array{array{'08:00', '12:00'}}}, y?: array{2, '2022-08-05', array{array{'08:00', '12:00'}}}}, non-empty-array{x?: array{2, '2022-08-06', array{array{'08:00', '12:00'}}}, y?: array{2, '2022-08-06', array{array{'08:00', '12:00'}}}}, non-empty-array{x?: array{3, '2022-08-06', array{array{'08:00', '12:00'}}}, y?: array{3, '2022-08-06', array{array{'08:00', '12:00'}}}}, non-empty-array{x?: array{3, '2022-08-10', array{array{'08:00', '12:00'}}}, y?: array{3, '2022-08-10', array{array{'08:00', '12:00'}}}}}, lookup: array{}, targets: array{array{1, '2022-08-04'}, array{2, '2022-08-05/2022-08-06'}, array{3, '2022-08-06'}, array{3, '2022-08-10'}}} given.
-78: Generator expects value type non-empty-array<literal-string&lowercase-string&non-falsy-string, (array<int, non-empty-array<0|1|(literal-string&lowercase-string&non-falsy-string), 1|2|3|'2022-08-04'|'2022-08-04/2022-08-05'|'2022-08-04/2022-08-06'|'2022-08-05/2022-08-06'|'2022-08-05T00:05/2022-08-05T02:00'|'2022-08-06'|'2022-08-10'|(array{0: 1, 1: '2022-08-04', 2: array{array{'16:00', '23:00'}}, lookupIds: array{42}}&oversized-array)|(list{1|2|3, '2022-08-04'|'2022-08-05'|'2022-08-06'|'2022-08-10', array{0: array{'08:00', '12:00'}, 1?: array{'14:00', '18:00'}}|array{array{'00:00', '00:00'}}|array{array{'22:00', '04:00'}}}&oversized-array)>&oversized-array>&oversized-array)|bool|string>&oversized-array, array{kind: string, entries: array{non-empty-array{x?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-04', array{array{'08:00', '12:00'}}}}, non-empty-array{x?: array{1, '2022-08-05', array{array{'08:00', '12:00'}}}, y?: array{1, '2022-08-05', array{array{'08:00', '12:00'}}}}}, lookup: array{}, targets: array{}, flag: false} given.
+No errors