Skip to content

Commit b2a6639

Browse files
wu-shengwu-sheng
authored
fix: enforce distinct matching in contains and add containsOnce with backtracking (#143)
Fix a bug where one actual item could satisfy multiple expected entries in `contains`, causing false-positive test results (apache/skywalking#8752). Now each actual item can only be claimed once (greedy distinct matching). Add `containsOnce` keyword that uses backtracking to find optimal assignment between expected entries and actual items, solving cases where greedy ordering fails.
1 parent e26033e commit b2a6639

7 files changed

Lines changed: 460 additions & 8 deletions

File tree

‎CLAUDE.md‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,4 +152,13 @@ verify:
152152
### GitHub Actions Integration
153153

154154
- `action.yaml` at project root defines the composite action
155-
- Inputs: e2e-file, log-dir, plus matrix vars for log isolation
155+
- Inputs: e2e-file, log-dir, plus matrix vars for log isolation
156+
157+
## GitHub Actions Allow List
158+
159+
Apache enforces an allow list for third-party GitHub Actions. All third-party actions must be pinned to an approved SHA from:
160+
https://github.com/apache/infrastructure-actions/blob/main/approved_patterns.yml
161+
162+
If a PR is blocked by "action is not allowed" errors, check the approved list and update `.github/workflows/` files to use the approved SHA pin instead of a version tag.
163+
164+
Actions owned by `actions/*` (GitHub), `github/*`, and `apache/*` are always allowed (enterprise-owned).

‎docs/en/setup/Configuration-File.md‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -220,16 +220,34 @@ Verify that the number fits the range.
220220

221221
##### List Matches
222222

223-
Verify the data in the condition list, Currently, it is only supported when all the conditions in the list are executed, it is considered as successful.
223+
Two keywords are available for verifying lists: `contains` and `containsOnce`.
224224

225-
Here is an example, It's means the list values must have value is greater than 0, also have value greater than 1, Otherwise verify is failure.
225+
**`contains`** checks that each expected entry matches a distinct actual item using greedy matching (first-come, first-served order).
226+
Each expected entry can only claim one actual item, and each actual item can only be claimed once.
227+
Extra actual items beyond the expected entries are allowed.
228+
229+
Here is an example, it means the list values must have an item with value greater than 0 and another item with value greater than 1. Otherwise verification fails.
226230
```yaml
227231
{{- contains .list }}
228232
- key: {{ gt .value 0 }}
229233
- key: {{ gt .value 1 }}
230234
{{- end }}
231235
```
232236

237+
**`containsOnce`** provides the same semantics as `contains` but uses backtracking to find a valid assignment.
238+
This is useful when the greedy order of `contains` cannot find a valid match but one exists.
239+
For example, if a generic expected entry (e.g., `notEmpty`) appears before a specific one (e.g., a hardcoded value),
240+
`contains` might greedily claim the wrong actual item, while `containsOnce` will try all combinations to find a valid assignment.
241+
242+
```yaml
243+
{{- containsOnce .list }}
244+
- name: {{ notEmpty .name }}
245+
language: {{ notEmpty .language }}
246+
- name: {{ notEmpty .name }}
247+
language: JAVA
248+
{{- end }}
249+
```
250+
233251
##### Encoding
234252

235253
In order to make the program easier for users to read and use, some code conversions are provided.

‎internal/components/verifier/verifier_test.go‎

Lines changed: 230 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -272,6 +272,236 @@ metrics:
272272
},
273273
wantErr: false,
274274
}, {
275+
name: "contains should require distinct actual items for each expected entry",
276+
args: args{
277+
actualData: `
278+
metrics:
279+
- name: service-A
280+
id: abc
281+
`,
282+
expectedTemplate: `
283+
metrics:
284+
{{- contains .metrics }}
285+
- name: {{ notEmpty .name }}
286+
id: {{ notEmpty .id }}
287+
- name: {{ notEmpty .name }}
288+
id: {{ notEmpty .id }}
289+
{{- end }}
290+
`,
291+
},
292+
wantErr: true, // expected 2 entries but actual only has 1
293+
},
294+
{
295+
name: "contains should not pass when extra expected entries are added with echo-only conditions",
296+
args: args{
297+
actualData: `
298+
metrics:
299+
- name: service-A
300+
id: abc
301+
- name: service-B
302+
id: def
303+
`,
304+
expectedTemplate: `
305+
metrics:
306+
{{- contains .metrics }}
307+
- name: {{ notEmpty .name }}
308+
id: {{ notEmpty .id }}
309+
- name: {{ notEmpty .name }}
310+
id: {{ notEmpty .id }}
311+
- name: {{ notEmpty .name }}
312+
id: {{ notEmpty .id }}
313+
{{- end }}
314+
`,
315+
},
316+
wantErr: true, // expected 3 entries but actual only has 2
317+
},
318+
{
319+
name: "contains greedy cannot solve reordered assignment",
320+
args: args{
321+
actualData: `
322+
- name: service-A
323+
language: JAVA
324+
- name: service-B
325+
language: GO
326+
`,
327+
expectedTemplate: `
328+
{{- contains . }}
329+
- name: {{ notEmpty .name }}
330+
language: {{ notEmpty .language }}
331+
- name: {{ notEmpty .name }}
332+
language: JAVA
333+
{{- end }}
334+
`,
335+
},
336+
wantErr: true, // greedy contains can't solve this; containsOnce with backtracking will
337+
},
338+
{
339+
name: "containsOnce should backtrack to find valid assignment",
340+
args: args{
341+
actualData: `
342+
- name: service-A
343+
language: JAVA
344+
- name: service-B
345+
language: GO
346+
`,
347+
expectedTemplate: `
348+
{{- containsOnce . }}
349+
- name: {{ notEmpty .name }}
350+
language: {{ notEmpty .language }}
351+
- name: {{ notEmpty .name }}
352+
language: JAVA
353+
{{- end }}
354+
`,
355+
},
356+
wantErr: false, // backtracking finds: expected[0]→actual[1], expected[1]→actual[0]
357+
},
358+
{
359+
name: "containsOnce should require distinct actual items",
360+
args: args{
361+
actualData: `
362+
metrics:
363+
- name: service-A
364+
id: abc
365+
`,
366+
expectedTemplate: `
367+
metrics:
368+
{{- containsOnce .metrics }}
369+
- name: {{ notEmpty .name }}
370+
id: {{ notEmpty .id }}
371+
- name: {{ notEmpty .name }}
372+
id: {{ notEmpty .id }}
373+
{{- end }}
374+
`,
375+
},
376+
wantErr: true, // 1 actual item cannot satisfy 2 expected entries
377+
},
378+
{
379+
name: "containsOnce should pass when enough distinct items match",
380+
args: args{
381+
actualData: `
382+
metrics:
383+
- name: service-A
384+
id: abc
385+
- name: service-B
386+
id: def
387+
- name: service-C
388+
id: ghi
389+
`,
390+
expectedTemplate: `
391+
metrics:
392+
{{- containsOnce .metrics }}
393+
- name: {{ notEmpty .name }}
394+
id: {{ notEmpty .id }}
395+
- name: {{ notEmpty .name }}
396+
id: {{ notEmpty .id }}
397+
{{- end }}
398+
`,
399+
},
400+
wantErr: false, // 3 actual items, 2 expected → enough distinct matches
401+
},
402+
{
403+
name: "containsOnce should fail when specific value missing",
404+
args: args{
405+
actualData: `
406+
- name: service-B
407+
value: "200"
408+
- name: service-C
409+
value: "300"
410+
`,
411+
expectedTemplate: `
412+
{{- containsOnce . }}
413+
- name: service-A
414+
value: "100"
415+
{{- end }}
416+
`,
417+
},
418+
wantErr: true, // service-A does not exist
419+
},
420+
{
421+
name: "contains should match specific values in reversed order",
422+
args: args{
423+
actualData: `
424+
metrics:
425+
- name: service-B
426+
value: "200"
427+
- name: service-A
428+
value: "100"
429+
`,
430+
expectedTemplate: `
431+
metrics:
432+
{{- contains .metrics }}
433+
- name: service-A
434+
value: "100"
435+
- name: service-B
436+
value: "200"
437+
{{- end }}
438+
`,
439+
},
440+
wantErr: false, // both exist, order shouldn't matter
441+
},
442+
{
443+
name: "contains should fail when one specific value is wrong",
444+
args: args{
445+
actualData: `
446+
metrics:
447+
- name: service-A
448+
value: "100"
449+
- name: service-B
450+
value: "200"
451+
`,
452+
expectedTemplate: `
453+
metrics:
454+
{{- contains .metrics }}
455+
- name: service-A
456+
value: "100"
457+
- name: service-C
458+
value: "300"
459+
{{- end }}
460+
`,
461+
},
462+
wantErr: true, // service-C does not exist in actual
463+
},
464+
{
465+
name: "contains should match unordered actual data correctly",
466+
args: args{
467+
actualData: `
468+
metrics:
469+
- name: service-B
470+
value: 200
471+
- name: service-A
472+
value: 100
473+
`,
474+
expectedTemplate: `
475+
metrics:
476+
{{- contains .metrics }}
477+
- name: service-A
478+
value: 100
479+
{{- end }}
480+
`,
481+
},
482+
wantErr: false, // service-A exists in actual, order shouldn't matter
483+
},
484+
{
485+
name: "contains should fail when specific expected value is missing from actual",
486+
args: args{
487+
actualData: `
488+
metrics:
489+
- name: service-B
490+
value: 200
491+
- name: service-C
492+
value: 300
493+
`,
494+
expectedTemplate: `
495+
metrics:
496+
{{- contains .metrics }}
497+
- name: service-A
498+
value: 100
499+
{{- end }}
500+
`,
501+
},
502+
wantErr: true, // service-A does not exist in actual
503+
},
504+
{
275505
name: "notEmpty with nil",
276506
args: args{
277507
actualData: `

0 commit comments

Comments
 (0)