Skip to content

Commit 29dd101

Browse files
carlos-almcarlos-alm
committed
fix(bench): update elixir/julia/objc expected-edges to module-qualified names
The resolution benchmark uses WASM-built graphs where the Elixir, Julia, and Objective-C extractors emit module-qualified symbol names (Main.run, App.main, UserService.create_user, etc.). The expected-edges manifests were written with bare unqualified names (run, main, create_user), so every correctly-resolved edge appeared as a false positive and every expected edge appeared as a false negative — causing all three languages to show 0% precision even though resolution was working correctly. Root cause: starting in v3.12.0, cross-module call resolution began working for these languages (via the improved receiver-dispatch and same-class fallback in resolveByMethodOrGlobal / build-edges.ts). With 0 edges previously resolved, the name mismatch was invisible; once edges started resolving, the manifests showed 17 FP (elixir), 11 FP (julia), 6 FP (objc) — all correctly resolved edges misidentified as false positives. Fix: - Update all three expected-edges.json manifests to use the module-qualified names matching actual extractor output: elixir: Main.run, UserService.create_user, Validators.validate_user, etc. julia: App.main, Service.create_user, Repository.new_repo, etc. objc: full ObjC selectors (createUserWithId:name:email:, isValidEmail:, etc.) plus add main -> run (plain C call correctly resolved) - Ratchet THRESHOLDS for all three: elixir: precision 0.0 -> 1.0, recall 0.0 -> 0.8 (17/21 resolved) julia: precision 0.0 -> 1.0, recall 0.0 -> 0.7 (11/15 resolved) objc: precision 0.0 -> 1.0, recall 0.0 -> 0.4 (6/13 resolved) Remaining FNs are genuine unresolved edges (same-file bare calls in elixir/julia, receiver-typed message sends in objc) — not regressions. Closes #1447
1 parent 5f5d4d2 commit 29dd101

4 files changed

Lines changed: 144 additions & 126 deletions

File tree

‎tests/benchmarks/resolution/fixtures/elixir/expected-edges.json‎

Lines changed: 46 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -4,148 +4,148 @@
44
"description": "Hand-annotated call edges for Elixir resolution benchmark",
55
"edges": [
66
{
7-
"source": { "name": "validate_user", "file": "validators.ex" },
8-
"target": { "name": "valid_name?", "file": "validators.ex" },
7+
"source": { "name": "Validators.validate_user", "file": "validators.ex" },
8+
"target": { "name": "Validators.valid_name?", "file": "validators.ex" },
99
"kind": "calls",
1010
"mode": "same-file",
11-
"notes": "Same-module helper call within Validators"
11+
"notes": "Same-module helper call within Validators — extractor emits module-qualified names"
1212
},
1313
{
14-
"source": { "name": "validate_user", "file": "validators.ex" },
15-
"target": { "name": "valid_email?", "file": "validators.ex" },
14+
"source": { "name": "Validators.validate_user", "file": "validators.ex" },
15+
"target": { "name": "Validators.valid_email?", "file": "validators.ex" },
1616
"kind": "calls",
1717
"mode": "same-file",
18-
"notes": "Same-module helper call within Validators"
18+
"notes": "Same-module helper call within Validators — extractor emits module-qualified names"
1919
},
2020
{
21-
"source": { "name": "create_user", "file": "service.ex" },
22-
"target": { "name": "validate_user", "file": "validators.ex" },
21+
"source": { "name": "UserService.create_user", "file": "service.ex" },
22+
"target": { "name": "Validators.validate_user", "file": "validators.ex" },
2323
"kind": "calls",
2424
"mode": "module-function",
2525
"notes": "Validators.validate_user() — cross-module qualified call"
2626
},
2727
{
28-
"source": { "name": "create_user", "file": "service.ex" },
29-
"target": { "name": "save", "file": "repository.ex" },
28+
"source": { "name": "UserService.create_user", "file": "service.ex" },
29+
"target": { "name": "UserRepository.save", "file": "repository.ex" },
3030
"kind": "calls",
3131
"mode": "module-function",
3232
"notes": "UserRepository.save() — cross-module qualified call"
3333
},
3434
{
35-
"source": { "name": "get_user", "file": "service.ex" },
36-
"target": { "name": "find_by_id", "file": "repository.ex" },
35+
"source": { "name": "UserService.get_user", "file": "service.ex" },
36+
"target": { "name": "UserRepository.find_by_id", "file": "repository.ex" },
3737
"kind": "calls",
3838
"mode": "module-function",
3939
"notes": "UserRepository.find_by_id() — cross-module qualified call"
4040
},
4141
{
42-
"source": { "name": "remove_user", "file": "service.ex" },
43-
"target": { "name": "delete", "file": "repository.ex" },
42+
"source": { "name": "UserService.remove_user", "file": "service.ex" },
43+
"target": { "name": "UserRepository.delete", "file": "repository.ex" },
4444
"kind": "calls",
4545
"mode": "module-function",
4646
"notes": "UserRepository.delete() — cross-module qualified call"
4747
},
4848
{
49-
"source": { "name": "list_users", "file": "service.ex" },
50-
"target": { "name": "list_all", "file": "repository.ex" },
49+
"source": { "name": "UserService.list_users", "file": "service.ex" },
50+
"target": { "name": "UserRepository.list_all", "file": "repository.ex" },
5151
"kind": "calls",
5252
"mode": "module-function",
5353
"notes": "UserRepository.list_all() — cross-module qualified call"
5454
},
5555
{
56-
"source": { "name": "display_user", "file": "service.ex" },
57-
"target": { "name": "get_user", "file": "service.ex" },
56+
"source": { "name": "UserService.display_user", "file": "service.ex" },
57+
"target": { "name": "UserService.get_user", "file": "service.ex" },
5858
"kind": "calls",
5959
"mode": "same-file",
60-
"notes": "Same-module call to get_user within UserService"
60+
"notes": "Same-module call to get_user within UserService — extractor emits module-qualified names"
6161
},
6262
{
63-
"source": { "name": "display_user", "file": "service.ex" },
64-
"target": { "name": "format_user", "file": "service.ex" },
63+
"source": { "name": "UserService.display_user", "file": "service.ex" },
64+
"target": { "name": "UserService.format_user", "file": "service.ex" },
6565
"kind": "calls",
6666
"mode": "same-file",
67-
"notes": "Same-module call to private helper format_user"
67+
"notes": "Same-module call to private helper format_user — extractor emits module-qualified names"
6868
},
6969
{
70-
"source": { "name": "run", "file": "main.ex" },
71-
"target": { "name": "new_store", "file": "repository.ex" },
70+
"source": { "name": "Main.run", "file": "main.ex" },
71+
"target": { "name": "UserRepository.new_store", "file": "repository.ex" },
7272
"kind": "calls",
7373
"mode": "module-function",
7474
"notes": "UserRepository.new_store() — cross-module qualified call"
7575
},
7676
{
77-
"source": { "name": "run", "file": "main.ex" },
78-
"target": { "name": "create_user", "file": "service.ex" },
77+
"source": { "name": "Main.run", "file": "main.ex" },
78+
"target": { "name": "UserService.create_user", "file": "service.ex" },
7979
"kind": "calls",
8080
"mode": "module-function",
8181
"notes": "UserService.create_user() — cross-module qualified call"
8282
},
8383
{
84-
"source": { "name": "run", "file": "main.ex" },
85-
"target": { "name": "get_user", "file": "service.ex" },
84+
"source": { "name": "Main.run", "file": "main.ex" },
85+
"target": { "name": "UserService.get_user", "file": "service.ex" },
8686
"kind": "calls",
8787
"mode": "module-function",
8888
"notes": "UserService.get_user() — cross-module qualified call"
8989
},
9090
{
91-
"source": { "name": "run", "file": "main.ex" },
92-
"target": { "name": "list_users", "file": "service.ex" },
91+
"source": { "name": "Main.run", "file": "main.ex" },
92+
"target": { "name": "UserService.list_users", "file": "service.ex" },
9393
"kind": "calls",
9494
"mode": "module-function",
9595
"notes": "UserService.list_users() — cross-module qualified call"
9696
},
9797
{
98-
"source": { "name": "run", "file": "main.ex" },
99-
"target": { "name": "display_user", "file": "service.ex" },
98+
"source": { "name": "Main.run", "file": "main.ex" },
99+
"target": { "name": "UserService.display_user", "file": "service.ex" },
100100
"kind": "calls",
101101
"mode": "module-function",
102102
"notes": "UserService.display_user() — cross-module qualified call"
103103
},
104104
{
105-
"source": { "name": "run", "file": "main.ex" },
106-
"target": { "name": "remove_user", "file": "service.ex" },
105+
"source": { "name": "Main.run", "file": "main.ex" },
106+
"target": { "name": "UserService.remove_user", "file": "service.ex" },
107107
"kind": "calls",
108108
"mode": "module-function",
109109
"notes": "UserService.remove_user() — cross-module qualified call"
110110
},
111111
{
112-
"source": { "name": "run", "file": "main.ex" },
113-
"target": { "name": "fetch", "file": "patterns.ex" },
112+
"source": { "name": "Main.run", "file": "main.ex" },
113+
"target": { "name": "Patterns.fetch", "file": "patterns.ex" },
114114
"kind": "calls",
115115
"mode": "module-function",
116116
"notes": "Patterns.fetch() — exercises default-value parameter extraction"
117117
},
118118
{
119-
"source": { "name": "run", "file": "main.ex" },
120-
"target": { "name": "first_of", "file": "patterns.ex" },
119+
"source": { "name": "Main.run", "file": "main.ex" },
120+
"target": { "name": "Patterns.first_of", "file": "patterns.ex" },
121121
"kind": "calls",
122122
"mode": "module-function",
123123
"notes": "Patterns.first_of() — exercises tuple-pattern parameter extraction"
124124
},
125125
{
126-
"source": { "name": "run", "file": "main.ex" },
127-
"target": { "name": "name_of", "file": "patterns.ex" },
126+
"source": { "name": "Main.run", "file": "main.ex" },
127+
"target": { "name": "Patterns.name_of", "file": "patterns.ex" },
128128
"kind": "calls",
129129
"mode": "module-function",
130130
"notes": "Patterns.name_of() — exercises map-pattern parameter extraction"
131131
},
132132
{
133-
"source": { "name": "run", "file": "main.ex" },
134-
"target": { "name": "id_of", "file": "patterns.ex" },
133+
"source": { "name": "Main.run", "file": "main.ex" },
134+
"target": { "name": "Patterns.id_of", "file": "patterns.ex" },
135135
"kind": "calls",
136136
"mode": "module-function",
137137
"notes": "Patterns.id_of() — exercises struct-pattern parameter extraction"
138138
},
139139
{
140-
"source": { "name": "run", "file": "main.ex" },
141-
"target": { "name": "head_of", "file": "patterns.ex" },
140+
"source": { "name": "Main.run", "file": "main.ex" },
141+
"target": { "name": "Patterns.head_of", "file": "patterns.ex" },
142142
"kind": "calls",
143143
"mode": "module-function",
144144
"notes": "Patterns.head_of() — exercises list-cons pattern parameter extraction"
145145
},
146146
{
147-
"source": { "name": "run", "file": "main.ex" },
148-
"target": { "name": "all_of", "file": "patterns.ex" },
147+
"source": { "name": "Main.run", "file": "main.ex" },
148+
"target": { "name": "Patterns.all_of", "file": "patterns.ex" },
149149
"kind": "calls",
150150
"mode": "module-function",
151151
"notes": "Patterns.all_of() — exercises list pattern parameter extraction"

‎tests/benchmarks/resolution/fixtures/julia/expected-edges.json‎

Lines changed: 45 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -4,109 +4,109 @@
44
"description": "Hand-annotated call edges for Julia resolution benchmark",
55
"edges": [
66
{
7-
"source": { "name": "main", "file": "main.jl" },
8-
"target": { "name": "new_repo", "file": "repository.jl" },
7+
"source": { "name": "App.main", "file": "main.jl" },
8+
"target": { "name": "Repository.new_repo", "file": "repository.jl" },
99
"kind": "calls",
1010
"mode": "module-function",
11-
"notes": "Repository.new_repo() — qualified call to Repository module"
11+
"notes": "Repository.new_repo() — qualified call to Repository module; extractor emits module-qualified names"
1212
},
1313
{
14-
"source": { "name": "main", "file": "main.jl" },
15-
"target": { "name": "create_user", "file": "service.jl" },
14+
"source": { "name": "App.main", "file": "main.jl" },
15+
"target": { "name": "Service.create_user", "file": "service.jl" },
1616
"kind": "calls",
1717
"mode": "module-function",
18-
"notes": "Service.create_user() — qualified call to Service module"
18+
"notes": "Service.create_user() — qualified call to Service module; extractor emits module-qualified names"
1919
},
2020
{
21-
"source": { "name": "main", "file": "main.jl" },
22-
"target": { "name": "get_user", "file": "service.jl" },
21+
"source": { "name": "App.main", "file": "main.jl" },
22+
"target": { "name": "Service.get_user", "file": "service.jl" },
2323
"kind": "calls",
2424
"mode": "module-function",
25-
"notes": "Service.get_user() — qualified call to Service module"
25+
"notes": "Service.get_user() — qualified call to Service module; extractor emits module-qualified names"
2626
},
2727
{
28-
"source": { "name": "main", "file": "main.jl" },
29-
"target": { "name": "remove_user", "file": "service.jl" },
28+
"source": { "name": "App.main", "file": "main.jl" },
29+
"target": { "name": "Service.remove_user", "file": "service.jl" },
3030
"kind": "calls",
3131
"mode": "module-function",
32-
"notes": "Service.remove_user() — qualified call to Service module"
32+
"notes": "Service.remove_user() — qualified call to Service module; extractor emits module-qualified names"
3333
},
3434
{
35-
"source": { "name": "main", "file": "main.jl" },
36-
"target": { "name": "summary", "file": "service.jl" },
35+
"source": { "name": "App.main", "file": "main.jl" },
36+
"target": { "name": "Service.summary", "file": "service.jl" },
3737
"kind": "calls",
3838
"mode": "module-function",
39-
"notes": "Service.summary() — qualified call to Service module"
39+
"notes": "Service.summary() — qualified call to Service module; extractor emits module-qualified names"
4040
},
4141
{
42-
"source": { "name": "create_user", "file": "service.jl" },
43-
"target": { "name": "validate_name", "file": "validators.jl" },
42+
"source": { "name": "Service.create_user", "file": "service.jl" },
43+
"target": { "name": "Validators.validate_name", "file": "validators.jl" },
4444
"kind": "calls",
4545
"mode": "module-function",
46-
"notes": "Validators.validate_name() — qualified call to Validators module"
46+
"notes": "Validators.validate_name() — qualified call to Validators module; extractor emits module-qualified names"
4747
},
4848
{
49-
"source": { "name": "create_user", "file": "service.jl" },
50-
"target": { "name": "validate_email", "file": "validators.jl" },
49+
"source": { "name": "Service.create_user", "file": "service.jl" },
50+
"target": { "name": "Validators.validate_email", "file": "validators.jl" },
5151
"kind": "calls",
5252
"mode": "module-function",
53-
"notes": "Validators.validate_email() — qualified call to Validators module"
53+
"notes": "Validators.validate_email() — qualified call to Validators module; extractor emits module-qualified names"
5454
},
5555
{
56-
"source": { "name": "create_user", "file": "service.jl" },
57-
"target": { "name": "save", "file": "repository.jl" },
56+
"source": { "name": "Service.create_user", "file": "service.jl" },
57+
"target": { "name": "Repository.save", "file": "repository.jl" },
5858
"kind": "calls",
5959
"mode": "module-function",
60-
"notes": "Repository.save() — qualified call to Repository module"
60+
"notes": "Repository.save() — qualified call to Repository module; extractor emits module-qualified names"
6161
},
6262
{
63-
"source": { "name": "get_user", "file": "service.jl" },
64-
"target": { "name": "find_by_id", "file": "repository.jl" },
63+
"source": { "name": "Service.get_user", "file": "service.jl" },
64+
"target": { "name": "Repository.find_by_id", "file": "repository.jl" },
6565
"kind": "calls",
6666
"mode": "module-function",
67-
"notes": "Repository.find_by_id() — qualified call to Repository module"
67+
"notes": "Repository.find_by_id() — qualified call to Repository module; extractor emits module-qualified names"
6868
},
6969
{
70-
"source": { "name": "remove_user", "file": "service.jl" },
71-
"target": { "name": "delete", "file": "repository.jl" },
70+
"source": { "name": "Service.remove_user", "file": "service.jl" },
71+
"target": { "name": "Repository.delete", "file": "repository.jl" },
7272
"kind": "calls",
7373
"mode": "module-function",
74-
"notes": "Repository.delete() — qualified call to Repository module"
74+
"notes": "Repository.delete() — qualified call to Repository module; extractor emits module-qualified names"
7575
},
7676
{
77-
"source": { "name": "summary", "file": "service.jl" },
78-
"target": { "name": "count", "file": "repository.jl" },
77+
"source": { "name": "Service.summary", "file": "service.jl" },
78+
"target": { "name": "Repository.count", "file": "repository.jl" },
7979
"kind": "calls",
8080
"mode": "module-function",
81-
"notes": "Repository.count() — qualified call to Repository module"
81+
"notes": "Repository.count() — qualified call to Repository module; extractor emits module-qualified names"
8282
},
8383
{
84-
"source": { "name": "summary", "file": "service.jl" },
85-
"target": { "name": "format_summary", "file": "service.jl" },
84+
"source": { "name": "Service.summary", "file": "service.jl" },
85+
"target": { "name": "Service.format_summary", "file": "service.jl" },
8686
"kind": "calls",
8787
"mode": "same-file",
88-
"notes": "Same-file call to private helper function"
88+
"notes": "Same-file call to private helper function — extractor emits module-qualified names"
8989
},
9090
{
91-
"source": { "name": "validate_name", "file": "validators.jl" },
92-
"target": { "name": "check_length", "file": "validators.jl" },
91+
"source": { "name": "Validators.validate_name", "file": "validators.jl" },
92+
"target": { "name": "Validators.check_length", "file": "validators.jl" },
9393
"kind": "calls",
9494
"mode": "same-file",
95-
"notes": "Same-file call to private helper function"
95+
"notes": "Same-file call to private helper function — extractor emits module-qualified names"
9696
},
9797
{
98-
"source": { "name": "validate_email", "file": "validators.jl" },
99-
"target": { "name": "contains_at", "file": "validators.jl" },
98+
"source": { "name": "Validators.validate_email", "file": "validators.jl" },
99+
"target": { "name": "Validators.contains_at", "file": "validators.jl" },
100100
"kind": "calls",
101101
"mode": "same-file",
102-
"notes": "Same-file call to private helper function"
102+
"notes": "Same-file call to private helper function — extractor emits module-qualified names"
103103
},
104104
{
105-
"source": { "name": "count", "file": "repository.jl" },
106-
"target": { "name": "count_entries", "file": "repository.jl" },
105+
"source": { "name": "Repository.count", "file": "repository.jl" },
106+
"target": { "name": "Repository.count_entries", "file": "repository.jl" },
107107
"kind": "calls",
108108
"mode": "same-file",
109-
"notes": "Same-file call to private helper function"
109+
"notes": "Same-file call to private helper function — extractor emits module-qualified names"
110110
}
111111
]
112112
}

0 commit comments

Comments
 (0)