Skip to content

Commit 3a43ba5

Browse files
chirinochirino
committed
feat: add DWARF debugging support and source mapping infrastructure
- Add comprehensive SMAP (Source Map) parsing infrastructure to runtime module - Add new dwarf-rust module for parsing DWARF debugging information from WebAssembly modules to SMAP stratums - Enhance InterpreterMachine to use parsed SMAP stratums to enhance stack traces. - Enhance compiler to add SMAPs to class files and use to enhance stack traces. This enhancement enables developers to get meaningful stack traces with original source line numbers when debugging WebAssembly modules compiled from high-level languages like Rust and Go. Where you would get exceptions that looked like. ``` com.dylibso.chicory.runtime.TrapException: Trapped on unreachable instruction at com.dylibso.chicory.runtime.InterpreterMachine.THROW_UNREACHABLE(InterpreterMachine.java:2212) at com.dylibso.chicory.runtime.InterpreterMachine.eval(InterpreterMachine.java:182) at com.dylibso.chicory.runtime.InterpreterMachine.call(InterpreterMachine.java:100) at com.dylibso.chicory.runtime.InterpreterMachine.CALL(InterpreterMachine.java:1715) ... more of the same … at com.dylibso.chicory.runtime.InterpreterMachine.eval(InterpreterMachine.java:550) at com.dylibso.chicory.runtime.InterpreterMachine.call(InterpreterMachine.java:100) at com.dylibso.chicory.runtime.InterpreterMachine.call(InterpreterMachine.java:65) at com.dylibso.chicory.runtime.Instance$Exports.lambda$function$0(Instance.java:219) at com.dylibso.chicory.testing.MachinesTest.lambda$shouldEmitUnderstandableStackTraces$0(MachinesTest.java:300) at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:53) at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:35) at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3128) at com.dylibso.chicory.testing.MachinesTest.shouldEmitUnderstandableStackTraces(MachinesTest.java:300) ``` You now get exceptions that look like: ``` com.dylibso.chicory.runtime.TrapException: Trapped on unreachable instruction at chicory interpreter 0x006721: func.94(library/std/src/sys/pal/wasm/../unsupported/common.rs:28) at chicory interpreter 0x005cc6: func.79(library/std/src/panicking.rs:699) at chicory interpreter 0x005c00: func.78(library/std/src/sys/backtrace.rs:168) at chicory interpreter 0x00627d: func.86(library/std/src/panicking.rs:697) at chicory interpreter 0x007e74: func.118(library/core/src/panicking.rs:117) at chicory interpreter 0x007ec8: func.119(library/core/src/panicking.rs:218) at chicory interpreter 0x001cf6: func.30(/rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/ub_checks.rs:68) at chicory interpreter 0x003948: func.54(/rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/ub_checks.rs:75) at chicory interpreter 0x000d7f: func.11(src/lib.rs:23) at com.dylibso.chicory.runtime.Instance$Exports.lambda$function$0(Instance.java:219) at com.dylibso.chicory.testing.MachinesTest.lambda$shouldEmitUnderstandableStackTraces$0(MachinesTest.java:300) at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:53) at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:35) at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3128) at com.dylibso.chicory.testing.MachinesTest.shouldEmitUnderstandableStackTraces(MachinesTest.java:300) ``` Signed-off-by: Hiram Chirino <hiram@hiramchirino.com>
1 parent 3a2fa87 commit 3a43ba5

47 files changed

Lines changed: 20683 additions & 25 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎compiler-tests/src/test/resources/com/dylibso/chicory/testing/MethodTooLargeTest.testBigFunc.approved.txt‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,11 @@ private final Lcom/dylibso/chicory/runtime/Instance; instance
33
private final Lcom/dylibso/chicory/runtime/internal/CompilerInterpreterMachine; compilerInterpreterMachine
44
}
55

6+
public final static INNERCLASS java/lang/invoke/MethodHandles$Lookup java/lang/invoke/MethodHandles Lookup
7+
68
private final static Z memCopyWorkaround
9+
10+
private final static Ljava/util/HashMap; STRATA_BY_FUNC_GROUP
711
}
812

913
final class com/dylibso/chicory/$gen/CompiledMachineFuncGroup_0 {

‎compiler/src/main/java/com/dylibso/chicory/compiler/MachineFactoryCompiler.java‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@ private MachineFactoryCompiler() {
2525
* </pre>
2626
* <p>
2727
* Every instance created by the builder will pay the cost of compiling the module.
28-
* </p>
2928
*
3029
* @see #compile(WasmModule) If you want to compile the module only once for multiple instances.
3130
*/
@@ -37,7 +36,6 @@ public static Machine compile(Instance instance) {
3736
* Compiles a machine factory that can used in instance builders.
3837
* The module is only compiled once and the machine factory is reused for every
3938
* instance created by the builder.
40-
* <p>
4139
* <pre>
4240
* var module = Parser.parse(is);
4341
* var builder = Instance.builder(module)
@@ -55,7 +53,6 @@ public static Function<Instance, Machine> compile(WasmModule module) {
5553
* The builder allows you to configure the compiler options used to compile the module to
5654
* byte code.
5755
* This should be used when you want to create multiple instances of the same module.
58-
* <p>
5956
* <pre>
6057
* var module = Parser.parse(is);
6158
* var builder = Instance.builder(module)

‎compiler/src/main/java/com/dylibso/chicory/compiler/internal/Compiler.java‎

Lines changed: 128 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@
2929
import static com.dylibso.chicory.compiler.internal.ShadedRefs.CALL_INDIRECT;
3030
import static com.dylibso.chicory.compiler.internal.ShadedRefs.CALL_INDIRECT_ON_INTERPRETER;
3131
import static com.dylibso.chicory.compiler.internal.ShadedRefs.CHECK_INTERRUPTION;
32+
import static com.dylibso.chicory.compiler.internal.ShadedRefs.ENHANCE_STACK_TRACE;
3233
import static com.dylibso.chicory.compiler.internal.ShadedRefs.INSTANCE_MEMORY;
3334
import static com.dylibso.chicory.compiler.internal.ShadedRefs.INSTANCE_TABLE;
3435
import static com.dylibso.chicory.compiler.internal.ShadedRefs.TABLE_INSTANCE;
@@ -45,6 +46,7 @@
4546
import static java.lang.invoke.MethodType.methodType;
4647
import static java.util.Objects.requireNonNull;
4748
import static java.util.Objects.requireNonNullElse;
49+
import static java.util.Objects.requireNonNullElseGet;
4850
import static java.util.stream.Collectors.toSet;
4951
import static org.objectweb.asm.Type.INT_TYPE;
5052
import static org.objectweb.asm.Type.LONG_TYPE;
@@ -60,6 +62,8 @@
6062
import com.dylibso.chicory.runtime.Machine;
6163
import com.dylibso.chicory.runtime.Memory;
6264
import com.dylibso.chicory.runtime.internal.CompilerInterpreterMachine;
65+
import com.dylibso.chicory.runtime.internal.smap.Smap;
66+
import com.dylibso.chicory.runtime.internal.smap.Stratum;
6367
import com.dylibso.chicory.wasm.ChicoryException;
6468
import com.dylibso.chicory.wasm.WasmModule;
6569
import com.dylibso.chicory.wasm.types.ExternalType;
@@ -121,16 +125,38 @@ public final class Compiler {
121125
private int maxFunctionsPerClass;
122126
private final HashSet<Integer> interpretedFunctions;
123127

128+
private final DebugContext debugContext;
129+
private int funcGroupCount;
130+
131+
// Debug information
132+
static final class DebugContext {
133+
final Stratum inputStratum;
134+
final Stratum outputStratum = new Stratum("WASM");
135+
int nextOutputLineNo = 1;
136+
137+
DebugContext(Stratum stratum) {
138+
this.inputStratum = requireNonNull(stratum, "stratum");
139+
this.inputStratum.optimizeForLookups();
140+
}
141+
142+
void reset() {
143+
outputStratum.clear();
144+
nextOutputLineNo = 1;
145+
}
146+
}
147+
124148
private Compiler(
125149
WasmModule module,
126150
String className,
127151
int maxFunctionsPerClass,
128152
InterpreterFallback interpreterFallback,
129-
Set<Integer> interpretedFunctions) {
153+
Set<Integer> interpretedFunctions,
154+
Stratum stratum) {
130155
this.className = requireNonNull(className, "className");
131156
this.module = requireNonNull(module, "module");
132157
this.analyzer = new WasmAnalyzer(module);
133158
this.functionImports = module.importSection().count(ExternalType.FUNCTION);
159+
this.debugContext = new DebugContext(requireNonNullElseGet(stratum, () -> new Stratum("")));
134160

135161
if (interpretedFunctions == null || interpretedFunctions.isEmpty()) {
136162
this.interpretedFunctions = new HashSet<>();
@@ -162,6 +188,7 @@ public static final class Builder {
162188
private int maxFunctionsPerClass;
163189
private InterpreterFallback interpreterFallback;
164190
private Set<Integer> interpretedFunctions;
191+
private Function<WasmModule, Stratum> debugParser;
165192

166193
private Builder(WasmModule module) {
167194
this.module = module;
@@ -172,6 +199,11 @@ public Builder withClassName(String className) {
172199
return this;
173200
}
174201

202+
public Builder withDebugParser(Function<WasmModule, Stratum> debugParser) {
203+
this.debugParser = debugParser;
204+
return this;
205+
}
206+
175207
public Builder withMaxFunctionsPerClass(int maxFunctionsPerClass) {
176208
this.maxFunctionsPerClass = maxFunctionsPerClass;
177209
return this;
@@ -197,12 +229,19 @@ public Compiler build() {
197229
if (maxFunctionsPerClass <= 0) {
198230
maxFunctionsPerClass = DEFAULT_MAX_FUNCTIONS_PER_CLASS;
199231
}
232+
233+
Stratum stratum = null;
234+
if (debugParser != null) {
235+
stratum = debugParser.apply(module);
236+
}
237+
200238
return new Compiler(
201239
module,
202240
className,
203241
maxFunctionsPerClass,
204242
interpreterFallback,
205-
interpretedFunctions);
243+
interpretedFunctions,
244+
stratum);
206245
}
207246
}
208247

@@ -286,12 +325,14 @@ private void compileExtraClasses() {
286325
//
287326
var originalMaxFunctionsPerClass = maxFunctionsPerClass;
288327
while (true) {
328+
funcGroupCount = 0;
289329
try {
290330
maxFunctionsPerClass =
291331
loadChunkedClass(
292332
totalFunctions,
293333
maxFunctionsPerClass,
294334
(start, end, chunkSize) -> {
335+
funcGroupCount++;
295336
maxFunctionsPerClass = chunkSize;
296337
String className = classNameForFuncGroup(start);
297338
return compileExtraClass(
@@ -396,6 +437,7 @@ private String classNameForFuncGroup(int funcId) {
396437

397438
private Consumer<ClassVisitor> emitFunctionGroup(int start, int end, String internalClassName) {
398439
return (classWriter) -> {
440+
debugContext.reset();
399441
for (int i = start; i < end; i++) {
400442
FunctionBody body = null;
401443
try {
@@ -436,6 +478,25 @@ private Consumer<ClassVisitor> emitFunctionGroup(int start, int end, String inte
436478
throw handleMethodTooLarge(e, module);
437479
}
438480
}
481+
if (!debugContext.outputStratum.isEmpty()) {
482+
// add the source map entries to the class
483+
var sourceFile = internalClassName + ".java";
484+
485+
String smap =
486+
new Smap()
487+
.withOutputFileName(sourceFile)
488+
.withStratum(debugContext.outputStratum.optimizeForWrite(), true)
489+
.toString();
490+
491+
classWriter.visitSource(internalClassName, smap);
492+
493+
classWriter.visitField(
494+
Opcodes.ACC_PUBLIC | Opcodes.ACC_FINAL | Opcodes.ACC_STATIC,
495+
"SMAP",
496+
getDescriptor(String.class),
497+
null,
498+
smap);
499+
}
439500
};
440501
}
441502

@@ -471,6 +532,35 @@ private byte[] compileClass() {
471532
null);
472533
}
473534

535+
// if we have debug info
536+
if (!debugContext.inputStratum.isEmpty()) {
537+
538+
// emit a: public static final String[] SMAPS = new String[]{ FuncGroup_0.SMAP,
539+
// FuncGroup_1.SMAP, ... };
540+
classWriter.visitField(
541+
Opcodes.ACC_PUBLIC | Opcodes.ACC_FINAL | Opcodes.ACC_STATIC,
542+
"SMAPS",
543+
getDescriptor(String[].class),
544+
null,
545+
null);
546+
547+
// static initializer for SMAPS field
548+
emitFunction(
549+
classWriter,
550+
"<clinit>",
551+
methodType(void.class),
552+
true,
553+
asm -> compileStaticInitializer(asm, internalClassName));
554+
555+
// emit: enhanceStackTrace method
556+
emitFunction(
557+
classWriter,
558+
"enhanceStackTrace",
559+
methodType(void.class, Throwable.class),
560+
true,
561+
asm -> compileEnhanceStackTrace(asm, internalClassName));
562+
}
563+
474564
// constructor
475565
emitFunction(
476566
classWriter,
@@ -574,6 +664,34 @@ private static void emitCallSuper(InstructionAdapter asm) {
574664
OBJECT_TYPE.getInternalName(), "<init>", getMethodDescriptor(VOID_TYPE), false);
575665
}
576666

667+
private void compileStaticInitializer(InstructionAdapter asm, String internalClassName) {
668+
669+
asm.iconst(funcGroupCount);
670+
asm.visitTypeInsn(Opcodes.ANEWARRAY, getInternalName(String.class));
671+
for (int i = 0; i < funcGroupCount; i++) {
672+
asm.dup();
673+
asm.iconst(i);
674+
asm.getstatic(
675+
internalClassName + classNameForFuncGroup(i * maxFunctionsPerClass),
676+
"SMAP",
677+
getDescriptor(String.class));
678+
asm.astore(OBJECT_TYPE);
679+
}
680+
681+
asm.putstatic(internalClassName, "SMAPS", getDescriptor(String[].class));
682+
asm.areturn(VOID_TYPE);
683+
}
684+
685+
private void compileEnhanceStackTrace(InstructionAdapter asm, String internalClassName) {
686+
// return Shaded.enhanceStackTrace(e, SMAPS, internalClassName+"FuncGroup_");
687+
asm.load(0, OBJECT_TYPE); // elements parameter
688+
asm.getstatic(internalClassName, "SMAPS", getDescriptor(String[].class));
689+
var classNamePrefix = internalClassName.replaceAll("/", ".") + "FuncGroup_";
690+
asm.aconst(classNamePrefix);
691+
emitInvokeStatic(asm, ENHANCE_STACK_TRACE);
692+
asm.areturn(VOID_TYPE);
693+
}
694+
577695
private void compileConstructor(InstructionAdapter asm, String internalClassName) {
578696
emitCallSuper(asm);
579697

@@ -1187,7 +1305,7 @@ private void compileFunction(
11871305
type,
11881306
body);
11891307

1190-
List<CompilerInstruction> instructions = analyzer.analyze(funcId);
1308+
List<CompilerInstruction> instructions = analyzer.analyze(funcId, debugContext);
11911309

11921310
int localsCount = type.params().size();
11931311
if (hasTooManyParameters(type)) {
@@ -1225,13 +1343,20 @@ private void compileFunction(
12251343

12261344
// compile the function body
12271345
for (CompilerInstruction ins : instructions) {
1346+
12281347
switch (ins.opcode()) {
12291348
case LABEL:
12301349
Label label = labels.get(ins.operand(0));
12311350
if (label != null) {
12321351
asm.mark(label);
12331352
visitedTargets.add(ins.operand(0));
12341353
}
1354+
1355+
break;
1356+
case LINE_NUMBER:
1357+
int line = (int) ins.operand(0);
1358+
label = labels.get(ins.operand(1));
1359+
asm.visitLineNumber(line, label);
12351360
break;
12361361
case GOTO:
12371362
if (visitedTargets.contains(ins.operand(0))) {

‎compiler/src/main/java/com/dylibso/chicory/compiler/internal/CompilerInstruction.java‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,8 @@ public long[] labelTargets() {
5656
case IFNE:
5757
case SWITCH:
5858
return operands;
59+
case LINE_NUMBER:
60+
return new long[] {operands[1]};
5961
default:
6062
return EMPTY;
6163
}

‎compiler/src/main/java/com/dylibso/chicory/compiler/internal/CompilerOpCode.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77

88
enum CompilerOpCode {
99
LABEL,
10+
LINE_NUMBER,
1011
DROP_KEEP,
1112
TRAP,
1213
GOTO,

‎compiler/src/main/java/com/dylibso/chicory/compiler/internal/Shaded.java‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,12 @@
1010
import com.dylibso.chicory.runtime.OpcodeImpl;
1111
import com.dylibso.chicory.runtime.TrapException;
1212
import com.dylibso.chicory.runtime.WasmRuntimeException;
13+
import com.dylibso.chicory.runtime.internal.smap.SmapParser;
14+
import com.dylibso.chicory.runtime.internal.smap.Stratum;
1315
import com.dylibso.chicory.wasm.ChicoryException;
1416
import com.dylibso.chicory.wasm.InvalidException;
1517
import com.dylibso.chicory.wasm.types.FunctionType;
18+
import java.util.HashMap;
1619

1720
/**
1821
* This class will get shaded into the compiled code.
@@ -204,4 +207,45 @@ public static long readGlobal(int index, Instance instance) {
204207
public static void writeGlobal(long value, int index, Instance instance) {
205208
instance.global(index).setValue(value);
206209
}
210+
211+
private static final HashMap<Integer, Stratum> STRATA_BY_FUNC_GROUP = new HashMap<>();
212+
213+
private static Stratum getStratum(int group, String smap) {
214+
synchronized (STRATA_BY_FUNC_GROUP) {
215+
return STRATA_BY_FUNC_GROUP.computeIfAbsent(
216+
group, (x) -> SmapParser.parse(smap).getDefaultStratum().optimizeForLookups());
217+
}
218+
}
219+
220+
public static void enhanceStackTrace(Throwable e, String[] smaps, String funcGroupClassPrefix) {
221+
var elements = e.getStackTrace();
222+
for (int i = 0; i < elements.length; i++) {
223+
var element = elements[i];
224+
if (element.getClassName().startsWith(funcGroupClassPrefix)) {
225+
226+
String suffix = element.getClassName().substring(funcGroupClassPrefix.length());
227+
var group = Integer.parseInt(suffix);
228+
if (group < smaps.length) {
229+
230+
var line = element.getLineNumber();
231+
232+
var stratum = getStratum(group, smaps[group]);
233+
var lineMapping = stratum.getLineMapping(line);
234+
var path = stratum.getPath(lineMapping.lineFileID());
235+
236+
// Find what it maps to.
237+
if (lineMapping != null) {
238+
elements[i] =
239+
new StackTraceElement(
240+
element.getClassName(),
241+
element.getMethodName(), // todo: can we get the wasm
242+
// function name?
243+
path,
244+
(int) lineMapping.inputStartLine());
245+
}
246+
}
247+
}
248+
}
249+
e.setStackTrace(elements);
250+
}
207251
}

‎compiler/src/main/java/com/dylibso/chicory/compiler/internal/ShadedRefs.java‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ public final class ShadedRefs {
5252
static final Method THROW_TRAP_EXCEPTION;
5353
static final Method THROW_UNKNOWN_FUNCTION;
5454
static final Method AOT_INTERPRETER_MACHINE_CALL;
55+
static final Method ENHANCE_STACK_TRACE;
5556

5657
static {
5758
try {
@@ -160,6 +161,9 @@ public final class ShadedRefs {
160161

161162
AOT_INTERPRETER_MACHINE_CALL =
162163
CompilerInterpreterMachine.class.getMethod("call", int.class, long[].class);
164+
ENHANCE_STACK_TRACE =
165+
Shaded.class.getMethod(
166+
"enhanceStackTrace", Throwable.class, String[].class, String.class);
163167
} catch (NoSuchMethodException e) {
164168
throw new AssertionError(e);
165169
}

0 commit comments

Comments
 (0)