chore: tracegrind plan

Author: art049

tracegrind-plan.md

@@ -0,0 +1,177 @@
+# Tracegrind Implementation Plan
+
+## Overview
+Create a new Valgrind tool called **tracegrind** by forking callgrind. Tracegrind uses the same cache simulation and trapdoor mechanisms but outputs **streaming CSV trace data** instead of an accumulated callgraph. A "sample" (CSV row) is emitted at every function entry/exit with delta counters since the last sample.
+
+## Commit Strategy
+1. **Commit 1**: Pure mechanical fork — copy callgrind → tracegrind, rename symbols, register in build system, verify it compiles and runs identically to callgrind.
+2. **Commit 2+**: Behavioral changes — replace dump with CSV tracing output.
+
+---
+
+## Phase 1: Fork callgrind → tracegrind (Commit 1)
+
+### 1.1 Copy directory
+```
+cp -r callgrind/ tracegrind/
+```
+
+### 1.2 Rename symbol prefixes
+In **all `.c` and `.h` files** in `tracegrind/`:
+- `CLG_(str)` macro → `TG_(str)` expanding to `VGAPPEND(vgTracegrind_, str)`
+- `CLG_` prefix (constants like `CLG_ENABLE_DEBUG`, `CLG_EXPERIMENTAL`, `CLG_ASSERT`, `CLG_DEBUG`, `CLG_DEBUGIF`, `CLG_MALLOC`) → `TG_` equivalents
+- Header guards: `CLG_GLOBAL` → `TG_GLOBAL`, `CLG_EVENTS` → `TG_EVENTS`, `CLG_COSTS` → `TG_COSTS`
+- `callgrind.h` → `tracegrind.h` (public header)
+- Tool name strings: `"Callgrind"` → `"Tracegrind"`, descriptions updated
+- File comment headers: `Callgrind` → `Tracegrind`
+- Client request base: keep `'C','T'` for compatibility (same trapdoor macros work)
+- Macro names in public header: `CALLGRIND_*` → `TRACEGRIND_*`
+
+### 1.3 Create `tracegrind/Makefile.am`
+Based on `callgrind/Makefile.am`, replace all `callgrind` → `tracegrind`, `CALLGRIND` → `TRACEGRIND`. Keep `TRACEGRIND_CFLAGS_COMMON = -I$(top_srcdir)/cachegrind`.
+
+### 1.4 Register in build system
+- **`Makefile.am`** (root): Add `tracegrind` to `TOOLS` list
+- **`configure.ac`**: Add `tracegrind/Makefile` to `AC_CONFIG_FILES`
+
+### 1.5 Verify
+- Run `autogen.sh && ./configure && make` — tracegrind binary should be produced
+- Run `valgrind --tool=tracegrind ls` — should produce callgrind-format output (still a clone at this point)
+
+---
+
+## Phase 2: Add CSV tracing infrastructure
+
+### 2.1 New tracing state in `global.h`
+Add to the per-thread or global state:
+```c
+/* Tracing output state */
+typedef struct {
+    Int       fd;              /* Output file descriptor */
+    ULong     seq;             /* Global sequence counter */
+    ULong*    last_sample_cost; /* Snapshot at last sample emission */
+    Bool      initialized;
+} trace_output;
+```
+Add a global `trace_output` and per-thread `last_sample_cost` arrays.
+
+### 2.2 CSV output functions in `dump.c` (heavily modified)
+Replace the callgraph dump logic with:
+
+**`trace_open_output()`** — Open CSV file, write header row:
+```csv
+seq,tid,event,fn,obj,file,line,Ir,I1mr,ILmr,Dr,D1mr,DLmr,Dw,D1mw,DLmw,Bc,Bcm,Bi,Bim,sysCount,sysTime
+```
+(Columns depend on which event groups are active. `obj` = shared object/binary name.)
+
+**`trace_emit_sample(tid, event_type, fn_node)`** — Core function:
+1. Compute delta: `current_cost[i] - last_sample_cost[i]` for each active event
+2. Resolve function name, obj (from `fn_node->from->obj->name`), file, line from debug info
+3. Write CSV row with: seq++, tid, event (ENTER/EXIT), fn, obj, file, line, deltas...
+4. Update `last_sample_cost = current_cost` (copy)
+
+**`trace_close_output()`** — Write a final summary/totals row (for verification), close fd.
+
+### 2.3 Output file naming
+Output file: `tracegrind.out.<pid>` (or configurable via `--tracegrind-out-file=<path>`)
+
+### 2.4 Counter delta tracking
+The key mechanism:
+- `TG_(current_state).cost` is the cumulative cost pointer (same as callgrind)
+- Maintain a per-thread `last_sample_cost[]` array (same size as full event set)
+- At each sample point: `delta[i] = current_cost[i] - last_sample_cost[i]`
+- After emitting: `last_sample_cost[i] = current_cost[i]`
+- Sum of all deltas across all samples = total cost (matches callgrind)
+
+---
+
+## Phase 3: Hook context switches to emit samples
+
+### 3.1 Modify `callstack.c` — `TG_(push_call_stack)()`
+After the existing enter_cost copy (line ~212) and before incrementing sp, add:
+```c
+if (!skip && TG_(current_state).collect) {
+    fn_node* to_fn = to->cxt->fn[0];
+    trace_emit_sample(TG_(current_tid), "ENTER", to_fn);
+}
+```
+
+### 3.2 Modify `callstack.c` — `TG_(pop_call_stack)()`
+After computing the cost diff (line ~345) and before restoring context, add:
+```c
+if (jcc && TG_(current_state).collect) {
+    fn_node* to_fn = jcc->to->cxt->fn[0];
+    trace_emit_sample(TG_(current_tid), "EXIT", to_fn);
+}
+```
+
+### 3.3 Thread switch handling
+In `threads.c` — `TG_(switch_thread)()`:
+- Save/restore per-thread `last_sample_cost` along with other per-thread state
+- Ensure the trace output fd is shared (single file, all threads)
+
+### 3.4 Keep jCC/BBCC accumulation
+Keep the existing cost accumulation in jCCs and BBCCs — this ensures the totals computed by the existing dump mechanism can be compared for correctness verification. The tracing is an additional output layer on top.
+
+---
+
+## Phase 4: Strip/simplify callgraph dump
+
+### 4.1 Simplify `dump.c`
+- Remove the complex callgraph output functions (`fprint_bbcc`, `fprint_jcc`, `print_bbccs`, etc.)
+- Keep `TG_(dump_profile)` but make it emit a totals summary instead of full callgraph
+- The totals summary serves as a verification mechanism
+
+### 4.2 Simplify or remove unused features
+- Remove `callgrind_annotate` and `callgrind_control` from tracegrind build (not applicable)
+- Keep all command-line options that affect instrumentation behavior
+- Remove dump-specific options (`--dump-instr`, `--dump-line`, `--compress-strings`, etc.)
+
+---
+
+## Phase 5: Verification
+
+### 5.1 Totals verification
+- Run both callgrind and tracegrind on the same program
+- Sum all delta columns from tracegrind CSV
+- Compare with callgrind totals line — should match exactly
+
+### 5.2 Test with existing test programs
+- `callgrind/tests/clreq.c` — test START/STOP/TOGGLE trapdoor mechanisms
+- Simple test programs with known call structures
+
+---
+
+## Key Files to Modify (after fork)
+
+| File | Changes |
+|------|---------|
+| `tracegrind/global.h` | Add `trace_output` struct, per-thread sample state |
+| `tracegrind/dump.c` | Replace callgraph output with CSV tracing + totals |
+| `tracegrind/callstack.c` | Hook `push_call_stack`/`pop_call_stack` to emit samples |
+| `tracegrind/threads.c` | Save/restore per-thread `last_sample_cost` |
+| `tracegrind/main.c` | Update tool name/description, init/fini for trace output |
+| `tracegrind/clo.c` | Add `--tracegrind-out-file`, remove callgraph-only options |
+| `tracegrind/Makefile.am` | Renamed from callgrind references |
+| `Makefile.am` (root) | Add `tracegrind` to TOOLS |
+| `configure.ac` | Add `tracegrind/Makefile` to AC_CONFIG_FILES |
+
+## Existing code to reuse (NOT rewrite)
+- **Cache simulation**: `tracegrind/sim.c` (unchanged from callgrind, includes `cachegrind/cg_arch.c`)
+- **Event/cost infrastructure**: `tracegrind/events.c`, `tracegrind/costs.c` (unchanged)
+- **BB/BBCC management**: `tracegrind/bb.c`, `tracegrind/bbcc.c` (unchanged)
+- **Function/jump tracking**: `tracegrind/fn.c`, `tracegrind/jumps.c` (unchanged)
+- **Client requests**: `tracegrind/main.c` handler (unchanged trapdoor logic)
+- **Instrumentation**: `tracegrind/main.c` instrument function (unchanged)
+
+## CSV Output Example
+```csv
+seq,tid,event,fn,obj,file,line,Ir,I1mr,ILmr,Dr,D1mr,DLmr,Dw,D1mw,DLmw,Bc,Bcm,Bi,Bim,sysCount,sysTime
+1,1,ENTER,main,/usr/bin/myapp,main.c,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
+2,1,ENTER,compute,/usr/bin/myapp,math.c,42,1523,12,3,487,8,2,102,4,1,95,3,0,0,0,0
+3,1,EXIT,compute,/usr/bin/myapp,math.c,42,8734,45,11,2104,31,7,512,15,4,401,12,2,1,0,0
+4,1,ENTER,write,/lib/x86_64-linux-gnu/libc.so.6,io.c,10,234,2,0,89,1,0,45,1,0,12,0,0,0,0,0
+5,1,EXIT,write,/lib/x86_64-linux-gnu/libc.so.6,io.c,10,1456,8,2,534,5,1,267,3,1,78,2,1,0,2,15000
+6,1,EXIT,main,/usr/bin/myapp,main.c,5,312,3,1,98,2,0,34,1,0,15,1,0,0,0,0
+```
+- Verification: sum of all delta rows across all columns should match callgrind's summary totals