Skip to content

Commit 0a5606c

Browse files
author
Alex J Lennon
committed
feat(remote): SSH gdbserver transport, board example, and design principles
Add remote_ssh backend with optional scp upload before ssh gdbserver; wire config and proxy connect path. Include board_test_app example (Makefile, rsgdb.remote.toml) plus install_ssh_key and debug_remote helper scripts with SSH key preflight. Document design principles (ease, automation, reliability), Linux target SSH setup, and CONTRIBUTING cross-links. Update rsgdb.toml.example and gitignore for the example binary. Made-with: Cursor
1 parent 660e3af commit 0a5606c

17 files changed

Lines changed: 798 additions & 17 deletions

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# Example board test binary (built locally)
2+
examples/board_test_app/board_test_app
3+
14
# Rust
25
/target/
36
**/*.rs.bk

CONTRIBUTING.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,9 @@ Work is tracked in [GitHub issues](https://github.com/DynamicDevices/rsgdb/issue
117117

118118
**Foundation (closed issues):** Part A (**#1#3**), session recording (**#4**), SVD baseline (**#5**), breakpoint/semihosting spike (**#6**), flash (**#7**), RTOS decode/log (**#8**). **Phase A/B** in-tree: RSP matrix + proxy tests, gdbinit example, `rsgdb::rtos` decode logs — see README **Key Features**.
119119

120-
**Current capabilities (same as README “Current”):** RSP codec + TCP proxy, **managed native stub spawn** (`transport = native`, `[backend.spawn]` with `{port}`), `tracing` logging, TOML/env config, JSONL **record** + **`rsgdb replay`**, SVD register/field/enum-name memory annotations, `rsgdb flash`, RTOS packet summaries, CI + optional GDB/Zephyr E2E scripts.
120+
**Current capabilities (same as README “Current”):** RSP codec + TCP proxy, **managed native stub spawn** (`transport = native`, `[backend.spawn]` with `{port}`), **SSH remote gdbserver** (`transport = remote_ssh`, `[backend.remote_ssh]`; optional **`upload_local`/`upload_remote`**`scp` before `ssh`; local `ssh`/`scp` + optional `RSGDB_SSH_PASSWORD`/`sshpass`), `tracing` logging, TOML/env config, JSONL **record** + **`rsgdb replay`**, SVD register/field/enum-name memory annotations, `rsgdb flash`, RTOS packet summaries, CI + optional GDB/Zephyr E2E scripts.
121+
122+
**Project aim — zero-touch remote debugging:** move toward **one configured flow** (target address, SSH user, credentials via key or env) that automates **upload → gdbserver → proxy** without ad-hoc copy steps; see README **Design principles**, **Project Goals**, and roadmap row **Zero-touch remote debug**.
121123

122124
**Roadmap — follow-ups:** Deeper probe integration (beyond managed TCP spawn; CLI `backend_type` stays a label). **Optional:** SVD value decode + recording correlation (see [#11](https://github.com/DynamicDevices/rsgdb/issues/11) history); TUI, logging export, proxy-side breakpoint management — open an issue before large changes. [#9](https://github.com/DynamicDevices/rsgdb/issues/9) tracks native-spawn history; close with `Closes #9` when you consider it fully done from the project’s side.
123125

@@ -131,6 +133,11 @@ Work is tracked in [GitHub issues](https://github.com/DynamicDevices/rsgdb/issue
131133

132134
Use this when a probe and target are available.
133135

136+
**Linux target over SSH (`transport = remote_ssh`) — setup first**
137+
138+
1. **One-time:** install your SSH public key on the target so `ssh`/`scp` and rsgdb do not depend on interactive passwords. From the repo root: `./examples/board_test_app/install_ssh_key.sh` (override host/user/port as needed; see `examples/board_test_app/README.md`).
139+
2. Point your config at `[backend.remote_ssh]` + `[proxy]` and connect GDB through rsgdb as in the main README. Optional: `examples/board_test_app/debug_remote.sh` for an end-to-end smoke.
140+
134141
**Option A — stub already running (`transport = tcp`, default)**
135142
1. Start your **backend** (e.g. OpenOCD) and note its **GDB TCP port** (often `3333`).
136143
2. Start **rsgdb** so it listens for GDB and forwards to that port, e.g.

README.md

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,18 @@
77

88
A Rust GDB **RSP proxy** with structured logging, optional CMSIS-SVD memory labels, JSONL session recording/replay, RTOS packet decode logs, and external flash orchestration — with room to grow into richer breakpoints and UIs.
99

10+
## Design principles
11+
12+
rsgdb exists to make **remote debugging practical for embedded developers**: boards on the bench, in the lab, or over the network, without unnecessary ceremony. **Automation** and **reliability** are first-class goals, not afterthoughts.
13+
14+
| Principle | What it means in practice |
15+
|-----------|---------------------------|
16+
| **Ease** | Few steps from “target is reachable” to GDB inspecting your ELF; sensible defaults; documented flows (config + helper scripts) that match how teams actually work. |
17+
| **Automation** | One place to describe the target (address, transport, optional binary upload); **`remote_ssh`** with optional **`scp`** so you are not hand-copying builds; preflight checks (e.g. SSH key access) before a long attach fails halfway. |
18+
| **Reliability** | Fail **early** with **actionable** errors (auth, ports, firewall, stub not listening); predictable behavior and exit codes; logging you can use in the field; CI and local E2E smoke where we can prove the path. |
19+
20+
These principles align with **zero-touch remote debugging** below and inform roadmap work (fewer manual steps, clearer credential UX, stronger health checks over time).
21+
1022
## 🎯 Project Goals
1123

1224
**rsgdb** aims to bridge the gap between traditional GDB debugging and modern embedded development needs by providing:
@@ -15,7 +27,8 @@ A Rust GDB **RSP proxy** with structured logging, optional CMSIS-SVD memory labe
1527
- **Advanced Breakpoint Management** (roadmap): Named breakpoints, conditional expressions, and hardware/software optimization — config and parsing exist; the proxy today **forwards** breakpoint RSP unchanged
1628
- **State Inspection** (partial today): Peripheral/register **labels** for memory traffic via CMSIS-SVD in logs; snapshots / deep state are not implemented yet
1729
- **Session Management**: JSONL **recording** and **`rsgdb replay`** mock-backend playback (see below)
18-
- **Backend Flexibility** (today): **`transport = tcp`** connects to an existing stub on **`proxy.target_host`:`proxy.target_port`**. **`transport = native`** spawns a configured command (`[backend.spawn] program` with `{port}`), waits for TCP on `bind_host`, then uses the same RSP path; the child is killed when the GDB session ends ([#9](https://github.com/DynamicDevices/rsgdb/issues/9)). `[backend].backend_type` / `--backend` is a **label** for logs only.
30+
- **Backend Flexibility** (today): **`transport = tcp`** connects to an existing stub on **`proxy.target_host`:`proxy.target_port`**. **`transport = native`** spawns a configured command (`[backend.spawn] program` with `{port}`), waits for TCP on `bind_host`, then uses the same RSP path; the child is killed when the GDB session ends ([#9](https://github.com/DynamicDevices/rsgdb/issues/9)). **`transport = remote_ssh`** runs **`gdbserver` on the target** over SSH; optional **`upload_local` / `upload_remote`** runs **`scp`** first so you are not forced to copy binaries by hand. `[backend].backend_type` / `--backend` is a **label** for logs only.
31+
- **Zero-touch remote debugging** (direction / aim): Reduce manual steps for **Linux targets** (e.g. Yocto boards): one config with **target address**, **login** (SSH user), and **credential** (SSH key or `RSGDB_SSH_PASSWORD` + `sshpass`) should drive **upload → gdbserver → proxy → GDB** where possible; more automation and polish are expected over time — see **Design principles** above and roadmap below.
1932
- **Modern Architecture**: Built with Rust for safety, performance, and reliability
2033

2134
## ✨ Key Features
@@ -29,7 +42,8 @@ A Rust GDB **RSP proxy** with structured logging, optional CMSIS-SVD memory labe
2942
- ▶️ **`rsgdb replay`** — load a recording and serve a **mock TCP backend** for one client (order-preserving playback / tests)
3043
- 📝 **SVD annotation (read-only)** — CMSIS-SVD → log labels for memory RSP (`m` / `M`): `Peripheral.REGISTER`, overlapping **fields**, and enumerated **variant names** where present (`target: rsgdb::svd`, debug)
3144
-**`rsgdb flash`** — run a configured external flash tool (`[flash].program` with `{image}` substitution; OpenOCD/probe-rs/etc.)
32-
- 🔌 **`transport = native`** — spawn a GDB stub per session (`[backend.spawn] program` with `{port}`); TCP to loopback; teardown on disconnect
45+
- 🔌 **`transport = native`** — spawn a GDB stub **on this machine** per session (`[backend.spawn]` + `{port}`); teardown on disconnect
46+
- 🖧 **`transport = remote_ssh`** — run **`gdbserver` on the target** via **`ssh user@host …`** (`[backend.remote_ssh]` + `{port}`); optional **`upload_local`/`upload_remote`****`scp`** first; TCP to `proxy.target_host`:`proxy.target_port`; optional `RSGDB_SSH_PASSWORD` + `sshpass`
3347
- 🧵 **RTOS RSP decode / log (Zephyr-first)** — thread-extension packets are decoded and logged at `target: rsgdb::rtos` (debug). Thread *data* comes from your stub (e.g. OpenOCD **Zephyr** RTOS awareness); other RTOSes use the same GDB RSP when the stub implements them (see below).
3448
- 🧪 **CI + local E2E smoke**`gdbserver``rsgdb``gdb` (batch), `scripts/e2e_gdb_smoke.sh` (Ubuntu job in **CI** workflow). **Zephyr `native_sim`** E2E (`scripts/e2e_zephyr_native_sim.sh`) runs in the **Zephyr E2E** workflow when those scripts/app change, on `main`/`develop`, weekly, or manually. See [CONTRIBUTING.md](CONTRIBUTING.md).
3549
-**Phase A (trust path)** — RSP codec matrix tests (`tests/rsp_codec_matrix.rs`, `scripts/e2e_rsp_regression.sh`), proxy TCP integration tests (`tests/proxy_integration.rs`).
@@ -86,12 +100,25 @@ rsgdb is a **transparent TCP proxy**: GDB speaks RSP to rsgdb; rsgdb forwards th
86100
| **rsgdb** | `0.0.0.0:3333``target_host:target_port` | `target extended-remote host:3333` |
87101
| **GDB** || rsgdb listen port |
88102

89-
**Choosing `tcp` vs `native`**
103+
**Choosing `tcp` vs `native` vs `remote_ssh`**
90104

91105
| Use | When |
92106
|-----|------|
93107
| **`transport = tcp`** (default) | The stub is **already running** (you started OpenOCD, probe-rs gdb, gdbserver, …). rsgdb **dials** `proxy.target_host`:`proxy.target_port`. |
94-
| **`transport = native`** | You want rsgdb to **spawn** the stub per GDB session with `[backend.spawn] program` and `{port}`, then connect to `bind_host` at that port. Ignores `target_host` / `target_port` for the backend. Kills the stub when GDB disconnects. |
108+
| **`transport = native`** | You want rsgdb to **spawn** the stub **on this machine** per GDB session with `[backend.spawn] program` and `{port}`, then connect to `bind_host` at that port. Kills the stub when GDB disconnects. |
109+
| **`transport = remote_ssh`** | The stub runs **on a remote host** (e.g. Yocto board). Optional **`upload_local`** + **`upload_remote`** run **`scp`** first (same auth as SSH). Then rsgdb runs **`ssh user@host …`** with `[backend.remote_ssh] program` (must include `{port}``proxy.target_port`), then connects TCP to **`proxy.target_host`:`proxy.target_port`**. Kills the **local** `ssh` process when GDB disconnects (typically ends remote `gdbserver`). Requires **OpenSSH** `ssh`/`scp` on PATH; optional **`RSGDB_SSH_PASSWORD`** + **`sshpass`** for non-interactive password auth. |
110+
111+
#### Setting up a Linux target for `remote_ssh` debugging
112+
113+
Do this **once per host/user** so `ssh`, `scp`, and rsgdb agree on the same auth (no password prompts in normal use).
114+
115+
1. **Install your SSH public key on the target (recommended)** — from the repository root, run [`examples/board_test_app/install_ssh_key.sh`](examples/board_test_app/install_ssh_key.sh). Defaults match the example [`examples/board_test_app/rsgdb.remote.toml`](examples/board_test_app/rsgdb.remote.toml) (`fio` @ `192.168.2.139`). Override with `SSH_HOST`, `SSH_USER`, `SSH_PORT`, or positional `host` / `user`:
116+
```bash
117+
./examples/board_test_app/install_ssh_key.sh
118+
```
119+
If you must pass the account password non-interactively (e.g. first-time automation), set `RSGDB_SSH_PASSWORD` and install **`sshpass`**; the script uses the same variable as rsgdb.
120+
2. **Or use password auth** — omit keys and rely on interactive prompts, or set **`RSGDB_SSH_PASSWORD`** + **`sshpass`** for rsgdb/`scp` (see table above).
121+
3. **Verify**`ssh -p <port> user@host` should succeed without a password after step 1. Then use your `[backend.remote_ssh]` + `[proxy]` config, or follow [`examples/board_test_app/README.md`](examples/board_test_app/README.md) for a full smoke (`debug_remote.sh`).
95122

96123
**Config:** `[proxy] listen_port`, `target_host`, `target_port`. **`timeout_secs`** applies only to **establishing** the TCP connection to the backend, not to idle GDB sessions (no read timeout on open connections).
97124

@@ -307,11 +334,13 @@ Source of truth for ordering and scope: **[GitHub Issues](https://github.com/Dyn
307334
| **Native spawn backend** | `BackendTransport::Native` + `[backend.spawn]` (`{port}`), managed `Child` lifecycle, stderr capture | [#9](https://github.com/DynamicDevices/rsgdb/issues/9) (implemented) |
308335
| **Replay** | `rsgdb replay` + mock TCP backend from `.jsonl` | [#10](https://github.com/DynamicDevices/rsgdb/issues/10) (closed) |
309336
| **Richer SVD** | Overlapping fields + enum variant names in annotations; value decode / recording correlation follow-ups | [#11](https://github.com/DynamicDevices/rsgdb/issues/11) (baseline closed; follow-ups optional) |
337+
| **Zero-touch remote debug** | Fewer manual steps: remote IP + SSH identity + optional `scp` upload + `remote_ssh` gdbserver orchestration; expand credential UX and workflows over time | Aim (see **Project Goals**) |
310338

311339
Older versioned bullets (v0.2–v0.4) below are **aspirational**; issue titles supersede them.
312340

313341
### Aspirational (not scheduled per-issue yet)
314342
- Enhanced logging export (JSON/CSV), advanced breakpoints, TUI, performance work — see **Planned** under Key Features and open an issue when starting.
343+
- Deeper **zero-touch remote debugging** (beyond current `scp` + `remote_ssh`): e.g. integrated workflows, fewer external tools, clearer security story for secrets — track as project aim above.
315344

316345
## 📄 License
317346

examples/board_test_app/Makefile

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Cross-compile for the target board (aarch64 example).
2+
# Yocto/Poky: source <sdk>/environment-setup-aarch64-poky-linux && make CC="$CC"
3+
#
4+
# Default is aarch64 cross; a shell-exported CC overrides this — use `make CC=aarch64-linux-gnu-gcc`
5+
# or `env -u CC make` if your environment forces host cc.
6+
CC := aarch64-linux-gnu-gcc
7+
CFLAGS ?= -O0 -g -Wall -Wextra -std=c11
8+
LDFLAGS ?=
9+
TARGET := board_test_app
10+
11+
.PHONY: all clean
12+
13+
all: $(TARGET)
14+
15+
$(TARGET): main.c
16+
$(CC) $(CFLAGS) -o $@ $< $(LDFLAGS)
17+
18+
clean:
19+
rm -f $(TARGET)

examples/board_test_app/README.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# board_test_app — GDB smoke test on a Linux target
2+
3+
Tiny C program: infinite loop, `volatile` globals/locals you can watch in GDB (`print g_counter`, breakpoints on `printf`, etc.).
4+
5+
## First-time target setup (SSH)
6+
7+
Before debugging with **`transport = remote_ssh`** (so `scp` + `ssh` work without typing a password every time):
8+
9+
1. **Install your public key on the board** (one-time). From the **repository root**:
10+
```bash
11+
./examples/board_test_app/install_ssh_key.sh
12+
```
13+
Defaults match `rsgdb.remote.toml` in this directory (`fio` @ `192.168.2.139`). Use `SSH_HOST`, `SSH_USER`, `SSH_PORT`, or `./examples/board_test_app/install_ssh_key.sh <host> <user>` to match your board. For a non-interactive password on that first run: `export RSGDB_SSH_PASSWORD=…` (requires **`sshpass`**).
14+
2. Confirm **`ssh`** to the target works without a password.
15+
3. Continue with **Build** and **Debug** below. See the main README **Setting up a Linux target for `remote_ssh` debugging** for the full project-wide checklist.
16+
17+
## Build (aarch64 default)
18+
19+
The Makefile defaults to **`aarch64-linux-gnu-gcc`**:
20+
21+
```bash
22+
cd examples/board_test_app
23+
make
24+
```
25+
26+
Host-only smoke build (x86_64): `make CC=gcc`
27+
28+
With a **Yocto/Poky SDK** (adjust path), use the SDK compiler:
29+
30+
```bash
31+
source /path/to/sdk/environment-setup-aarch64-poky-linux
32+
cd examples/board_test_app
33+
make CC="$CC"
34+
```
35+
36+
## Deploy to the board
37+
38+
**Manual:** `scp` the binary and `chmod +x` on the target.
39+
40+
**With rsgdb** (`transport = remote_ssh`): set `upload_local` to this built `board_test_app` path and `upload_remote` to e.g. `/tmp/board_test_app` — rsgdb can **`scp`** before starting `gdbserver` (see main README).
41+
42+
## Debug (gdbserver on board)
43+
44+
### Automated (rsgdb `remote_ssh` + `scp`)
45+
46+
From the **repository root**, with the board reachable at the address in `rsgdb.remote.toml` (default `192.168.2.139`):
47+
48+
```bash
49+
cargo build --release
50+
cd examples/board_test_app && make && cd ../..
51+
export RSGDB_SSH_PASSWORD=… # if using password auth; prefer SSH keys
52+
./examples/board_test_app/debug_remote.sh
53+
```
54+
55+
Use **`gdb-multiarch`** on Ubuntu (not always `aarch64-linux-gnu-gdb`). Pass an **absolute** path to `file` if you run GDB by hand:
56+
57+
```bash
58+
gdb-multiarch -ex "set debuginfod enabled off" \
59+
-ex "file $(pwd)/examples/board_test_app/board_test_app" \
60+
-ex "target extended-remote 127.0.0.1:3333"
61+
```
62+
63+
### Manual gdbserver on target
64+
65+
On the **target**:
66+
67+
```bash
68+
/tmp/board_test_app &
69+
/tmp/gdbserver :2345 --attach $!
70+
# or: gdbserver :2345 /tmp/board_test_app
71+
```
72+
73+
On the **host**: `transport = tcp` in rsgdb; point GDB at `127.0.0.1:<rsgdb listen port>`.
74+
75+
```text
76+
(gdb) file /absolute/path/to/board_test_app
77+
(gdb) target extended-remote 127.0.0.1:3333
78+
```
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
#!/usr/bin/env bash
2+
# End-to-end smoke: rsgdb (remote_ssh + scp) + gdb-multiarch to the board.
3+
# Usage from repository root:
4+
# export RSGDB_SSH_PASSWORD=yourpassword # if not using SSH keys
5+
# ./examples/board_test_app/debug_remote.sh
6+
set -euo pipefail
7+
8+
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
9+
cd "$ROOT"
10+
11+
BIN="${ROOT}/examples/board_test_app/board_test_app"
12+
CFG="${ROOT}/examples/board_test_app/rsgdb.remote.toml"
13+
14+
if [[ ! -f "$BIN" ]]; then
15+
echo "Build the app first: cd examples/board_test_app && make" >&2
16+
exit 1
17+
fi
18+
19+
RSGDB_BIN="${ROOT}/target/release/rsgdb"
20+
if [[ ! -x "$RSGDB_BIN" ]]; then
21+
echo "Building rsgdb release…" >&2
22+
( cd "$ROOT" && cargo build --release )
23+
fi
24+
25+
command -v gdb-multiarch >/dev/null || {
26+
echo "Install gdb-multiarch (e.g. sudo apt install gdb-multiarch)" >&2
27+
exit 1
28+
}
29+
30+
# Match rsgdb.remote.toml (same defaults as install_ssh_key.sh). Skipped when using password auth.
31+
check_ssh_key_access() {
32+
if [[ -n "${RSGDB_SSH_PASSWORD:-}" ]]; then
33+
echo "RSGDB_SSH_PASSWORD is set; skipping SSH key check." >&2
34+
return 0
35+
fi
36+
local ssh_host ssh_user ssh_port
37+
ssh_host=$(awk -F'"' '/target_host/ {print $2; exit}' "$CFG")
38+
ssh_user=$(awk -F'"' '/^user =/ {print $2; exit}' "$CFG")
39+
ssh_port="${SSH_PORT:-}"
40+
if [[ -z "$ssh_port" ]] && grep -qE '^ssh_port[[:space:]]*=' "$CFG" 2>/dev/null; then
41+
ssh_port=$(awk -F'=' '/^ssh_port/ {gsub(/[^0-9]/,"",$2); print $2; exit}' "$CFG")
42+
fi
43+
[[ -z "$ssh_port" ]] && ssh_port=22
44+
if [[ -z "$ssh_host" || -z "$ssh_user" ]]; then
45+
echo "Could not parse target_host / user from $CFG" >&2
46+
exit 1
47+
fi
48+
if ! ssh -p "$ssh_port" -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new \
49+
"${ssh_user}@${ssh_host}" "echo ok" >/dev/null 2>&1; then
50+
echo "SSH key access check failed for ${ssh_user}@${ssh_host} (port ${ssh_port})." >&2
51+
echo "Install your key: ./examples/board_test_app/install_ssh_key.sh" >&2
52+
echo "Or use password auth: export RSGDB_SSH_PASSWORD=... && $0" >&2
53+
exit 1
54+
fi
55+
echo "SSH key access OK (${ssh_user}@${ssh_host}:${ssh_port})" >&2
56+
}
57+
58+
check_ssh_key_access
59+
60+
fuser -k 3333/tcp 2>/dev/null || true
61+
sleep 1
62+
63+
echo "Starting rsgdb (scp + ssh gdbserver on connect)…" >&2
64+
"$RSGDB_BIN" --config "$CFG" 2>&1 &
65+
RPID=$!
66+
sleep 2
67+
68+
cleanup() {
69+
kill "$RPID" 2>/dev/null || true
70+
wait "$RPID" 2>/dev/null || true
71+
}
72+
trap cleanup EXIT
73+
74+
echo "Connecting GDB (batch)…" >&2
75+
gdb-multiarch -batch -nx \
76+
-ex "set debuginfod enabled off" \
77+
-ex "file $BIN" \
78+
-ex "target extended-remote 127.0.0.1:3333" \
79+
-ex "print g_counter" \
80+
-ex "detach" \
81+
-ex "quit"
82+
echo "OK." >&2

0 commit comments

Comments
 (0)