Skip to content

Latest commit

 

History

History
936 lines (749 loc) · 40.5 KB

File metadata and controls

936 lines (749 loc) · 40.5 KB

Extracting sensor drivers from binary firmware with ipctool trace

A guide for researchers who need to recover an image-sensor init sequence (and the surrounding ISP/MIPI setup) from a running but source-less streamer — typically because the camera vendor has stopped shipping updates and the only remaining trace of the driver lives inside an .so or a statically-linked binary.

This document walks through the full pipeline end-to-end, using SmartSens SC2315E on a Hisilicon HI3516EV200 as the worked example, captured separately from OpenIPC's Majestic streamer and from XiongMai's proprietary Sofia. Both produce a register-by-register identical init sequence — and one that matches the public OpenIPC smart_sc2315e reference source exactly.

What ipctool trace does

It is a ptrace(2)-based syscall interceptor, similar in spirit to strace, but specialised for camera I/O. It hooks open, read, write, ioctl, and nanosleep, and decodes the device-specific payloads on the fly:

  • /dev/i2c-*, /dev/hi_i2csensor_write_register(0x..., 0x..) / sensor_read_register(0x...) /* -> 0x.. */ / sensor_i2c_change_addr(0x..)
  • /dev/spidev*, /dev/sspssp_write_register(...) / SPI message dumps
  • /dev/hi_mipi, /dev/vi → pretty-printed combo_dev_attr_t SENSOR_ATTR = {...}
  • /dev/xm_gpio, /sys/class/gpio/... → GPIO request/direction/write events
  • /dev/mtd* → MTD ioctl dumps

The output is a stream of C-pseudocode interleaved with bus-banner markers (========================== i2c-29 ==========================). ARM-only (the syscall-number table is hardcoded for ARM 32-bit EABI).

Quick worked example

# build  (downloads OpenIPC's CI toolchain, UPX-packs the binary)
$ tools/capture_sensor.sh build

# capture from a Majestic camera over ssh
$ tools/capture_sensor.sh majestic --host openipc-hi3516ev200.lan \
                                   --secs 35 --out tools/dumps/cap.log

# segment, generate, diff against a known-good vendor source
$ python3 tools/trace_segment.py    tools/dumps/cap.log
$ python3 tools/trace_to_driver.py  tools/dumps/cap.log.segments.json \
                                    --sensor sc2315e \
                                    --out tools/dumps/cap.c
$ python3 tools/trace_diff.py       tools/dumps/cap.c \
                                    /path/to/smart_sc2315e/sc2315e_sensor_ctl.c \
                                    --gen-scope sc2315e_linear_init \
                                    --ref-scope sc2315e_linear_1080P30_init

generated: tools/dumps/cap.c (172 writes, 169 unique regs)
reference: smart_sc2315e/sc2315e_sensor_ctl.c (172 writes, 169 unique regs)

address match:  169 / 169 ref regs present (100.0%)
value match:    169 / 169 matching addrs (100.0%)
sequence (LCS): 100.0% of max(len)

A 100/100/100 match confirms that what the binary writes to the sensor is exactly what the published vendor driver writes. If you trust the reference, the trace gives you a buildable scaffold.

Pipeline overview

camera (ARM)                         host (x86)
─────────────                        ──────────
streamer                ┌─────────► trace_segment.py
   │                    │              │
   ▼                    │              ▼
ipctool trace ──► .log file ──► trace_to_driver.py ──► .c scaffold
                                       │                    │
                                       ▼                    ▼
                                  segments.json     trace_diff.py vs reference

Each stage is dependency-free Python (segment / generate / diff) and one shell wrapper for the camera-side capture (capture_sensor.sh). Nothing needs to be installed on the camera beyond the staged ipctool binary.

Stage 1 — Build a portable ipctool

You need an ARM static binary that will run on whichever embedded kernel the target camera ships. Cross-compile flags vary, but two non-obvious constraints apply:

1. Use the OpenIPC CI toolchain

Generic Buildroot/musl ARM toolchains often produce binaries with slightly different code generation than the canonical CI build, which is what every released ipctool tested against. Stick with the toolchain referenced in .github/workflows/release-arm32.yml:

wget -qO- \
  https://github.com/OpenIPC/firmware/releases/download/toolchain/toolchain.hisilicon-hi3516cv100.tgz \
  | tar xzf - -C /opt

Then build with direct compiler override, no toolchain file:

PATH=/opt/arm-openipc-linux-musleabi_sdk-buildroot/bin:$PATH \
  cmake -H. -Bbuild \
        -DCMAKE_C_COMPILER=arm-openipc-linux-musleabi-gcc \
        -DCMAKE_BUILD_TYPE=Release
cmake --build build

2. UPX-pack the binary

Some embedded kernels (notably XiongMai HiLinux based on Linux 4.9) will refuse to load a regular musl-static ELF and the busybox shell falls back to interpreting the file as a script:

/utils/ipctool: line 1: syntax error: unexpected word (expecting ")")

That cryptic message is the kernel returning ENOEXEC for the binary, followed by sh trying to read the ELF magic as shell. UPX wraps the binary in a tiny self-decompressing stub with a single PT_LOAD segment and (as a side effect) sets the OS/ABI byte to UNIX - GNU. Both the OpenIPC and XiongMai kernels accept that:

upx --best build/ipctool -o build/ipctool-upx

The released ipctool from the OpenIPC GitHub releases is UPX-packed for exactly this reason. tools/capture_sensor.sh build does this automatically.

Stage 2 — Camera-side capture

ipctool trace forks, ptrace(TRACEME)s, and execvs the target binary. All clones/forks are followed via PTRACE_O_TRACECLONE, so a multi-threaded streamer is captured as one stream.

The --output=PATH flag (added in this branch) is important: without it, the trace pseudocode shares the child's stdout, and verbose streamers (Majestic at INFO level, Sofia with its [31m...[0m ANSI-coloured logs) interleave their own log lines with ours, sometimes truncating mid-token. With --output=PATH, the parent freopens its stdout to PATH after fork, so the child's stdout keeps pointing at the original tty.

Capturing from Majestic (OpenIPC)

Majestic is a "good citizen": no watchdog, no path-based supervisors, restartable via /etc/init.d/S95majestic. Capture is straightforward:

# uses ssh, expects passwordless key-based access
tools/capture_sensor.sh majestic --host openipc-hi3516ev200.lan \
                                 --secs 35

What it does:

killall majestic                                  # stop the live stream
ipctool-upx trace --output=/utils/dumps/log \
                  /usr/bin/majestic &             # run a fresh one under trace
sleep 35                                          # init + a few seconds AE
killall ipctool-upx majestic                      # stop the trace
/etc/init.d/S95majestic start                     # restore camera service

Output is a clean trace with no log interleaving. About 3000 lines for 40 s of capture (init + a few hundred AE-loop iterations).

Capturing from XiongMai Sofia (HiLinux)

This is harder for three independent reasons. Document each gotcha separately so you can recognise them on adjacent firmware variants.

(a) Sofia is a network client of XmServices_Mgr, not a peer. Sofia opens a local TCP socket to talk to XmServices_Mgr for system services like GetWritableDir. If XmServices_Mgr is dead, Sofia spins forever in LibXmComm_NetTcp_recvPacket: timeout, never reaching sensor init. So the capture flow must keep XmServices_Mgr alive.

(b) XmServices_Mgr is not a supervisor. Despite the name, it forks SofiaRun.sh exactly once at boot — from /etc/init.d/rcS — and does not restart it if Sofia or SofiaRun.sh exits. Killing Sofia leaves the camera with no running streamer until manual intervention.

(c) The watchdog is fed by XmServices_Mgr, not Sofia. Killing Sofia by itself does not trigger a hardware reboot, so you have time. But killing XmServices_Mgr will reboot the camera within ~30 s.

The working recipe is a bind-mount wrapper: insert ipctool between SofiaRun.sh and /usr/bin/Sofia without disturbing XmServices_Mgr:

# 1. Stage a copy of the real Sofia under a path the wrapper can exec
cp /usr/bin/Sofia /utils/Sofia.real

# 2. Create a wrapper that runs Sofia under trace, redirecting trace
#    output to a file (Sofia's own stdout/stderr go to /dev/null as
#    they did under the real SofiaRun.sh anyway)
cat > /utils/sofia.wrap.sh <<'EOF'
#!/bin/sh
exec /utils/ipctool-upx trace --skip=usleep \
                              --output=/utils/dumps/sofia.log \
                              /utils/Sofia.real
EOF
chmod +x /utils/sofia.wrap.sh

# 3. Make /usr/bin/Sofia point at the wrapper. Squashfs is read-only,
#    so we use a bind mount - the inode of /usr/bin/Sofia is now
#    /utils/sofia.wrap.sh.
mount --bind /utils/sofia.wrap.sh /usr/bin/Sofia

# 4. Kill Sofia and SofiaRun.sh, then restart SofiaRun.sh manually -
#    XmServices_Mgr will not do this for us. The < /dev/null is
#    important: a backgrounded shell pipeline that contains a
#    ptraced-stopped child receives SIGTTIN if any process in it
#    tries to read from the controlling tty. /dev/null skips that.
killall Sofia SofiaRun.sh
sleep 1
/usr/sbin/SofiaRun.sh </dev/null >/dev/null 2>&1 &

# 5. Wait long enough for sensor init + a few seconds of steady state
sleep 55

# 6. Reboot. The bind mount is ephemeral and goes away cleanly.
reboot

tools/capture_sensor.sh sofia automates this through expect(1):

tools/capture_sensor.sh sofia --host 10.216.128.106 --secs 55

What --skip=usleep is for here: Sofia spends a lot of time busy-polling nanosleep(0) (yield) while it waits for various subsystems to come up. With --skip=usleep, those entries don't pollute the trace, and we still keep the structurally meaningful sleeps that matter for phase segmentation. (Majestic uses real sleeps so --skip=usleep would be lossy there; the script keeps it on by default.)

Why these recipes don't transfer wholesale to other firmwares

The two camera-side recipes encode two different operational models:

Concern OpenIPC / Majestic XiongMai HiLinux / Sofia
Streamer restart init.d script Manual via the supervisor parent
Supervisor behaviour None — init.d script is fire-and-forget Spawns once, doesn't restart
Hardware watchdog Not in play during dev Fed by supervisor; ~30 s to reboot
Streamer log destination stdout (mixes with trace if not redirected) Same
ELF kernel acceptance Plain musl-static OK Requires UPX

For a third firmware, work out:

  1. Does the kernel accept your ARM static ELF? If sh: line 1: syntax error on launch, UPX-pack.
  2. Does anything respawn the streamer? Try kill <streamer-pid> and watch ps. If yes, you can let it respawn through your wrapper. If no, you'll need to start it manually after killing.
  3. Is there a watchdog? ls /dev/watchdog*. If yes, time-bound your capture and have a clean exit path.
  4. Does the streamer write to stdout? If yes, always use --output=PATH to avoid interleaving.

Stage 3 — Post-processing

Three Python scripts, no external dependencies, all read/write under tools/dumps/ by convention.

trace_segment.py

Splits the raw log into phases:

Phase What it contains
pre_sensor Bus probe, MIPI/VI struct dumps, pre-init noise
init From the sensor's standby/reset write to its stream-on write (see "Sensor-family init patterns" below)
mode_switch_N Each subsequent stream-off → reconfigure → stream-on cycle
post_init A short burst of AE/exposure prime writes between stream-on and the steady-state loop
runtime Per-frame writes during steady-state (e.g. AE updating exposure registers)

The init/post-init split exists for diff-friendliness: the AE loop in post_init typically rewrites the same exposure registers (0x3E00.., 0x320E/F, …) that init had set to default values. Comparing init-only against the reference's init function avoids spurious mismatches.

Sensor-family init patterns

Different sensor vendors gate "stream on" through different registers. The segmenter has a small table of (family, reg, init_val, stream_val) patterns and tries each in order; the first that finds both endpoints in a trace wins. The matched family is recorded as init_pattern in the segments JSON.

Family Register Init value Stream-on value Sensors
smartsens 0x0100 0x00 (reset) 0x01 (stream-on) SC2315E, SC2335, SC*, SmartSens generally
sony_imx 0x3000 0x01 (standby) 0x00 (release) IMX291, IMX385, IMX307, Sony IMX line
soi_jx 0x0012 0x40 (standby/reset) 0x00 (stream-on) JXF22, JXF23, JXH62, SOI / JX 8-bit register family

Adding a family is one entry in INIT_PATTERNS at the top of trace_segment.py. If your trace is recognised but no init phase is detected, your sensor probably uses a third pattern — write the addresses and values in here and the segmenter will pick it up.

If no pattern matches, the segmenter emits everything as pre_sensor and the generator skips the function-body emission. Most often that means your sensor uses a third stream-control register convention not yet in INIT_PATTERNS. Check the raw trace for the obvious bracket (a register written once near the start, then again near the end with the opposite value) and add an entry.

Decoder coverage across HiSilicon families

Different HiSilicon families take different paths to the I2C bus. ipctool's ptrace decoder handles each:

Family Sensor driver path What ipctool decodes
HISI_V1 ioctl(/dev/hi_i2c, CMD_I2C_WRITE, &I2C_DATA_S) xm_i2c_* callbacks decode the structured payload
HISI_V2 / V2A write(/dev/i2c-X, buf, reg+data) little-endian, after I2C_16BIT_REG/DATA ioctls i2c_write_exit_cb infers widths from nbyte, picks LE for V2/V2A; hisi_gen2_ioctl_exit_cb decodes I2C_SLAVE_FORCE
HISI_V3 / V3A / V4 / V4A ioctl(/dev/i2c-X, I2C_RDWR, &i2c_msg) big-endian hisi_i2c_read_*_cb decodes the rdwr message

uClibc on some V1/V2 firmwares wraps the libc write() call as a single-iovec writev() rather than direct __NR_write. ipctool handles both — syscall_writev_exit decodes the iovec and forwards to the same fd callback as plain write().

When threads share an fd table (CLONE_FILES, the standard for multi-thread streamers), opening a fd in one thread makes it usable in all of them. ipctool maintains this invariant explicitly: on open() it broadcasts the new fd state to every tracked process; on close() it clears it everywhere. Without this, a thread peer's write on a fd opened by the parent silently drops to no callback.

When the trace is empty anyway

A V1/V2 capture that shows 0 sensor_write_register lines despite the streamer reporting init success usually means one of:

  • ipctool segfaulted mid-trace. Symptom signature: trace ends at exactly i2c_read() (or shortly after the i2c-N banner), no further events, wait $! reports exit status 139 (SIGSEGV). The streamer keeps running untraced, so its real register-write burst goes unobserved and the captured log stays at ~10 lines. To confirm, capture a core dump and inspect the backtrace:

    ssh root@<camera> "ulimit -c unlimited; cd /tmp; \
      /tmp/ipctool trace --output=/tmp/dumps/trace.log /usr/bin/majestic"
    scp root@<camera>:/tmp/core /tmp/core
    arm-linux-gdb --batch -ex 'core /tmp/core' -ex 'bt full' \
      build-arm-ci/ipctool

    If the backtrace lands inside a decoder callback, the bug is in src/ptrace.c's callback (most likely a NULL-deref on the copy_from_process buffer or filename argument). The historical example was hisi_gen2_read_exit_cb doing memset(&buf, 0, …) instead of memset(buf, 0, …), nullifying the pointer before copy_from_process was called - this killed Hi3518EV200 + libsns_jxf22.so traces immediately after the first sensor-ID read.

  • Sensor .so opens its own I2C handle in a path our trace doesn't see. Check /proc/<streamer-pid>/fd while it's running: if the live /dev/i2c-N fd in the running process is different from the one the trace caught (or arrived later than the kill), capture for longer.

  • Sensor .so uses a HiSilicon-specific /dev/* device that isn't in our dispatch table (e.g. /dev/sys, /dev/sns_drv0). The signature is the trace ending shortly after i2c-N banner with no writes; live /proc/<pid>/fd shows the unfamiliar device open. Add it to syscall_open's dispatch in src/ptrace.c.

  • Sensor .so invokes a ptrace-incompatible code path (some vendor binaries detect ptrace and skip the writes; rare).

python3 tools/trace_segment.py tools/dumps/cap.log
# wrote tools/dumps/cap.log.segments.json
#   pre_sensor   39 events
#   init         173 events
#   post_init    15 events
#   runtime      2585 events

trace_to_driver.py

Emits HiSilicon-SDK-shaped C from the segmented JSON. Three functions per sensor:

  • <sensor>_linear_init — the cold-init register table.
  • <sensor>_post_init_exposure_prime — the AE/exposure values applied once when the AE loop first kicks in.
  • <sensor>_ae_step — a per-frame AE callback skeleton listing the registers the AE loop hits in steady state, with their last-seen values as placeholders. The math driving those values (gain LUTs, exposure scaling, threshold branches) is not in the trace; the skeleton points the reverse-engineer at the vendor's cmos_inttime_update / cmos_gains_update equivalents to fill in.

For SC2315E captured under steady ambient light, the _ae_step body matches the else-branches of the reference's cmos_inttime_update (0x3314=0x14) and cmos_gains_update (0x5781=0x60, 0x5785=0x30) — exactly the registers and values the running AE loop wrote each frame. A capture under varying lighting would surface the if-branches too; the skeleton would then need a conditional that you'd derive by hand from the value distribution.

For value-distribution data without a fresh trace, use ipctool sensor monitor (see "Live-reading the AE state" below).

The output is standalone-buildable — it includes a small "SDK stubs" block (typedef for VI_PIPE, a no-op sensor_write_register) so that gcc -fsyntax-only and gcc -c succeed without the vendor headers. Delete that block and replace it with #include "hi_comm_video.h" / #include "hi_sns_ctrl.h" plus the vendor's bus-aware sensor_write_register to integrate into a HiSilicon SDK build.

tools/test_pipeline.sh runs the full segment → generate → compile flow end-to-end on a synthetic trace and is wired into CI (pr-build-check.yml::test-extraction-pipeline), so a regression in any of the three Python scripts that breaks the generator output is caught at PR time.

python3 tools/trace_to_driver.py tools/dumps/cap.log.segments.json \
                                 --sensor sc2315e \
                                 --out tools/dumps/cap.c

The output is a scaffold, not a finished driver:

  • The init function is a register table you can hand-edit into the vendor's sensor_init callback.
  • The post_init_exposure_prime function hints at what the SDK's cmos_inittime_update sets up.
  • The _ae_step function is the runtime AE/AGC callback skeleton: the registers the AE loop touches every frame, with their last-seen trace values as /* TODO: derive */ placeholders. The math driving those values is not in the trace — see "Live-reading the AE state" below for how to capture value distributions.
  • File-scope MIPI/VI struct declarations (e.g., combo_dev_attr_t SENSOR_ATTR = { … }, pstViDevAttr VI_DEV_ATTR_S = { … }) are emitted between the SDK stubs and the init function, wrapped in #if 0 / #endif. Standalone builds skip them (the references to vendor enums like INPUT_MODE_MIPI aren't visible without hi_mipi.h); when integrating into a HiSilicon SDK build, remove the #if 0/#endif to drop the struct definitions in alongside the rest of the scaffold.

trace_diff.py

Diff-by-register-write. Extracts every sensor_write_register(reg, val) call (or the vendor-shaped <sensor>_write_register(ViPipe, reg, val) variant) from each file and reports:

  • address coverage (how many ref regs the trace actually wrote)
  • value match (last value seen per address agrees)
  • sequence similarity (LCS, order-aware)
  • per-side asymmetric diffs and value mismatches

--gen-scope and --ref-scope restrict each side to a named function body. Critical for the AE-overwrite issue mentioned above:

python3 tools/trace_diff.py \
    tools/dumps/cap.c \
    /path/to/smart_sc2315e/sc2315e_sensor_ctl.c \
    --gen-scope sc2315e_linear_init \
    --ref-scope sc2315e_linear_1080P30_init

Capturing mode switches

A "mode switch" here is a runtime sensor reconfiguration — switching 1080p25 to 720p, or linear to WDR — without a full streamer restart.

The capture-side mechanism is streamer-specific. OpenIPC Majestic does not support runtime mode switching: configuration changes go through /etc/sensors/*.ini files and require a streamer restart, so each mode is a separate cold-init capture. XiongMai Sofia does support several runtime knobs via the DVR-IP TCP protocol on port 34567; the python-dvr client exposes them. Example, toggling Sofia's BroadTrends.AutoGain knob:

from dvrip import DVRIPCam
cam = DVRIPCam('10.216.128.106', user='admin', password='')
cam.login()
cam.set_info("Camera.ParamEx.[0]",
             {"BroadTrends": {"AutoGain": 1, "Gain": 50}})

Whether a given knob actually causes a sensor-side reconfigure is sensor-specific — Sofia's BroadTrends.AutoGain path lands in software-side gain control on most sensors and only triggers a sensor-side WDR-mode change on sensors whose firmware has a separate WDR-mode init function. As a data point, when toggling AutoGain 0→1→0 on the SC2315E camera at 10.216.128.106 while ipctool trace was watching, the trace shows zero additional 0x100 cycles after init — the sensor stays in linear mode regardless.

The runtime path lands in Public_LinearWDR_SwitchLibXmCap_System_switchWdrModeXmCap_IspVi_switchWdrMode, which calls HI_MPI_ISP_GetPubAttr, possibly does a VI unbind/rebind dance, and persists the new mode to /mnt/mtd/Config/wdrmode.dat via libdvr.so's SetWdrMode (a thin fopen + fwrite of a 4-byte LE int). It does not call HI_MPI_ISP_SetWDRMode and does not invoke any sensor-driver code — so toggling at runtime never reconfigures the sensor itself. The persisted wdrmode.dat is the real sensor-mode knob — read at the next Sofia startup. See Boot-time WDR via wdrmode.dat below.

Boot-time WDR via wdrmode.dat

For Sofia builds that ship multi-mode sensor drivers (IMX290/IMX291 have been verified, others likely too — see the binary-string check below), the actual sensor-mode dispatch happens inside the sensor-driver layer at startup:

Sofia main → load /mnt/mtd/Config/wdrmode.dat (4 bytes LE)
           → cmos_set_image_mode(mode)
           → puts("linear mode" | "2to1 line WDR ..." | "...")
           → sensor_init() runs the register table for that mode

cmos_set_image_mode is a per-sensor function whose body is a switch on the requested mode that sets globals (genSensorMode, gu8SensorImageMode, gu32FullLinesStd, etc.) and prints a banner. Subsequent sensor_init consults those globals to pick which init register table to apply. For Sony IMX29x the typical case-set is:

mode banner sensor output
0 linear mode 1080p ≤30 fps SDR
2 2to1 line WDR 1080p mode (60fps→30fps) 1080p HDR @ 30 fps via line-interleaved DOL
3 2to1 half frame WDR mode spatial half-frame HDR
4 2to1 full frame WDR mode sequential full-frame HDR

Procedure

# 1. Set up the bind-mount Sofia trace wrapper as in the Sofia recipe above.
#    Backup the original mode setting:
cp /mnt/mtd/Config/wdrmode.dat /utils/dumps/wdrmode.dat.orig

# 2. Capture each mode of interest. Mode is a 4-byte LE int.
#    Mode 0 = linear (baseline):
printf '\x00\x00\x00\x00' > /mnt/mtd/Config/wdrmode.dat ; sync
killall Sofia ; sleep 1
WDR_TRACE=/utils/dumps/mode0.log /usr/bin/Sofia </dev/null \
    > /utils/dumps/mode0-stdout.log 2>&1 &
sleep 30 ; killall ipctool-upx Sofia

#    Mode 2 = line-WDR:
printf '\x02\x00\x00\x00' > /mnt/mtd/Config/wdrmode.dat ; sync
killall Sofia ; sleep 1
WDR_TRACE=/utils/dumps/mode2.log /usr/bin/Sofia </dev/null \
    > /utils/dumps/mode2-stdout.log 2>&1 &
sleep 30 ; killall ipctool-upx Sofia

# 3. Restore the original mode and reboot.
cp /utils/dumps/wdrmode.dat.orig /mnt/mtd/Config/wdrmode.dat ; sync
reboot

The wrapper reads WDR_TRACE from its env so the same /usr/bin/Sofia bind-mount serves both runs:

#!/bin/sh
exec /utils/ipctool-upx trace --output=$WDR_TRACE /utils/Sofia.real

Verifying the right mode fired

Sofia's cmos_set_image_mode prints the banner string before the init-register table runs. Grep the captured stdout to confirm:

$ grep -aE 'linear mode|WDR.*mode|LINE Init OK' mode0-stdout.log
linear mode
--------------------IMX290 1080P 30fps LINE Init OK!---

$ grep -aE 'linear mode|WDR.*mode|LINE Init OK' mode2-stdout.log
2to1 half frame WDR mode
--IMX291 1080P 60fps LINE Init OK!-------------------

The post-cmos init banners (30fps LINE Init OK, 60fps LINE Init OK) come from inside the per-mode init function and confirm the trace contains a different register sequence.

Pipeline output for the WDR capture

trace_segment.py recognises Sofia's two-pass init pattern (linear table baseline followed by mode-specific overrides) as init + mode_switch_1 automatically — same find_mode_switches heuristic used for genuine runtime mode switches. The Sony IMX 0x3000=01 ... 00 pattern is the bracket. trace_to_driver.py emits the WDR-mode body as <sensor>_set_mode_1 next to <sensor>_linear_init.

Worked example: IMX291 line-WDR vs linear

Captured on a Hi3516CV300 + IMX291 + Sofia camera (board model 50H20L):

metric mode 0 (linear) mode 2 (line-WDR)
trace size 20 KB 51 KB
total writes 70 140
unique reg/val pairs 61 69
init pattern detected sony_imx sony_imx
segmenter phases init(106) init(106) + mode_switch_1(114)
Sofia banner linear mode 2to1 half frame WDR mode (case 3) and 60fps LINE Init OK

WDR-only register writes (in mode 2, absent from mode 0):

0x3000 = 0x01     ← STANDBY assert before WDR reconfigure
0x3007 = 0x00     ← WIN_MODE clear (linear had 0x60)
0x3009 = 0x02     ← clock divider for 60 fps internal readout
0x300A = 0xF0     ← BLKLEVEL black-level override
0x3014 = 0x20     ← AGAIN initial value for WDR
0x3018 = 0x6D     ← VMAX[7:0] DOL frame timing
0x3046 = 0xE1     ← OPB_SIZE / DOL output config
0x3020,0x3021     ← SHS1 short-exposure shutter (DOL-specific)

These match Sony's IMX29x datasheet for the linear ↔ DOL transition.

Which sensors have WDR firmware in your Sofia binary?

Search the binary's strings for _init markers and LINE Init OK banners — sensors with both have multi-mode drivers:

$ strings /usr/bin/Sofia | grep -E '_init  |LINE Init OK' | sort -u
~~~~~~~~~~AR0237_init  JJ ~~~~~~~~~          ← linear-only
...
--IMX291 1080P 60fps LINE Init OK!           ← has WDR
--IMX290 1080P 30fps LINE Init OK!           ← has WDR
~~~~~~~~~~IMX323_init  gjj ~~~~~~~~~         ← linear-only

A sensor with only a ~~~_init~~~ debug marker and no LINE Init OK banner won't reconfigure regardless of wdrmode.dat — it has only the linear path. The mode write still gets persisted to flash, but nothing on the sensor side responds to it.

Segmenter heuristic

trace_segment.py detects mode switches by watching for the same stream-control register pair that the matched init_pattern used — e.g. 0x100=0 → 0x100=1 for SmartSens, 0x3000=01 → 0x3000=00 for Sony IMX, 0x12=0x40 → 0x12=0x00 for SOI/JX — appearing after init has completed (init_end). Each such cycle becomes a mode_switch_N phase, with N counting from 1. The post-init AE prime and runtime steady state anchor on the last such cycle, so a trace with no mode switch is identical to before.

This is the same heuristic used for Sofia's two-pass boot-time WDR init (linear baseline → WDR overrides). It works because Sofia's cmos_set_image_mode per-mode init functions all wrap their register table between a standby-assert/release pair on the family's stream- control register.

trace_to_driver.py emits one <sensor>_set_mode_N function per mode-switch phase, in the same shape as _linear_init.

If a sensor hot-swaps modes via a group-hold (e.g. 0x3812=0x00 ... 0x3812=0x30 block) without toggling its stream-control register, this heuristic misses the boundary — extend the segmenter when you hit such a sensor.

Stage 4 — Live-reading the AE state with ipctool sensor monitor

ipctool sensor monitor is a built-in subcommand (separate from trace) that reads a fixed list of AE/exposure registers from the running sensor over I2C/SPI in a loop, decoded as labelled fields. Same idea as _ae_step but read-side: instead of inferring AE registers from a captured trace, you can poll the actual sensor while it's running.

$ ipctool sensor monitor
EXP   100  AGAIN  310  DGAIN  80  VMAX  546  R3301  f  R3314  14  R3632  8  HOLD  0  R5781  60  R5785  30
EXP   100  AGAIN  310  DGAIN  80  VMAX  546  R3301  f  R3314  14  R3632  8  HOLD  0  R5781  60  R5785  30
EXP   ff   AGAIN  330  DGAIN  80  VMAX  546  R3301  f  R3314  14  R3632  8  HOLD  0  R5781  60  R5785  30
                                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^
                                                            same hot regs trace_to_driver picked up

Currently supported sensors and what they expose:

Sensor Registers monitored
SC2315E EXP, AGAIN, DGAIN, VMAX, plus tuning regs R3301/3314/3632/HOLD/R5781/R5785
IMX291 HCG_FRSEL (0x3009), GAIN (0x3014), VMAX (0x3018), HMAX (0x301C), SHS1 (0x3020), OPORTSEL (0x3046)
IMX385 SHS1, GAIN, HCG, SHS2, VMAX, RHS1, YOUT

The reg list per supported sensor lives in src/snstool.c (a small table, ~10 entries). For SC2315E that table mirrors what _ae_step emits — both are the registers the running AE loop writes per frame.

The IMX291 set is geared at DOL/WDR debugging in particular — HCG_FRSEL catches the case where AE flips the High Conversion Gain bit and clobbers FRSEL on a packed-register write, dropping the sensor out of WDR for one frame. SHS1 alone is the integration-time control on this part (Sony's multi-exposure WDR — there's no DOL on IMX291, despite the inherited IMX290 silicon; SHS2 / RHS1 read as 0 and aren't in the table).

Example output on a hi3516cv300 + IMX291 camera in WDR mode, sampled across a varying-light scene:

$ ipctool sensor monitor
HCG_FRSEL 2  GAIN 0   VMAX 550  HMAX 1130  SHS1 528  OPORTSEL e1
HCG_FRSEL 2  GAIN 0   VMAX 550  HMAX 1130  SHS1 528  OPORTSEL e1
HCG_FRSEL 2  GAIN 1c  VMAX 550  HMAX 1130  SHS1 4a3  OPORTSEL e1   # mid-light: SHS1 dropped, gain bumped
HCG_FRSEL 12 GAIN 50  VMAX afd  HMAX 1130  SHS1 73   OPORTSEL e1   # low-light: HCG enabled, AE slowed VMAX, gain high

HCG_FRSEL going from 2 (HCG=0, FRSEL=2) to 12 (HCG=1, FRSEL=2) is exactly the pattern AE uses to enable high-conversion-gain mode. If you ever see the low nibble of HCG_FRSEL go to 1 instead of 2, that's the bug fixed in widgetii/sony_imx291@b51850c — DOL FRSEL got clobbered.

Pairing sensor monitor with _ae_step

The trace-and-extract workflow gives you the register set of the AE loop. sensor monitor gives you value time-series under whatever lighting conditions you can produce. Pair them:

  1. Extract _ae_step via trace. Note the placeholder values (e.g., 0x5781=0x60, 0x3314=0x14 on SC2315E).
  2. Cover/uncover the lens, point at varying scenes, switch the IR cut etc. while running ipctool sensor monitor and recording the output (tee monitor.log).
  3. Plot or grep the resulting log for value transitions on the _ae_step registers.
  4. Derive the conditional / LUT that maps trace inputs (often gain index, integration time) to those values. This is the manual step that no register-trace tool can automate.

For SC2315E, the reference's cmos_gains_update writes 0x5781=1, 0x5785=2 once gain ≥ a fixed threshold; the rest of the time it writes the 0x60, 0x30 we captured. A varying-light monitor session that pushes the AE into high-gain regime will show that transition directly.

Adding a new sensor to monitor

Edit src/snstool.c:

  1. Add a Reg[] table for the sensor: name, address, byte length. Cover the AE-modified registers (exposure, gain, vmax) plus any tuning registers that change per-frame in cmos_gains_update.
  2. Add an entry to sns_regs[]: sensor ID (uppercase, matching ctx->sensor_id from src/sensors.c), pointer to the reg table, and .be = 1 if the sensor is big-endian on multi-byte registers.
  3. The dispatch in monitor() is automatic — no other glue needed.

When _ae_step emits a register set you didn't have in monitor, that's a good cue to extend the monitor table to match. A nice property of the workflow: the trace tells you which registers the running streamer cares about, the monitor lets you watch them change in real time.

Reading the diff

For SC2315E captured from Majestic without scoping, the unscoped diff shows ~5 value mismatches at registers 0x320E, 0x320F, 0x3E01, 0x3E02, 0x3E09 — these are VMAX and exposure registers. They appear in both init (defaults) and post_init (AE prime). The generator's "last value seen" is the AE prime value, while the reference's init function carries the default. This is expected, not a bug; scope the diff to the linear_init function on both sides and the mismatches disappear.

Triangulating against multiple references

When a sensor has more than one published source — e.g. an OpenIPC port and an older reverse-engineering effort — diffing against both lets you triangulate registers that appear in only one as either:

  • In trace + only in ref-A: ref-B is incomplete (the RE missed this register, or the port pruned it).
  • In trace + only in ref-B: ref-A drifted from the binary (vendor patched the closed driver, port didn't follow).
  • In both refs but not in trace: probably dead code in both refs, or behind a build flag the trace didn't exercise.
  • In trace and both refs: high confidence, ship it.

For SC2315E specifically, the four artifacts available — the trace from Majestic, the trace from Sofia, widgetii/smart_sc2315e (OpenIPC port from SC2231 SDK template), and widgetii/sc_sc2315e (older RE port from SC2235 SDK template) — agree byte-for-byte on init: 172 writes, 169 unique addresses, identical values, identical order, every pair-wise comparison at 100/100/100%.

# OpenIPC port
python3 tools/trace_diff.py generated.c \
    /tmp/smart_sc2315e/sc2315e_sensor_ctl.c \
    --gen-scope sc2315e_linear_init \
    --ref-scope sc2315e_linear_1080P30_init

# Reverse-engineered port (note: int return, sensor_write_register_0)
python3 tools/trace_diff.py generated.c \
    /tmp/sc_sc2315e/sc2235_sensor_ctl.c \
    --gen-scope sc2315e_linear_init \
    --ref-scope sc2235_init

trace_diff.py accepts both void <name>(...) and int <name>(...) function definitions, and its register-write regex matches both sensor_write_register(...) and sensor_write_register_0(...) (the bus-numbered suffix used by the older RE port).

Other expected sources of mismatch on a real capture:

  • Registers in the reference source but not in the trace: these are often gated by mode (HDR-only registers, sub-pipe configurations, OTP reads). The reference is the union over all branches; your trace exercises one. Cross-check with a second reference if you have one (sc_sc2315e does this for SC2315E).
  • Registers in the trace but not in the reference source: these are binary updates the source missed (vendor patched the closed driver but not the published one) or runtime tuning that the trace happened to capture in init slot. Inspect manually — these are the interesting ones.
  • Order differences in LCS but ≥99% address match: usually reorderings inside an unordered group of writes (BLC defaults, test-pattern setup). Not material for sensor function.

A 100 % triple match like the one in the worked example is the ceiling, not the expected outcome. 90+/85+/80+ is a healthy result that lets you trust the scaffold as a porting starting point.

Troubleshooting

sh: line 1: syntax error: unexpected word (expecting ")") on ipctool launch

The kernel returned ENOEXEC for your binary. Cause is almost always "binary built with toolchain whose ELF flags this kernel doesn't accept". Solution: UPX-pack the binary (see Stage 1). The CI toolchain alone is not sufficient; UPX is the actual fix.

Trace contains lines like sensor_write_register(0x5785xmcap_video_api.c

Streamer log lines and trace pseudocode were both written to the same fd. Use --output=PATH. If you can't (older ipctool build), strip ANSI escapes before parsing:

sed 's/\x1b\[[0-9;]*m//g' raw.log > clean.log

…and accept that some lines mid-write may be truncated.

Trace shows lots of error copy_from_process from 0 (I/O error)

The exit-side hook tried to read the result buffer for a syscall but the address was zero or unmapped. This usually means the syscall returned an error before populating the buffer (e.g. an I2C read at an unresponsive address during bus probe). Filter these out — they convey no information.

Trace is dominated by usleep(0)

Streamer is busy-polling. Pass --skip=usleep to silence them. You will lose phase boundaries that depend on actual sleep durations, but the segmenter falls back on the 0x100=0 / 0x100=1 reset/stream-on markers, which is enough for most sensors.

expect'd capture session ends with Connection closed by foreign host mid-flow

Either the camera rebooted under you (watchdog or RebootAbnormal trigger after a stream death), or your kill sequence reaped a process the supervisor depended on. For XiongMai cameras, do not kill XmServices_Mgr; it provides services the streamer needs and feeds the hardware watchdog. Use the bind-mount approach in the Sofia recipe.

Diff shows 100 % address match but only 60 % sequence match

You're capturing a different SDK build than the reference was generated from. Some vendors reorder writes between releases without changing behaviour. The address+value match is what matters; sequence is a helpful proxy that breaks down across versions.

Narrowing "fd N is open live but not in trace" with IPCTOOL_TRACE_DEBUG=1

Setting this in the tracee's environment makes ipctool trace log every open() and write() syscall to stderr (filename, fd, callback state). Off by default, zero overhead unless set. Useful when /proc/<streamer-pid>/fd shows /dev/i2c-N open but the trace contains no banner/writes for it - the dbg log will say whether syscall_open ever ran for that fd and what filename it resolved.

IPCTOOL_TRACE_DEBUG=1 ipctool trace --output=/tmp/trace.log \
    /usr/bin/majestic 2>/tmp/trace.dbg
grep -E "i2c|fd=26" /tmp/trace.dbg

File layout

ipctool/
├── src/
│   ├── ptrace.c              # ARM-only ptrace engine, --output flag added
│   └── hal/hisi/ptrace.c     # HiSilicon-specific MIPI/VI struct decoders
├── tools/
│   ├── capture_sensor.sh     # build / majestic / sofia subcommands
│   ├── trace_segment.py      # parse + phase split
│   ├── trace_to_driver.py    # JSON → C scaffold
│   └── trace_diff.py         # generated vs reference, scoped
└── docs/
    └── sensor-driver-extraction.md  # this file