step:complete crashes with Postgrex 22P05 when output_dataclip body contains a NUL byte

[midigofrank]

Describe the bug

When a worker reports step:complete with an output_dataclip whose JSON contains a NUL byte (\u0000), the dataclip insert crashes with an unhandled Postgrex.Error and the step never completes:

Postgrex.Error: ERROR 22P05 (untranslatable_character)
unsupported Unicode escape sequence
\u0000 cannot be converted to text.

Sentry: LIGHTNING-13C (48 events, first seen 2025-08-07, still occurring as of 2026-06-24; prod, app.openfn.org).

Version number v2.16.6 (1.5.86) (from the Sentry event); reproduced on current main.

I have reproduced this locally on main:

• Yes
• No

Root cause

The crash is in Lightning.Runs.Handlers.CompleteStep, inserting the Dataclip body (lib/lightning/runs/handlers.ex:467-482):

Dataclip.new(%{
  id: dataclip_id,
  project_id: project_id,
  body: output_dataclip |> Jason.decode!() |> ensure_map(),  # <- NUL byte ends up here
  type: :step_result
})
|> Repo.insert()

Stacktrace (confirmed via Sentry):

CompleteStep.call/2:398
  -> update_step/2:410          (maybe_save_dataclip)
    -> Ecto.Repo.Schema.do_insert/4
      -> Ecto.Adapters.SQL.raise_sql_call_error/1

The worker sends literal JSON text containing the 6-character escape \u0000 (valid JSON — this is not a JSON-validity problem). Jason.decode! turns that escape into a real NUL byte in the Elixir binary. PostgreSQL jsonb/text physically cannot store a NUL byte (0x00), so the insert raises. Because the error is unhandled, the whole step:complete reply fails and the step/run is left without a clean completion.

To Reproduce

A step:complete payload of the form (note output_dataclip carries the escape):

{
  "step_id": "...",
  "run_id": "...",
  "project_id": "...",
  "reason": "success",
  "output_dataclip_id": "...",
  "output_dataclip": "{\"value\": \"foo\\u0000bar\"}"
}

A reproduction test (currently asserts the crash):

test "reproduces LIGHTNING-13C: output_dataclip with a NUL byte crashes the insert" do
  # ... set up workflow/run/step ...
  output_dataclip = ~s({"value": "foo\\u0000bar"})
  assert Jason.decode!(output_dataclip) == %{"value" => "foo\0bar"}

  assert_raise Postgrex.Error, ~r/unsupported Unicode escape sequence/, fn ->
    Runs.complete_step(%{
      step_id: step.id, reason: "success",
      output_dataclip: output_dataclip,
      output_dataclip_id: Ecto.UUID.generate(),
      run_id: run.id, project_id: workflow.project_id
    })
  end
end

This passes — confirming the exact error class, code path, and message match Sentry.

Expected behavior

The step/run should complete successfully without crashing. NUL bytes (and other PostgreSQL-incompatible control chars) in the dataclip body should be handled gracefully rather than raising an unhandled error.

Proposed handling

There's already a precedent: Lightning.LogMessage (lib/lightning/ecto_types.ex:63) sanitizes exactly this for the run:log path — stripping NUL bytes, control chars, and literal \u0000 escapes, replacing them with the Unicode replacement character. The dataclip body never got the same protection.

Options:

• A — targeted: sanitize the raw output_dataclip string before Jason.decode! in maybe_save_dataclip, reusing the LogMessage regex logic.
• B — boundary (recommended): a custom jsonb Ecto type (the map analogue of LogMessage) that recursively sanitizes string keys/values, applied to Dataclip.body. This protects all dataclip write paths (webhook-received dataclips, run input dataclips, step results), not just CompleteStep, which can hit the same NUL crash elsewhere.

Open product question: sanitize-and-keep (replace NUL, run completes — consistent with logs) vs reject (return an error reply; avoids silently mutating output but loses the whole output and risks stuck runs). Recommendation: sanitize, matching existing log behaviour.

Additional context

• error_message is commented out of the Step schema (step.ex:55) and not persisted, so it can't trigger this; the dataclip body (jsonb) is the only crashing path observed. error_type/exit_reason are short :string codes — same class, low risk.
• Retained Sentry events cluster suspiciously: 4 within ~12 min on 2026-06-23 and 5 within ~17 min on 2026-04-15, suggesting individual workflows repeatedly emitting NUL-containing output.

[josephjclark]

Related (and probbaly affecting kit) when this sort of thing happens the run should exist with state EXCEPTION. It should not be lost.

The worker gets a timeout on this event - it's not replied to. So actually we might just need logic in the event handler to reject the event? Then the worker can add logic to exit with the correct reason

[midigofrank]

@josephjclark yeah, I think rejecting / returning an error might be the safe approach here. Messing with input/output data sounds like a terrible idea

[midigofrank]

@josephjclark it seems the worker doesn't respect any error response from lightning on step:complete

[josephjclark]

Thanks @midigofrank - I've raised OpenFn/kit#1478 to investigate