Skip to content

Commit b6de509

Browse files
claudehyperpolymath
authored andcommitted
fix(nif): fail loudly instead of faking success (silent data-loss tripwire)
The verisim-nif Rustler bridge is a scaffold with no store wiring, but every NIF returned canned success JSON: delete_octad -> {"status": "deleted"}, create_octad -> {"status":"created"}, get_drift_score -> all-zeros. When the .so was compiled in, :erlang.load_nif overrode the honest Elixir fallback stubs, NifBridge.loaded? reported true, and VERISIM_TRANSPORT=nif|auto selected the NIF — so a write returned success having stored/deleted nothing: silent data loss reported as success. (Without the .so the pure-Elixir stubs already errored, so the danger was the compiled artifact only.) - rust-core/verisim-nif: every NIF now returns {:error, :not_implemented} via rustler atoms — fail loudly, never fabricate. The compiled NIF and the absent-library stub are now indistinguishable (both non-operational, both error). Honest module doc; dead-code runtime scaffold retained for the eventual real wiring. - nif_bridge.ex: loaded? redefined to mean OPERATIONAL (probe returns a real binary), so a loaded-but-unimplemented NIF reads as unavailable. - transport.ex: health/0 no longer returns a hardcoded {:ok, healthy} under nif; reports {:error, :nif_not_operational} with a warning. auto now correctly falls back to HTTP; explicit nif surfaces the error. - test/verisim/transport_test.exs: regression guard — asserts no NIF operation ever returns fake success, auto won't select a non-operational NIF, and health doesn't fake under nif. - KNOWN-ISSUES #28: documents the fix and frames it as an instance of the recurring silent-fake-success defect class (cf. #26 drift, #27 scores) that the CI scanners structurally cannot catch. Rust: workspace clippy clean under -D warnings; verisim-nif builds. Elixir: parse + mix format clean (full suite runs in CI; repo.hex.pm is blocked in this environment so mix deps.get cannot run locally). https://claude.ai/code/session_01E8BpV19yxhf67UrCrvkfTr
1 parent 66252f8 commit b6de509

5 files changed

Lines changed: 181 additions & 143 deletions

File tree

KNOWN-ISSUES.adoc

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,3 +275,15 @@ Added `rescript.json` for build configuration.
275275
**Resolved:** 2026-06-13. The text, vector, and similar-query search endpoints all reported `score: 1.0 - (i as f32 * 0.1)` — a synthetic sequence derived from result position, discarding the real relevance scores the modality stores already compute (Tantivy BM25 for documents, cosine similarity for vectors). The octad store now exposes `search_text_scored` / `search_similar_scored` (returning `Vec<(Octad, f32)>`); the three handlers surface those real scores. Test `test_vector_search_returns_real_cosine_scores` pins the distinction (the second hit reports its true ~0.994 cosine, not synthetic 0.9).
276276

277277
**Original issue:** API search responses carried fabricated scores, so any client ranking or thresholding on `score` was meaningless.
278+
279+
=== 28. NIF Transport Faked Success (Silent Data Loss) — ✅ RESOLVED
280+
281+
**Location:** `rust-core/verisim-nif/src/lib.rs`, `elixir-orchestration/lib/verisim/{nif_bridge,transport}.ex`
282+
283+
**Resolved:** 2026-06-13. The `verisim-nif` Rustler bridge is a scaffold — no operation is wired to the store — but every NIF returned *canned success* JSON: `delete_octad` replied `{"status":"deleted"}`, `create_octad` `{"status":"created"}`, `get_drift_score` all-zeros. When the `.so` was compiled in, `:erlang.load_nif` overrode the honest Elixir fallback stubs with these, `NifBridge.loaded?` reported `true`, and `VERISIM_TRANSPORT=nif` *or* `auto` selected the NIF — so a write returned success having stored/deleted nothing: **silent data loss reported as success.** (Without the `.so`, the pure-Elixir stubs already returned `{:error, :nif_not_loaded}`, so the danger was the compiled artifact only.)
284+
285+
Now every NIF returns `{:error, :not_implemented}` — fail loudly, never fake. The compiled NIF and the absent-library stub are now indistinguishable to callers (both non-operational, both error). `NifBridge.loaded?` is redefined to mean *operational* (a probe returns a real result), so `auto` correctly falls back to HTTP; explicit `nif` surfaces the error rather than fabricating data; and `Transport.health/0` no longer returns a hardcoded `healthy` under nif. Regression test `VeriSim.TransportTest` asserts no operation ever returns fake success.
286+
287+
This was an instance of the project's recurring *silent fake-success* defect class (see also issues 26, 27): code returning a success value without performing the named effect. None of the CI scanners (Hypatia, panic-attack, gitleaks) flag it — it has no dangerous *shape* — so it is caught only by reading code against its claims.
288+
289+
**Original issue:** The NIF bridge fabricated success for unimplemented operations, risking silent data loss under the NIF/auto transports.

elixir-orchestration/lib/verisim/nif_bridge.ex

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,11 @@ defmodule VeriSim.NifBridge do
1313
The NIF shared library is loaded from `priv/native/libverisim_nif.so` (Linux)
1414
or `priv/native/libverisim_nif.dylib` (macOS). If the library is not present
1515
(e.g., in a pure-Elixir development setup), all functions return
16-
`{:error, :nif_not_loaded}`.
16+
`{:error, :nif_not_loaded}`. If the library IS present but an operation is
17+
not yet wired to the store, that operation returns `{:error, :not_implemented}`
18+
— it never fabricates success. Today every operation is a not-implemented
19+
stub (see `rust-core/verisim-nif`), so the NIF is non-operational and `auto`
20+
transport always falls back to HTTP.
1721
1822
## Transport Selection
1923
@@ -107,12 +111,18 @@ defmodule VeriSim.NifBridge do
107111
def trigger_normalise(_octad_id), do: {:error, :nif_not_loaded}
108112

109113
@doc """
110-
Check whether the NIF bridge is loaded and operational.
114+
Check whether the NIF bridge is loaded **and operational**.
115+
116+
Returns `true` only when a probe operation returns a real result. Both the
117+
absent-library fallback (`{:error, :nif_not_loaded}`) and a loaded-but-
118+
unimplemented NIF (`{:error, :not_implemented}`) read as *not operational*,
119+
so `auto` transport never selects a NIF that cannot actually serve a
120+
request. When every operation is a not-implemented stub, this is `false`.
111121
"""
112122
def loaded? do
113123
case get_octad("__health_check__") do
114-
{:error, :nif_not_loaded} -> false
115-
_ -> true
124+
result when is_binary(result) -> true
125+
_ -> false
116126
end
117127
end
118128
end

elixir-orchestration/lib/verisim/transport.ex

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,10 +71,21 @@ defmodule VeriSim.Transport do
7171

7272
@doc """
7373
Check health of the Rust core.
74+
75+
Under `:http` (and `:auto`, which falls back to HTTP while the NIF is
76+
non-operational) this probes the real verisim-api. Under an explicit
77+
`:nif` it reports honestly that the NIF transport is not operational
78+
rather than fabricating a `healthy` status — the NIF has no implemented
79+
operations to be healthy about.
7480
"""
7581
def health do
7682
if use_nif?() do
77-
{:ok, %{"status" => "healthy", "transport" => "nif"}}
83+
Logger.warning(
84+
"VERISIM_TRANSPORT=nif selected but the NIF bridge has no implemented " <>
85+
"operations; reporting not operational. Use VERISIM_TRANSPORT=http."
86+
)
87+
88+
{:error, :nif_not_operational}
7889
else
7990
RustClient.health()
8091
end
Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
3+
defmodule VeriSim.TransportTest do
4+
# async: false — these tests mutate the VERISIM_TRANSPORT OS env var.
5+
use ExUnit.Case, async: false
6+
7+
alias VeriSim.{NifBridge, Transport}
8+
9+
setup do
10+
prior = System.get_env("VERISIM_TRANSPORT")
11+
12+
on_exit(fn ->
13+
if prior,
14+
do: System.put_env("VERISIM_TRANSPORT", prior),
15+
else: System.delete_env("VERISIM_TRANSPORT")
16+
end)
17+
18+
:ok
19+
end
20+
21+
describe "transport mode selection" do
22+
test "defaults to :http when unset" do
23+
System.delete_env("VERISIM_TRANSPORT")
24+
assert Transport.mode() == :http
25+
refute Transport.use_nif?()
26+
end
27+
28+
test "explicit nif selects :nif" do
29+
System.put_env("VERISIM_TRANSPORT", "nif")
30+
assert Transport.mode() == :nif
31+
end
32+
33+
test "auto does NOT select the NIF while it is non-operational" do
34+
System.put_env("VERISIM_TRANSPORT", "auto")
35+
assert Transport.mode() == :auto
36+
# No operation is implemented (every NIF/stub returns an error), so the
37+
# bridge is non-operational and auto must fall back to HTTP. This is the
38+
# guard against silently routing writes into a NIF that does nothing.
39+
refute Transport.use_nif?()
40+
end
41+
42+
test "unknown value falls back to :http" do
43+
System.put_env("VERISIM_TRANSPORT", "banana")
44+
assert Transport.mode() == :http
45+
end
46+
end
47+
48+
describe "the NIF bridge never fabricates success (regression for silent data loss)" do
49+
test "every operation returns an error tuple, never fake success" do
50+
# The previous Rust NIF returned canned success (e.g. delete -> "deleted")
51+
# having done nothing. After the fail-loudly fix, the compiled NIF and the
52+
# pure-Elixir stub are indistinguishable: both error, neither fabricates.
53+
assert {:error, _} = NifBridge.create_octad(~s({"document":{"title":"t","body":"b"}}))
54+
assert {:error, _} = NifBridge.get_octad("e-1")
55+
assert {:error, _} = NifBridge.delete_octad("e-1")
56+
assert {:error, _} = NifBridge.search_text("q", 10)
57+
assert {:error, _} = NifBridge.search_vector("[0.1,0.2,0.3]", 5)
58+
assert {:error, _} = NifBridge.list_octads(10, 0)
59+
assert {:error, _} = NifBridge.get_drift_score("e-1")
60+
assert {:error, _} = NifBridge.trigger_normalise("e-1")
61+
end
62+
63+
test "loaded? is false when no operation is operational" do
64+
refute NifBridge.loaded?()
65+
end
66+
end
67+
68+
describe "health does not fake success under an explicit nif transport" do
69+
test "nif health reports not operational, never {:ok, healthy}" do
70+
System.put_env("VERISIM_TRANSPORT", "nif")
71+
assert {:error, :nif_not_operational} = Transport.health()
72+
end
73+
end
74+
end

0 commit comments

Comments
 (0)