Reject extra row values, formalize inline comments, harden quoting; isonantic-ts parse(); fix all Dependabot alerts

Parser strictness (all implementations: py, js, ts, go, rust, cpp, n8n
inline parser), continuing the integrity work:
- Rows (ISON and ISONL) with more values than declared fields now raise
  a syntax error with the line number instead of silently truncating;
  missing values still pad null
- Inline comments formalized: an unquoted token starting with '#' in a
  data row or ISONL values section begins a comment; quoted tokens are
  always data. Previously inline comments only "worked" as a side
  effect of the silent-truncation bug
- Regular ISON serializer now quotes strings containing \r or
  backslashes, starting with '#' (a leading-'#' value used to turn its
  row into a comment on re-parse), shaped like a kind.name block header
  (a single-field value like 'a.true' used to be re-parsed as a new
  block header), and empty strings (ts/rust dropped the column)
- ISONL serializers quote leading-'#' values
- Additional parity fixes surfaced by the shared property test: quoted
  tokens stay strings in the regular go/ts/rust/n8n parsers; ts/rust
  header detection tightened to exact kind.name identifiers; go pads
  missing fields explicitly; n8n keeps empty quoted values
- New tests in every implementation: extra-value rejection, inline
  comment semantics, and a seeded 300-trial regular-format round-trip
  property test (twin of the ISONL one)

isonantic-ts 1.1.0:
- New parse()/parseSafe() that delegate all text parsing to ison-ts
  (docs advertised parse() but it never existed); references mapped,
  rows validated through the schema with per-field errors
- VERSION constant aligned with package.json (was drifted at 1.0.0)

Dependency security - all 39 Dependabot alerts resolved (0 remaining):
- ison-vscode: npm audit fix + @vscode/vsce 2.x -> 3.9.2
- ison-ts / isonantic-ts: npm audit fix + vitest 1.x -> 3.2.6
- n8n-nodes-ison: npm audit fix + n8n-workflow 1.x -> 2.16 (dev) +
  gulp 4 -> 5; removed vestigial ison-parser runtime dependency (the
  node uses its inline parser)

n8n-nodes-ison packaging fix: tsconfig rootDir corrected so the
compiled node lands at dist/nodes/Ison/Ison.node.js where the n8n
manifest points (it previously emitted to dist/Ison/ and the published
package would not load)

All test suites pass: ison-py (full), isonantic 45, ison-cli 27,
ison-js 39+47, ison-ts 35, ison-go, ison-rust 15, ison-cpp 36 (MSVC),
isonantic-ts 58, n8n harness 20/20 + 14/14.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: maheshvaikri-code

ison-cpp/CHANGELOG.md

@@ -6,9 +6,13 @@
 - **ISONL Round-Trip Corruption**: Fixed quote-tracking desync in the ISONL section splitter — a quoted value ending in an escaped backslash (e.g. `"x \\"`) let a later `|` split the line in the wrong place. The splitter now consumes escape pairs instead of using look-behind.
 - **ISONL Serialization of `\r` and `\\`**: The ISONL serializer now quotes strings containing carriage returns or backslashes (and bare numeric strings), so they survive a round-trip instead of being emitted raw.
 - **Explicit `\|` Unescape**: The tokenizer now maps the `\|` escape to `|` explicitly.
+- **Extra Values Now Error**: A data row (regular ISON or ISONL) with more values than fields now throws `ISONSyntaxError` (`"Row has N values but only M fields (extra value: ...)"`) instead of silently truncating. Missing trailing values still pad with `null`.
+- **Regular Serializer Quoting**: `Serializer` now quotes strings containing `\r` or `\\`, strings starting with `#` (which would be re-parsed as a comment), and strings shaped like a `kind.name` block header (which, alone on a row line, would be re-parsed as the start of a new block), so they survive a round-trip.
+- **ISONL Serializer Quoting**: The ISONL serializer now also quotes strings starting with `#`, which would otherwise be re-parsed as an inline comment in the values section.
 
 ### Added
 - **ISONL Envelope Validation**: `dumps_isonl` now throws `ISONError` for block kind/name or field names that cannot be serialized (empty, or containing pipe, quote, backslash, or whitespace; kind additionally must not contain `.` or start with `#`) instead of silently emitting corrupt lines. Dots in the block name remain legal — the parser splits `kind.name` on the first dot.
+- **Inline Comments Formalized**: In data rows (regular ISON) and ISONL values sections, an unquoted token starting with `#` begins an inline comment — it and everything after it on the line are ignored. Quoted tokens are always data, never comments.
 
 ## [1.0.1] - 2025-12-29
 

ison-cpp/include/ison_parser.hpp

@@ -579,6 +579,50 @@ class Parser {
         return doc;
     }
 
+    /**
+     * Return the number of leading tokens that are data: an unquoted token
+     * starting with '#' begins an inline comment, ignoring it and everything
+     * after it. Quoted tokens are always data. (Shared with ISONLParser.)
+     */
+    static size_t strip_inline_comment(const std::vector<std::string>& raw_tokens,
+                                       const std::vector<bool>& quoted_flags) {
+        for (size_t i = 0; i < raw_tokens.size(); ++i) {
+            if (!quoted_flags[i] && !raw_tokens[i].empty() && raw_tokens[i][0] == '#') {
+                return i;
+            }
+        }
+        return raw_tokens.size();
+    }
+
+    /**
+     * Reject rows with more values than fields instead of silently
+     * truncating them. (Shared with ISONLParser.)
+     */
+    static void check_extra_tokens(const std::vector<std::string>& raw_tokens,
+                                   size_t field_count, int line_num) {
+        if (raw_tokens.size() <= field_count) return;
+        throw ISONSyntaxError(
+            "Row has " + std::to_string(raw_tokens.size()) + " values but only " +
+            std::to_string(field_count) + " fields (extra value: '" +
+            raw_tokens[field_count] + "')",
+            line_num, 0);
+    }
+
+    /**
+     * Check whether a bare line scans as a 'kind.name' block header.
+     * (Also used by Serializer to quote strings that would be misread
+     * as a header when they land alone on a row line.)
+     */
+    static bool looks_like_header(const std::string& line) {
+        size_t dot_pos = line.find('.');
+        if (dot_pos == std::string::npos) return false;
+        if (line.find(' ') != std::string::npos) return false;
+
+        std::string kind = line.substr(0, dot_pos);
+        std::string name = line.substr(dot_pos + 1);
+        return is_valid_id(kind) && is_valid_id(name);
+    }
+
 private:
     std::vector<std::string> lines_;
     size_t line_num_;
@@ -666,16 +710,6 @@ class Parser {
         return block;
     }
 
-    bool looks_like_header(const std::string& line) const {
-        size_t dot_pos = line.find('.');
-        if (dot_pos == std::string::npos) return false;
-        if (line.find(' ') != std::string::npos) return false;
-
-        std::string kind = line.substr(0, dot_pos);
-        std::string name = line.substr(dot_pos + 1);
-        return is_valid_id(kind) && is_valid_id(name);
-    }
-
     static bool is_valid_id(const std::string& s) {
         if (s.empty()) return false;
         if (!std::isalpha(static_cast<unsigned char>(s[0])) && s[0] != '_') return false;
@@ -691,6 +725,7 @@ class Parser {
         std::vector<std::string> raw_tokens = tokenizer.tokenize();
 
         std::vector<Value> values;
+        std::vector<bool> quoted_flags;
         size_t pos = 0;
 
         for (size_t i = 0; i < raw_tokens.size(); ++i) {
@@ -699,6 +734,7 @@ class Parser {
             }
             bool was_quoted = pos < line.size() && line[pos] == '"';
             values.push_back(TypeInferrer::infer(raw_tokens[i], was_quoted));
+            quoted_flags.push_back(was_quoted);
 
             if (was_quoted) {
                 ++pos;
@@ -712,6 +748,11 @@ class Parser {
             }
         }
 
+        size_t keep = strip_inline_comment(raw_tokens, quoted_flags);
+        raw_tokens.resize(keep);
+        values.resize(keep);
+        check_extra_tokens(raw_tokens, fields.size(), static_cast<int>(line_num_ + 1));
+
         Row row;
         for (size_t i = 0; i < fields.size(); ++i) {
             if (i < values.size()) {
@@ -849,12 +890,17 @@ class Serializer {
     static std::string quote_if_needed(const std::string& s) {
         if (s.empty()) return "\"\"";
 
-        bool needs_quote = (s == "true" || s == "false" || s == "null" || s[0] == ':');
+        // '\r' and '\\' would be emitted raw and corrupt on re-parse; a
+        // leading '#' would turn the line into a comment (or an inline
+        // comment) and silently lose data.
+        bool needs_quote = (s == "true" || s == "false" || s == "null" ||
+                            s[0] == ':' || s[0] == '#');
 
         if (!needs_quote) {
             for (size_t i = 0; i < s.size(); ++i) {
                 char c = s[i];
-                if (c == ' ' || c == '\t' || c == '"' || c == '\n' || c == '\r') {
+                if (c == ' ' || c == '\t' || c == '"' || c == '\n' ||
+                    c == '\r' || c == '\\') {
                     needs_quote = true;
                     break;
                 }
@@ -865,6 +911,12 @@ class Serializer {
             needs_quote = true;
         }
 
+        // A bare 'kind.name'-shaped value alone on a row line would be
+        // misread as the next block header on re-parse
+        if (!needs_quote && Parser::looks_like_header(s)) {
+            needs_quote = true;
+        }
+
         if (needs_quote) {
             std::string escaped;
             for (size_t i = 0; i < s.size(); ++i) {
@@ -993,13 +1045,16 @@ class ISONLParser {
         Tokenizer value_tokenizer(sections[2], line_num);
         std::vector<std::string> raw_values = value_tokenizer.tokenize();
 
+        std::vector<Value> typed_values;
+        std::vector<bool> quoted_flags;
         size_t pos = 0;
-        for (size_t i = 0; i < record.fields.size() && i < raw_values.size(); ++i) {
+        for (size_t i = 0; i < raw_values.size(); ++i) {
             while (pos < sections[2].size() && (sections[2][pos] == ' ' || sections[2][pos] == '\t')) {
                 ++pos;
             }
             bool was_quoted = pos < sections[2].size() && sections[2][pos] == '"';
-            record.values[record.fields[i]] = TypeInferrer::infer(raw_values[i], was_quoted);
+            typed_values.push_back(TypeInferrer::infer(raw_values[i], was_quoted));
+            quoted_flags.push_back(was_quoted);
 
             if (was_quoted) {
                 ++pos;
@@ -1013,6 +1068,17 @@ class ISONLParser {
             }
         }
 
+        size_t keep = Parser::strip_inline_comment(raw_values, quoted_flags);
+        raw_values.resize(keep);
+        typed_values.resize(keep);
+        Parser::check_extra_tokens(raw_values, record.fields.size(), line_num);
+
+        for (size_t i = 0; i < record.fields.size(); ++i) {
+            record.values[record.fields[i]] = (i < typed_values.size())
+                ? typed_values[i]
+                : Value(nullptr);
+        }
+
         return record;
     }
 
@@ -1206,8 +1272,11 @@ class ISONLSerializer {
     static std::string quote_if_needed(const std::string& s) {
         if (s.empty()) return "\"\"";
 
+        // A leading '#' would start an inline comment in the values section
+        // and silently drop the rest of the row on re-parse
         bool needs_quote = (s == "true" || s == "false" || s == "null" ||
-                           s[0] == ':' || s.find(' ') != std::string::npos ||
+                           s[0] == ':' || s[0] == '#' ||
+                           s.find(' ') != std::string::npos ||
                            s.find('\t') != std::string::npos ||
                            s.find('"') != std::string::npos ||
                            s.find('\n') != std::string::npos ||

ison-cpp/tests/test_ison_parser.cpp

@@ -589,6 +589,158 @@ TEST(isonl_envelope_validation) {
     ASSERT_EQ(parsed.blocks[0].name, "v1.2");
 }
 
+// =============================================================================
+// Row Integrity Tests
+// =============================================================================
+
+TEST(extra_values_rejected) {
+    // Regression: rows with more values than fields must error, not truncate
+    bool threw = false;
+    try {
+        parse("table.t\na b\n1 2 3");
+    } catch (const ISONSyntaxError& e) {
+        threw = true;
+        ASSERT(std::string(e.what()).find("3 values") != std::string::npos);
+    }
+    ASSERT(threw);
+
+    // A quoted token is data, never a comment
+    threw = false;
+    try {
+        parse("table.t\na b\n1 2 \"#not-a-comment\"");
+    } catch (const ISONSyntaxError&) {
+        threw = true;
+    }
+    ASSERT(threw);
+
+    // ISONL
+    threw = false;
+    try {
+        loads_isonl("table.t|a b|1 2 3");
+    } catch (const ISONSyntaxError& e) {
+        threw = true;
+        ASSERT(std::string(e.what()).find("3 values") != std::string::npos);
+    }
+    ASSERT(threw);
+}
+
+TEST(inline_trailing_comment) {
+    // An unquoted token starting with '#' begins an inline comment
+    auto doc = parse("table.t\na b\n1 2 # note ignored");
+    ASSERT_EQ(as_int(doc["t"][0].at("a")), 1);
+    ASSERT_EQ(as_int(doc["t"][0].at("b")), 2);
+
+    auto doc2 = loads_isonl("table.t|a b|1 2 # note ignored");
+    ASSERT_EQ(as_int(doc2["t"][0].at("a")), 1);
+    ASSERT_EQ(as_int(doc2["t"][0].at("b")), 2);
+
+    // Comment mid-row: remaining fields are missing (null), not data
+    auto doc3 = parse("table.t\na b\n1 #tag");
+    ASSERT_EQ(as_int(doc3["t"][0].at("a")), 1);
+    ASSERT(is_null(doc3["t"][0].at("b")));
+
+    // Quoted tokens are always data, never comments
+    auto doc4 = parse("table.t\na b\n1 \"#tag\"");
+    ASSERT_EQ(as_int(doc4["t"][0].at("a")), 1);
+    ASSERT_EQ(as_string(doc4["t"][0].at("b")), "#tag");
+
+    // Serializer quotes leading-'#' strings so they round-trip as data
+    Document out_doc;
+    Block block("table", "t");
+    block.fields = {"a"};
+    Row row;
+    row["a"] = std::string("#tag");
+    block.rows.push_back(row);
+    out_doc.blocks.push_back(block);
+    auto roundtrip = parse(dumps(out_doc));
+    ASSERT_EQ(as_string(roundtrip["t"][0].at("a")), "#tag");
+}
+
+TEST(ison_roundtrip_property) {
+    // Explicit regression: header-shaped string values ('kind.name') must be
+    // quoted by the serializer, or a single-field row line like 'a.true'
+    // would be re-parsed as a NEW block header and split the block
+    {
+        Document doc;
+        Block block("table", "t");
+        block.fields = {"v"};
+        Row r1;
+        r1["v"] = std::string("a.true");
+        Row r2;
+        r2["v"] = std::string("object.config");
+        block.rows.push_back(r1);
+        block.rows.push_back(r2);
+        doc.blocks.push_back(block);
+
+        auto parsed = parse(dumps(doc));
+        ASSERT_EQ(parsed.size(), 1);
+        ASSERT_EQ(parsed.blocks[0].rows.size(), 2);
+        ASSERT_EQ(as_string(parsed.blocks[0].rows[0].at("v")), "a.true");
+        ASSERT_EQ(as_string(parsed.blocks[0].rows[1].at("v")), "object.config");
+    }
+
+    // Regular-format twin of isonl_roundtrip_property: random strings over a
+    // hostile alphabet must round-trip through dumps/parse.
+    // Deterministic LCG so failures are reproducible without <random>.
+    const char* alphabet[] = {
+        "a", "b", " ", "|", "\"", "\\", "\n", "\r", "\t",
+        ".", ":", "#", "0", "1", "true", "null"
+    };
+    const int alphabet_size = static_cast<int>(sizeof(alphabet) / sizeof(alphabet[0]));
+
+    uint64_t state = 20260713ULL;
+    auto next_int = [&state](int lo, int hi) -> int {
+        state = (state * 1103515245ULL + 12345ULL) % 2147483648ULL;
+        return lo + static_cast<int>(state % static_cast<uint64_t>(hi - lo + 1));
+    };
+
+    for (int trial = 0; trial < 300; ++trial) {
+        int num_fields = next_int(1, 4);
+        std::vector<std::string> fields;
+        for (int i = 0; i < num_fields; ++i) {
+            fields.push_back("f" + std::to_string(i));
+        }
+
+        int num_rows = next_int(1, 3);
+        std::vector<Row> rows;
+        for (int r = 0; r < num_rows; ++r) {
+            Row row;
+            for (int f = 0; f < num_fields; ++f) {
+                int len = next_int(0, 12);
+                std::string value;
+                for (int k = 0; k < len; ++k) {
+                    value += alphabet[next_int(0, alphabet_size - 1)];
+                }
+                row[fields[f]] = value;
+            }
+            rows.push_back(row);
+        }
+
+        Document doc;
+        Block block("table", "t");
+        block.fields = fields;
+        block.rows = rows;
+        doc.blocks.push_back(block);
+
+        std::string out = dumps(doc);
+        auto parsed = parse(out);
+
+        ASSERT_EQ(parsed.size(), 1);
+        ASSERT_EQ(parsed.blocks[0].rows.size(), rows.size());
+        for (size_t r = 0; r < rows.size(); ++r) {
+            for (size_t f = 0; f < fields.size(); ++f) {
+                const Value& got = parsed.blocks[0].rows[r].at(fields[f]);
+                const std::string& expected = as_string(rows[r].at(fields[f]));
+                if (!is_string(got) || as_string(got) != expected) {
+                    throw std::runtime_error(
+                        "trial " + std::to_string(trial) + " row " + std::to_string(r) +
+                        " field " + fields[f] + ": round-trip mismatch, doc: " + out);
+                }
+            }
+        }
+    }
+}
+
 // =============================================================================
 // JSON Conversion Tests
 // =============================================================================
@@ -758,6 +910,11 @@ int main() {
     RUN_TEST(isonl_roundtrip_property);
     RUN_TEST(isonl_envelope_validation);
 
+    // Row integrity
+    RUN_TEST(extra_values_rejected);
+    RUN_TEST(inline_trailing_comment);
+    RUN_TEST(ison_roundtrip_property);
+
     // JSON
     RUN_TEST(to_json);