Skip to content

42 amend calls in api and mcp and cli#77

Merged
eviltester merged 5 commits into
masterfrom
42-amend-calls-in-api-and-mcp-and-cli
May 9, 2026
Merged

42 amend calls in api and mcp and cli#77
eviltester merged 5 commits into
masterfrom
42-amend-calls-in-api-and-mcp-and-cli

Conversation

@eviltester

@eviltester eviltester commented May 9, 2026

Copy link
Copy Markdown
Owner

closes #42

Summary by CodeRabbit

  • New Features

    • Added an "amend" workflow: CLI subcommand, REST endpoint, and MCP tool to apply a schema to existing data; returns the full amended dataset (stream flags accepted but ignored).
  • Documentation

    • CLI, REST API, and MCP docs updated with amend usage, parameters, and examples.
  • Tests

    • Extensive tests for amend across CLI, API, MCP, core, and cross-format fixtures; web sorting assertions strengthened.
  • Behavior

    • Response cell types broadened beyond strings; CSV parsing avoids synthetic trailing rows (empty-line handling tightened).

Copilot AI review requested due to automatic review settings May 9, 2026 22:19
@eviltester eviltester linked an issue May 9, 2026 that may be closed by this pull request
@qa-tech-integration

Copy link
Copy Markdown

👀 QA.tech will run a exploratory tests to review this PR as soon as a deployment is available for this PR. Alternatively, you can comment @qa.tech to manually trigger a review.

Learn more about configuring preview deployments.

What happens next

  1. ⏳ Waiting for your preview deployment or a comment mentioning @qa.tech
  2. 🔍 Running full end-to-end tests to validate your changes
  3. 📝 Delivering detailed results and actionable feedback

🤖 AI end-to-end testing powered by QA.tech
For organization eviltester project Web app (iK6q). You can customize integration settings for this project.
💡 Comment @qa.tech to re-run testing. Add instructions for a focused review, e.g. @qa.tech test the checkout flow.

@coderabbitai

coderabbitai Bot commented May 9, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fd3a3e75-54fe-4826-a154-3b45c12c23f4

📥 Commits

Reviewing files that changed from the base of the PR and between 1e62286 and f345bad.

📒 Files selected for processing (3)
  • packages/core/js/data_formats/delimiter-options.js
  • packages/core/src/index.js
  • packages/core/src/tests/core-api/amendFromTextSpecAndData.test.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/core/src/index.js

📝 Walkthrough

Walkthrough

Adds an "amend" capability: new core export amendFromTextSpecAndData, POST /v1/generate/amend endpoint and OpenAPI, CLI amend command, MCP amend_data_from_spec tool, tests (unit, integration, Playwright), parser option tweak, fixtures, and docs updates.

Changes

Amend Feature Implementation

Layer / File(s) Summary
Core Amend API
packages/core/src/index.js
Adds amendFromTextSpecAndData with PapaParse exposure, importer factory, rowCount normalization, header merging, row padding, amendment of first N rows, export, and diagnostics.
Core Tests
packages/core/src/tests/core-api/amendFromTextSpecAndData.test.js
Jest suite validating rowCount defaults/bounds, invalid formats, non-integer rejection, stream ignored warning, CSV newline handling, JSON trailing-record behavior, explicit empty-record preservation, and parse errors.
Parser Options
packages/core/js/data_formats/delimiter-options.js
Defaults skipEmptyLines to true so terminal newlines don't create synthetic trailing CSV rows while preserving explicit empty records.
REST API Implementation
apps/api/src/index.js
Adds runAmend and sendAmendResponse, imports core amend function, validates options/seed/responseFormat, invokes core amend, and formats responses (rows/rendered/raw/all); registers POST /v1/generate/amend.
OpenAPI Spec
apps/api/src/openapi.js
Widened /v1/generate/fromschema rows item type to oneOf and added POST /v1/generate/amend operation with required request properties and 200/400 response schemas (including diagnostics).
API Tests
apps/api/src/tests/generate/v1-generate-amend.spec.js
Playwright tests for CSV/JSON input, responseFormat permutations, raw rendering, stream-ignored warning, validation failures, and fixture-driven rendered/header checks.
CLI Command
apps/cli/src/cli-options.js, apps/cli/src/run-cli.js
Adds amend subcommand with --schema-file,--data-file,--input-format; command-aware parsing and validation; buffered amend execution path that calls core amend and writes rendered output.
CLI Tests
apps/cli/src/tests/*
Unit and integration tests verifying amend option parsing, temp-file integration runs, runCliCommand amend behavior, and cross-format fixture assertions (CSV↔DSV).
MCP Tool
apps/mcp/src/index.js
Registers amend_data_from_spec tool with input/output JSON schemas, updates tools/list and tools/call to dispatch amend vs generate, and returns tool-specific error codes on failure.
MCP Tests
apps/mcp/src/mcp.test.js
Updates tool-listing assertions, adds amend tool call tests, validates stream-ignored warning, and adds parity test asserting MCP rendered output equals core rendered.
Documentation
README.md, docs-src/docs/070-interfaces-and-deployment/*, apps/cli/README.md
Adds CLI and API examples and signatures, documents POST /v1/generate/amend behavior (rowCount semantics, full dataset return, stream ignored), and updates MCP tool list.
Test Fixtures
test-fixtures/amend-cross-format/*
Adds schema, DSV input, and expected DSV output fixtures used by API and CLI fixture-driven tests.
Web Test Assertions
apps/web/src/tests/browser/*
Refines Playwright assertions to compare the first N column texts after sort/filter instead of single-cell checks.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Poem

🐰 I nibble old rows into something new,

Headers join and values hop through,
CSV, DSV, JSON dance in line,
Amend the top rows — now they shine,
A carrot clap for code divine!

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR includes some unrelated test updates to grid sorting assertions across multiple web browser test files that are not directly related to implementing amend functionality. Remove or separate the web browser test updates (grid-header-controls, filtering-sorting, column-sorting, clear-sort-before-add-rows-below) into a distinct PR.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: implementing amend functionality across three interfaces (API, MCP, and CLI).
Linked Issues check ✅ Passed The PR fully implements issue #42 requirements: amend endpoints/tools across API, MCP, and CLI accept both input data and schema in the amend calls.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 42-amend-calls-in-api-and-mcp-and-cli

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2cf0fcb10c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/core/src/index.js Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/mcp/src/index.js (1)

557-565: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Make the failure message tool-specific.

When amend_data_from_spec fails, clients still get "Failed to generate data from specification.", which is misleading for the new tool and makes troubleshooting harder.

Suggested tweak
     if (!result.ok) {
+      const failureMessage =
+        name === 'amend_data_from_spec'
+          ? 'Failed to amend data from specification.'
+          : 'Failed to generate data from specification.';
       return writeToolResult(
         id,
-        createToolError('generation_failed', 'Failed to generate data from specification.', {
+        createToolError('generation_failed', failureMessage, {
           errors: result.errors,
           diagnostics: result.diagnostics,
         }),
         true
       );
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/mcp/src/index.js` around lines 557 - 565, When handling a failed
generation result, make the error message tool-specific: inspect the incoming
name (used to choose amendFromTextSpecAndData vs generateFromTextSpec) and pass
a different human-readable message into createToolError before calling
writeToolResult; for example, if name === 'amend_data_from_spec' use a message
like "Failed to amend data from specification." otherwise keep "Failed to
generate data from specification.", and continue to include result.errors and
result.diagnostics when calling writeToolResult with id and normalizedArgs.
🧹 Nitpick comments (1)
apps/cli/src/run-cli.js (1)

61-73: ⚡ Quick win

Pass stream: false explicitly in amend mode.

The CLI warns that streaming is ignored but still forwards options.shouldStream to amendFromTextSpecAndData(). The core function ignores this parameter and emits a diagnostic warning. Hardcoding stream: false makes the intent explicit—we don't want streaming in amend mode—and prevents any future confusion about whether the parameter is actually used.

Proposed change
     const result = amendFromTextSpecAndData({
       textSpec,
       inputData,
       inputFormat: options.inputFormat,
       rowCount: options.rowCount,
       outputFormat: options.format,
       unsafeFakerExpressions: options.unsafeFakerExpressions,
-      stream: options.shouldStream,
+      stream: false,
     });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/cli/src/run-cli.js` around lines 61 - 73, The code warns streaming is
ignored in amend mode but still passes options.shouldStream into
amendFromTextSpecAndData; change the call to explicitly set stream: false so
amendFromTextSpecAndData receives a clear non-streaming intent. Locate the call
to amendFromTextSpecAndData and replace the stream argument (currently stream:
options.shouldStream) with stream: false, keeping all other parameters the same
and retaining the existing progress warning about ignored flags.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/api/src/index.js`:
- Around line 553-563: The catch block is leaking internal stack traces to
clients by returning error?.stack in the diagnostics field; update the handler
around the return that calls toErrorResponse so it does not include
diagnostics.stack (remove or replace with a generic diagnostics value), and
instead log the full error/server stack locally using the existing server logger
or console.error (referencing the same error variable), keeping the public
response from toErrorResponse generic (e.g., only include errors:
[error?.message || 'Unhandled amend error'] and no stack) while preserving the
500 status path.

In `@apps/api/src/openapi.js`:
- Around line 428-429: The OpenAPI response schema currently restricts the amend
response "rows" cells to strings (rows: items: items: { type: 'string' }), which
is too narrow; update the schema for the "rows" cell items in the amend response
to allow any JSON value (e.g., replace the inner items type with a free-form
schema such as an empty schema {} or a union/anyOf of string, number, boolean,
object, null) so clients/validators accept non-string cell values while leaving
"rendered" as string.

In `@packages/core/src/index.js`:
- Around line 167-175: normaliseAmendCount currently uses Number.parseInt which
silently accepts "1.5" and "2abc"; change the validation to explicitly reject
non-integer or junk-suffixed strings by validating the raw input string (e.g.,
use a regex like /^[+-]?\d+$/ on String(value).trim() or parse with Number and
verify Number.isInteger and that String(Number(value)) equals the trimmed input)
before converting to an integer; keep the existing behavior of returning
fallbackCount when value is undefined/null/'' and returning undefined for
invalid inputs, and update the implementation in the normaliseAmendCount
function.

---

Outside diff comments:
In `@apps/mcp/src/index.js`:
- Around line 557-565: When handling a failed generation result, make the error
message tool-specific: inspect the incoming name (used to choose
amendFromTextSpecAndData vs generateFromTextSpec) and pass a different
human-readable message into createToolError before calling writeToolResult; for
example, if name === 'amend_data_from_spec' use a message like "Failed to amend
data from specification." otherwise keep "Failed to generate data from
specification.", and continue to include result.errors and result.diagnostics
when calling writeToolResult with id and normalizedArgs.

---

Nitpick comments:
In `@apps/cli/src/run-cli.js`:
- Around line 61-73: The code warns streaming is ignored in amend mode but still
passes options.shouldStream into amendFromTextSpecAndData; change the call to
explicitly set stream: false so amendFromTextSpecAndData receives a clear
non-streaming intent. Locate the call to amendFromTextSpecAndData and replace
the stream argument (currently stream: options.shouldStream) with stream: false,
keeping all other parameters the same and retaining the existing progress
warning about ignored flags.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 67c53d6a-0e6e-4ef5-9966-72433d13d093

📥 Commits

Reviewing files that changed from the base of the PR and between 8226cb4 and 2cf0fcb.

⛔ Files ignored due to path filters (2)
  • test-fixtures/amend-cross-format/expected-output.csv is excluded by !**/*.csv
  • test-fixtures/amend-cross-format/input.csv is excluded by !**/*.csv
📒 Files selected for processing (24)
  • README.md
  • apps/api/src/index.js
  • apps/api/src/openapi.js
  • apps/api/src/tests/generate/v1-generate-amend.spec.js
  • apps/cli/README.md
  • apps/cli/src/cli-options.js
  • apps/cli/src/run-cli.js
  • apps/cli/src/tests/cli-options.test.js
  • apps/cli/src/tests/integration.cli-params.test.js
  • apps/cli/src/tests/run-cli.test.js
  • apps/mcp/src/index.js
  • apps/mcp/src/mcp.test.js
  • apps/web/src/tests/browser/abstraction-smoke-tests/grid-header-controls.spec.js
  • apps/web/src/tests/browser/app-coverage/filtering-sorting/filtering-sorting.spec.js
  • apps/web/src/tests/browser/planned-functional/filtering-sorting/column-sorting.spec.js
  • apps/web/src/tests/browser/planned-functional/grid-operations/clear-sort-before-add-rows-below.spec.js
  • docs-src/docs/070-interfaces-and-deployment/030-rest-api.md
  • docs-src/docs/070-interfaces-and-deployment/040-mcp.md
  • docs-src/docs/070-interfaces-and-deployment/050-cli-node-and-bun.md
  • packages/core/src/index.js
  • packages/core/src/tests/core-api/amendFromTextSpecAndData.test.js
  • test-fixtures/amend-cross-format/expected-output.dsv
  • test-fixtures/amend-cross-format/input.dsv
  • test-fixtures/amend-cross-format/schema.txt

Comment thread apps/api/src/index.js
Comment thread apps/api/src/openapi.js
Comment thread packages/core/src/index.js Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
apps/mcp/src/index.js (1)

408-411: 🏗️ Heavy lift

Derive amend inputFormat values from core instead of hardcoding them here.

This enum is now a second source of truth alongside the importer-backed validation in core. If importer support changes, MCP discovery can advertise the wrong contract even though runtime behavior is correct.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/mcp/src/index.js` around lines 408 - 411, Replace the hardcoded enum in
the inputFormat schema with the canonical list exported by core: import or call
the core module that exposes the supported importer formats and assign that
array to inputFormat.enum (i.e., remove ['csv','dsv',...] and use core's
supported formats). Ensure you reference the core export that represents
importer-backed validation (the single source of truth for supported formats)
and use it where inputFormat.enum is defined so MCP discovery always reflects
core's importer support.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/core/js/data_formats/delimiter-options.js`:
- Around line 10-12: The comment and behavior for skipEmptyLines in
delimiter-options.js are wrong: using skipEmptyLines: 'greedy' will drop rows
like ",," and ""; change the parser option to skipEmptyLines: true (in the same
config object where skipEmptyLines is currently set) so only trailing blank
lines are skipped and explicit empty records are preserved, and update the
inline comment to reflect the correct behavior; ensure this change occurs in the
configuration that defines skipEmptyLines in
packages/core/js/data_formats/delimiter-options.js.

In `@packages/core/src/index.js`:
- Around line 349-361: The code currently trims inputFormat only when validating
but then lowercases it without trimming before asking the importer, causing
values like " csv " to be rejected; update the normalization step to trim and
lowercase (e.g., set normalisedInputFormat =
String(inputFormat).trim().toLowerCase()) before calling createImporter() and
importer.canImport(normalisedInputFormat) so the importer lookup uses the
trimmed value; ensure references to inputFormat, normalisedInputFormat,
createImporter(), and importer.canImport() are updated accordingly.

---

Nitpick comments:
In `@apps/mcp/src/index.js`:
- Around line 408-411: Replace the hardcoded enum in the inputFormat schema with
the canonical list exported by core: import or call the core module that exposes
the supported importer formats and assign that array to inputFormat.enum (i.e.,
remove ['csv','dsv',...] and use core's supported formats). Ensure you reference
the core export that represents importer-backed validation (the single source of
truth for supported formats) and use it where inputFormat.enum is defined so MCP
discovery always reflects core's importer support.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c12bf2e5-60e7-499c-b92e-83038322e80a

📥 Commits

Reviewing files that changed from the base of the PR and between 2cf0fcb and 1e62286.

📒 Files selected for processing (8)
  • apps/api/src/index.js
  • apps/api/src/openapi.js
  • apps/api/src/tests/generate/v1-generate-amend.spec.js
  • apps/mcp/src/index.js
  • apps/mcp/src/mcp.test.js
  • packages/core/js/data_formats/delimiter-options.js
  • packages/core/src/index.js
  • packages/core/src/tests/core-api/amendFromTextSpecAndData.test.js
🚧 Files skipped from review as they are similar to previous changes (3)
  • apps/api/src/openapi.js
  • apps/mcp/src/mcp.test.js
  • apps/api/src/index.js

Comment thread packages/core/js/data_formats/delimiter-options.js Outdated
Comment thread packages/core/src/index.js
@eviltester eviltester merged commit 393e932 into master May 9, 2026
5 checks passed
@eviltester eviltester deleted the 42-amend-calls-in-api-and-mcp-and-cli branch May 9, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Amend calls in API and MCP and CLI

2 participants