Skip to content

feat: add jsonb[] type support in export-meta.ts and csv-to-pg parser#853

Merged
pyramation merged 2 commits intomainfrom
devin/1773967756-export-meta-jsonb-array
Mar 20, 2026
Merged

feat: add jsonb[] type support in export-meta.ts and csv-to-pg parser#853
pyramation merged 2 commits intomainfrom
devin/1773967756-export-meta-jsonb-array

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

@pyramation pyramation commented Mar 20, 2026

Summary

Companion PR to constructive-db#629, which migrates secure_table_provision.fields and secure_table_provision.grant_privileges from jsonb to jsonb[].

This PR updates the export-meta config to match the new column types:

  • Adds 'jsonb[]' to the FieldType union and maps PostgreSQL's internal _jsonb type name to it
  • Changes grant_privileges from 'jsonb' to 'jsonb[]' in the secure_table_provision config
  • Adds fields: 'jsonb[]' to the secure_table_provision config (this column existed in the DB but was previously missing from the export config)

Updates since last revision

Added a jsonb[] coercion handler in packages/csv-to-pg/src/parse.ts (getCoercionFunc) to prevent corrupted SQL output. Without this, jsonb[] values fell through to the default handler, which called String(value) on parsed JavaScript arrays — producing [object Object],[object Object] instead of valid PostgreSQL array literals.

The new handler:

  • Array input (primary path, e.g. from node-postgres): JSON.stringifys each element, then formats via psqlArray() into a PG array literal like {"{\"name\":\"col\"}","{\"type\":\"text\"}"}
  • String input (fallback): passes through parseJson as-is
  • Null/empty: delegates to makeNullOrThrow

Review & Testing Checklist for Human

  • Verify jsonb[] coercion produces valid SQL: The new jsonb[] case in csv-to-pg/src/parse.ts has no test coverage. Manually verify that psqlArray + escapeArrayElement correctly handles JSON strings containing quotes, backslashes, and nested objects — the double-escaping of " inside PG array literals is error-prone. Consider adding a unit test in packages/csv-to-pg.
  • Verify the string fallback path is correct: When rawValue is a string (not an array), the handler passes it through parseJson which returns it as-is. For a jsonb[] column, this would produce a SQL string literal like '[{"a":1}]' — a JSON array, not a PostgreSQL array literal (ARRAY[...] or {...} syntax). Confirm whether PostgreSQL accepts this implicit cast, or if this path needs to produce a proper PG array literal instead.
  • Confirm adding fields column is safe: The fields column was missing from the export config before this PR. Verify this was an oversight (not intentional) and that adding it won't cause issues with existing export data or the Parser.
  • Merge order: This PR should be merged after constructive-db#629, since the DB column types must be jsonb[] before the export config expects them to be.

Notes

  • The buildDynamicFields function intersects the hardcoded config with actual DB columns, so both the static and dynamic paths now resolve _jsonb'jsonb[]' rather than the old default of 'text'.
  • constructive-db#629 has already been merged.

Link to Devin session: https://app.devin.ai/sessions/5d550def7a314c97854ec12fa09dd4ca
Requested by: @pyramation


Open with Devin

- Add 'jsonb[]' to FieldType union
- Map '_jsonb' PostgreSQL type to 'jsonb[]' in mapPgTypeToFieldType
- Add missing 'fields' column (jsonb[]) to secure_table_provision config
- Update grant_privileges from 'jsonb' to 'jsonb[]' in secure_table_provision config

Companion to constructive-db Phase 0 provision layer cleanup.
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 3 additional findings in Devin Review.

Open in Devin Review

Comment on lines +860 to +861
fields: 'jsonb[]',
grant_privileges: 'jsonb[]',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 jsonb[] field type has no coercion handler in csv-to-pg parser, producing corrupted SQL output

The PR adds jsonb[] as a valid FieldType and uses it for secure_table_provision.fields and secure_table_provision.grant_privileges, but the downstream csv-to-pg parser (packages/csv-to-pg/src/parse.ts:461-472) has no 'jsonb[]' case in its getCoercionFunc switch statement. This causes jsonb[] values to fall through to the default handler, which calls String(value) on the raw value. Since node-postgres returns jsonb[] columns as JavaScript arrays of parsed objects, String([{"a":1},{"b":2}]) produces "[object Object],[object Object]" — corrupted, unrecoverable data in the generated SQL INSERT statements.

Prompt for agents
The csv-to-pg parser at packages/csv-to-pg/src/parse.ts needs a new case for 'jsonb[]' in the getCoercionFunc function (around line 460, before the default case). The handler should:
1. Get the rawValue from the record
2. Check for null tokens
3. Verify the value is an array
4. JSON.stringify each element of the array
5. Format as a PostgreSQL array literal string, e.g. using the psqlArray helper but with JSON-stringified elements, or a custom approach like: '{' + array.map(el => '"' + JSON.stringify(el).replace(/\\/g, '\\\\').replace(/"/g, '\\"') + '"').join(',') + '}'
6. Return it as a string constant node

Example case to add in packages/csv-to-pg/src/parse.ts after the 'jsonb' case (around line 460):

    case 'jsonb[]':
      return (record) => {
        const rawValue = record[from[0]];
        if (isNullToken(rawValue)) {
          return makeNullOrThrow(fieldName, rawValue, type, required, 'value is empty or null');
        }
        if (Array.isArray(rawValue)) {
          if (rawValue.length === 0) {
            return makeNullOrThrow(fieldName, rawValue, type, required, 'array is empty');
          }
          const elements = rawValue.map(el => JSON.stringify(el));
          const arrayLiteral = psqlArray(elements);
          if (isEmpty(arrayLiteral)) {
            return makeNullOrThrow(fieldName, rawValue, type, required, 'failed to format array');
          }
          const val = nodes.aConst({ sval: ast.string({ sval: String(arrayLiteral) }) });
          return wrapValue(val, opts);
        }
        return makeNullOrThrow(fieldName, rawValue, type, required, 'value is not an array');
      };
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@devin-ai-integration devin-ai-integration Bot changed the title feat: add jsonb[] type support in export-meta.ts for provision layer cleanup feat: add jsonb[] type support in export-meta.ts and csv-to-pg parser Mar 20, 2026
@pyramation pyramation merged commit c3b742f into main Mar 20, 2026
43 checks passed
@pyramation pyramation deleted the devin/1773967756-export-meta-jsonb-array branch March 20, 2026 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant