Skip to content

feat: add jsonb[] type support in export-meta.ts and csv-to-pg parser #853

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pyramation merged 2 commits into from
Mar 20, 2026
Merged

feat: add jsonb[] type support in export-meta.ts and csv-to-pg parser #853

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1ba8a0e
feat: add jsonb[] type support for secure_table_provision export
pyramation Mar 20, 2026
aec7dac
fix: add jsonb[] coercion handler in csv-to-pg parser to prevent corr…
pyramation Mar 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions packages/csv-to-pg/src/parse.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -458,6 +458,39 @@ const getCoercionFunc = (type: string, from: string[], opts: FieldOptions, field
});
return wrapValue(val, opts);
};
case 'jsonb[]':
return (record: Record<string, unknown>): Node => {
const rawValue = record[from[0]];
if (isNullToken(rawValue)) {
return makeNullOrThrow(fieldName, rawValue, type, required, 'value is empty or null');
}
if (Array.isArray(rawValue)) {
if (rawValue.length === 0) {
return makeNullOrThrow(fieldName, rawValue, type, required, 'array is empty');
}
const elements = rawValue.map(el => JSON.stringify(el));
const arrayLiteral = psqlArray(elements);
if (isEmpty(arrayLiteral)) {
return makeNullOrThrow(fieldName, rawValue, type, required, 'failed to format array');
}
const val = nodes.aConst({
sval: ast.string({ sval: String(arrayLiteral) })
});
return wrapValue(val, opts);
}
// If it's a string, try to parse as JSON array
if (typeof rawValue === 'string') {
const parsed = parseJson(cleanseEmptyStrings(rawValue));
if (isEmpty(parsed)) {
return makeNullOrThrow(fieldName, rawValue, type, required, 'value is empty or null');
}
const val = nodes.aConst({
sval: ast.string({ sval: String(parsed) })
});
return wrapValue(val, opts);
}
return makeNullOrThrow(fieldName, rawValue, type, required, 'value is not an array');
};
default:
return (record: Record<string, unknown>): Node => {
const rawValue = record[from[0]];
Expand Down
7 changes: 5 additions & 2 deletions pgpm/core/src/export/export-meta.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { Parser } from 'csv-to-pg';
import { getPgPool } from 'pg-cache';
import type { Pool } from 'pg';

type FieldType = 'uuid' | 'uuid[]' | 'text' | 'text[]' | 'boolean' | 'image' | 'upload' | 'url' | 'jsonb' | 'int' | 'interval' | 'timestamptz';
type FieldType = 'uuid' | 'uuid[]' | 'text' | 'text[]' | 'boolean' | 'image' | 'upload' | 'url' | 'jsonb' | 'jsonb[]' | 'int' | 'interval' | 'timestamptz';

interface TableConfig {
schema: string;
Expand Down Expand Up @@ -35,6 +35,8 @@ const mapPgTypeToFieldType = (udtName: string): FieldType => {
case 'jsonb':
case 'json':
return 'jsonb';
case '_jsonb':
return 'jsonb[]';
case 'int4':
case 'int8':
case 'int2':
Expand Down Expand Up @@ -855,7 +857,8 @@ const config: Record<string, TableConfig> = {
use_rls: 'boolean',
node_data: 'jsonb',
grant_roles: 'text[]',
grant_privileges: 'jsonb',
fields: 'jsonb[]',
grant_privileges: 'jsonb[]',
Comment on lines +860 to +861
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 jsonb[] field type has no coercion handler in csv-to-pg parser, producing corrupted SQL output

The PR adds jsonb[] as a valid FieldType and uses it for secure_table_provision.fields and secure_table_provision.grant_privileges, but the downstream csv-to-pg parser (packages/csv-to-pg/src/parse.ts:461-472) has no 'jsonb[]' case in its getCoercionFunc switch statement. This causes jsonb[] values to fall through to the default handler, which calls String(value) on the raw value. Since node-postgres returns jsonb[] columns as JavaScript arrays of parsed objects, String([{"a":1},{"b":2}]) produces "[object Object],[object Object]" — corrupted, unrecoverable data in the generated SQL INSERT statements.

Prompt for agents 
The csv-to-pg parser at packages/csv-to-pg/src/parse.ts needs a new case for 'jsonb[]' in the getCoercionFunc function (around line 460, before the default case). The handler should:
1. Get the rawValue from the record
2. Check for null tokens
3. Verify the value is an array
4. JSON.stringify each element of the array
5. Format as a PostgreSQL array literal string, e.g. using the psqlArray helper but with JSON-stringified elements, or a custom approach like: '{' + array.map(el => '"' + JSON.stringify(el).replace(/\\/g, '\\\\').replace(/"/g, '\\"') + '"').join(',') + '}'
6. Return it as a string constant node

Example case to add in packages/csv-to-pg/src/parse.ts after the 'jsonb' case (around line 460):

    case 'jsonb[]':
      return (record) => {
        const rawValue = record[from[0]];
        if (isNullToken(rawValue)) {
          return makeNullOrThrow(fieldName, rawValue, type, required, 'value is empty or null');
        }
        if (Array.isArray(rawValue)) {
          if (rawValue.length === 0) {
            return makeNullOrThrow(fieldName, rawValue, type, required, 'array is empty');
          }
          const elements = rawValue.map(el => JSON.stringify(el));
          const arrayLiteral = psqlArray(elements);
          if (isEmpty(arrayLiteral)) {
            return makeNullOrThrow(fieldName, rawValue, type, required, 'failed to format array');
          }
          const val = nodes.aConst({ sval: ast.string({ sval: String(arrayLiteral) }) });
          return wrapValue(val, opts);
        }
        return makeNullOrThrow(fieldName, rawValue, type, required, 'value is not an array');
      };
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

policy_type: 'text',
policy_privileges: 'text[]',
policy_role: 'text',
Expand Down
Loading