Skip to content

Commit 74ffdd5

Browse files
CahidArdaAndriiShermanjawjdankochetovSukairo-02
authored
Use lua scripts for get tag and onMutate (#4399)
* DPRINT!!! * Updates to neon-http for `@neondatabase/serverless@1.0.0`, when released (#4237) * Updates for @neondatabase/serverless@1.0.0 compatibility * Clearer comments * Linting * Add release notes * Shard integration tests, parallelise attw * Thank you pnpm 10 * Split int tests by provider * Update Neon config, fix bash * Restore webSocketConstructor * Use Docker for Neon * Use docker-compose for Neon * Fix env var * Run Neon HTTP tests on real DB * Downgrade attw * Use Bun for attw * Split unit tests * Fix command * Update latest pipeline * Fix test * Remove await * Split relational tests * Disable singlestore-relational * Remove gel-relational * Various fixes, features bundled for v0.41.0 (#4293) * Various fixes, features bundled for v0.41.0 * Fixed broken test case * D1 Buffer mapping fix, tests fix * Disabled type parsers for `neon-http` driver * [drizzle-kit] push to d1-http failed (#4268) * drizzle-kit and push to d1-http failed There are two reasons: - d1-http don't support transactions with db.run("begin/commit/rollback") - introspections run against some CF-D1 internal tables and fails later. that could mitigated with a right tablesFilter: ['!_cf_KV']. There was a incomplete mitigation in place which now includes _cf_KV tables. * chore: there are more _cf_ prefixed tables * fix: Add escaping to sqlite pull queries * should we pass-with-no-tests ?? * D P R I N T * Add ci vitest config * Bump kit, added release notes --------- Co-authored-by: Roman <nabukhotnyiroman@gmail.com> Co-authored-by: AndriiSherman <andreysherman11@gmail.com> * Add Arktype validation (via `drizzle-arktype` package) (#4314) * Export (almost) everything from validator packages * Support infinitely recursive types in JSON columns * Fix import * Format * Init drizzle-arktype * Finish drizzle-arktype * Sync fixes from other branch * Update READMEs * Update README * Type optimizations and better debugging for tests * Add CI/CD updates for arktype package * Bump arktype * bump to 0.1.2 * Update turbo config * Bump all validator packages versions --------- Co-authored-by: Andrii Sherman <andreysherman11@gmail.com> Co-authored-by: David Blass <david@arktype.io> * feat: add lua scripts for get tag and onMutate * Added hexp + peerDeps * fix: use hexpire option in hexpire * fix: add hset back * dprint * Update pnpm lock * fix: getByTag script * fix: exit getByTag script if compositeTable doesn't exist * Add all test cases for pg * Add mysql cache functions * 0.41 (#4416) * fix: incorrect types for inArray (#1774) Co-authored-by: Andrii Sherman <andreysherman11@gmail.com> * Pass row type parameter to @planetscale/database's execute (#1852) * Update session.ts No need to cast, you can just pass a type parameter * Update package.json --------- Co-authored-by: AndriiSherman <andreysherman11@gmail.com> * Don't enforce type restrictions on mysqlEnum and pgEnum to be non-empty arrays (#2429) * Removed type restriction on non-empty arrays for mysqEnum * Removed type restriction on non-empty arrays for pgEnum * check values argument is not an empty array for pgEnum * fix: typings * Add type tests --------- Co-authored-by: AndriiSherman <andreysherman11@gmail.com> * Export mapColumnToSchema function (#2495) Co-authored-by: Andrii Sherman <andreysherman11@gmail.com> * [Pg-kit] Fix malformed array literal error on indexes (#2884) * Fix malformed array literal error on indexes The main issue is the expression text to array conversion happening in the edited line. Commas in an expression become delimiters and split the expression up prematurely. Some special characters like double quotes can cause the malformed array literal errors. The postgres function pg_get_indexdef does what the snippet above is trying to do, but safely. * Add index introspect test * Update pg.test.ts Remove .only in basic index test --------- Co-authored-by: Andrii Sherman <andreysherman11@gmail.com> * add infer enum type (#2552) * Update how enums work in pg and mysql * Remove duplicated exports, add related test (#4413) * Remove duplicated exports, add related test Fixes #4079 * Fix test * chore: updating esbuild version in drizzle-kit (#4046) * chore: updating esbuild version in drizzle-kit * Fix build errors --------- Co-authored-by: AndriiSherman <andreysherman11@gmail.com> * Drizzle-kit: fix recreate enums + altering data type to enums, from enums in pg (#4330) Co-authored-by: AndriiSherman <andreysherman11@gmail.com> * Skip test and try latest gel * Add release notes --------- Co-authored-by: James <5511220+Zamiell@users.noreply.github.com> Co-authored-by: Ayrton <git@ayrton.be> Co-authored-by: April Mintac Pineda <21032419+aprilmintacpineda@users.noreply.github.com> Co-authored-by: Matthew Ary <157217+MatthewAry@users.noreply.github.com> Co-authored-by: Kratious <Kratious@users.noreply.github.com> Co-authored-by: Toti Muñoz <64804554+totigm@users.noreply.github.com> Co-authored-by: Dan Kochetov <danil.kochetov@gmail.com> Co-authored-by: Paul Marsicovetere <71470776+paulmarsicloud@users.noreply.github.com> Co-authored-by: Aleksandr Sherman <102579553+AleksandrSherman@users.noreply.github.com> * Update CI/CD to 22.04 * add planetscale cache tests * Add sqlite cache * Add singlestore db * Add gel cache * Fix build errors * Fix imports * fix pg default schema --------- Co-authored-by: Andrii Sherman <andreysherman11@gmail.com> Co-authored-by: George MacKerron <george@mackerron.co.uk> Co-authored-by: Dan Kochetov <danil.kochetov@gmail.com> Co-authored-by: Sergey Reka <71607800+Sukairo-02@users.noreply.github.com> Co-authored-by: Meno Abels <meno.abels@adviser.com> Co-authored-by: Roman <nabukhotnyiroman@gmail.com> Co-authored-by: L-Mario564 <ka.mario564@gmail.com> Co-authored-by: David Blass <david@arktype.io> Co-authored-by: James <5511220+Zamiell@users.noreply.github.com> Co-authored-by: Ayrton <git@ayrton.be> Co-authored-by: April Mintac Pineda <21032419+aprilmintacpineda@users.noreply.github.com> Co-authored-by: Matthew Ary <157217+MatthewAry@users.noreply.github.com> Co-authored-by: Kratious <Kratious@users.noreply.github.com> Co-authored-by: Toti Muñoz <64804554+totigm@users.noreply.github.com> Co-authored-by: Paul Marsicovetere <71470776+paulmarsicloud@users.noreply.github.com> Co-authored-by: Aleksandr Sherman <102579553+AleksandrSherman@users.noreply.github.com>
1 parent 9acf9da commit 74ffdd5

270 files changed

Lines changed: 18448 additions & 9880 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/codeql.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ on:
1212
jobs:
1313
analyze:
1414
name: Analyze
15-
runs-on: ubuntu-20.04
15+
runs-on: ubuntu-22.04
1616
permissions:
1717
actions: read
1818
contents: read

.github/workflows/release-feature-branch.yaml

Lines changed: 262 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -7,23 +7,29 @@ on:
77
pull_request: {}
88

99
jobs:
10-
release:
10+
test:
1111
# only run on all pushes or pull requests from forks
1212
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository
1313
strategy:
1414
matrix:
15-
package:
15+
shard:
16+
- gel
17+
- planetscale
18+
- singlestore-core
19+
- singlestore-proxy
20+
- singlestore-prefixed
21+
- singlestore-custom
22+
- neon-http
23+
- neon-serverless
1624
- drizzle-orm
1725
- drizzle-kit
1826
- drizzle-zod
1927
- drizzle-seed
2028
- drizzle-typebox
2129
- drizzle-valibot
22-
- eslint-plugin-drizzle
23-
runs-on: ubuntu-20.04
24-
permissions:
25-
contents: read
26-
id-token: write
30+
- drizzle-arktype
31+
- other
32+
runs-on: ubuntu-22.04
2733
services:
2834
postgres-postgis:
2935
image: postgis/postgis:16-3.4
@@ -76,12 +82,162 @@ jobs:
7682
--health-retries 5
7783
ports:
7884
- 33306:3306
85+
singlestore:
86+
image: ghcr.io/singlestore-labs/singlestoredb-dev:latest
87+
env:
88+
ROOT_PASSWORD: singlestore
89+
ports:
90+
- 33307:3306
7991
steps:
8092
- uses: actions/checkout@v4
8193

8294
- uses: actions/setup-node@v4
8395
with:
84-
node-version: '22'
96+
node-version: '20.19'
97+
registry-url: 'https://registry.npmjs.org'
98+
99+
- uses: pnpm/action-setup@v3
100+
name: Install pnpm
101+
id: pnpm-install
102+
with:
103+
version: latest
104+
run_install: false
105+
106+
- name: Get pnpm store directory
107+
id: pnpm-cache
108+
shell: bash
109+
run: |
110+
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
111+
112+
- uses: actions/cache@v4
113+
name: Setup pnpm cache
114+
with:
115+
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
116+
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
117+
restore-keys: |
118+
${{ runner.os }}-pnpm-store-
119+
120+
- name: Install dependencies
121+
run: pnpm install
122+
123+
- name: Build Prisma client
124+
working-directory: drizzle-orm
125+
run: pnpm prisma generate --schema src/prisma/schema.prisma
126+
127+
- name: Build
128+
run: pnpm build
129+
130+
- name: Run tests
131+
env:
132+
PG_CONNECTION_STRING: postgres://postgres:postgres@localhost:55433/drizzle
133+
PG_VECTOR_CONNECTION_STRING: postgres://postgres:postgres@localhost:54321/drizzle
134+
PG_POSTGIS_CONNECTION_STRING: postgres://postgres:postgres@localhost:54322/drizzle
135+
MYSQL_CONNECTION_STRING: mysql://root:root@localhost:33306/drizzle
136+
PLANETSCALE_CONNECTION_STRING: ${{ secrets.PLANETSCALE_CONNECTION_STRING }}
137+
NEON_CONNECTION_STRING: ${{ secrets.NEON_CONNECTION_STRING }}
138+
# NEON_HTTP_CONNECTION_STRING: postgres://postgres:postgres@db.localtest.me:5432/postgres
139+
NEON_HTTP_CONNECTION_STRING: ${{ secrets.NEON_CONNECTION_STRING }}
140+
NEON_SERVERLESS_CONNECTION_STRING: postgres://postgres:postgres@localhost:5445/postgres
141+
TIDB_CONNECTION_STRING: ${{ secrets.TIDB_CONNECTION_STRING }}
142+
XATA_API_KEY: ${{ secrets.XATA_API_KEY }}
143+
XATA_BRANCH: ${{ secrets.XATA_BRANCH }}
144+
LIBSQL_URL: file:local.db
145+
LIBSQL_REMOTE_URL: ${{ secrets.LIBSQL_REMOTE_URL }}
146+
LIBSQL_REMOTE_TOKEN: ${{ secrets.LIBSQL_REMOTE_TOKEN }}
147+
SINGLESTORE_CONNECTION_STRING: singlestore://root:singlestore@localhost:33307/
148+
working-directory: integration-tests
149+
run: |
150+
if [[ ${{ github.event_name }} != "push" && "${{ github.event.pull_request.head.repo.full_name }}" != "${{ github.repository }}" ]]; then
151+
export SKIP_EXTERNAL_DB_TESTS=1
152+
fi
153+
154+
case ${{ matrix.shard }} in
155+
156+
gel)
157+
if [[ -z "$SKIP_EXTERNAL_DB_TESTS" ]]; then
158+
pnpm vitest run tests/gel
159+
fi
160+
;;
161+
162+
planetscale)
163+
if [[ -z "$SKIP_EXTERNAL_DB_TESTS" ]]; then
164+
pnpm vitest run \
165+
tests/mysql/mysql-planetscale.test.ts \
166+
tests/relational/mysql.planetscale-v1.test.ts \
167+
tests/relational/mysql.planetscale.test.ts
168+
fi
169+
;;
170+
171+
singlestore-core)
172+
pnpm vitest run tests/singlestore/singlestore.test.ts
173+
;;
174+
175+
singlestore-proxy)
176+
pnpm vitest run tests/singlestore/singlestore-proxy.test.ts
177+
;;
178+
179+
singlestore-prefixed)
180+
pnpm vitest run tests/singlestore/singlestore-prefixed.test.ts
181+
;;
182+
183+
singlestore-custom)
184+
pnpm vitest run tests/singlestore/singlestore-custom.test.ts
185+
;;
186+
187+
neon-http)
188+
if [[ -z "$SKIP_EXTERNAL_DB_TESTS" ]]; then
189+
pnpm vitest run tests/pg/neon-http.test.ts tests/pg/neon-http-batch.test.ts
190+
fi
191+
;;
192+
193+
neon-serverless)
194+
docker compose -f docker-neon.yml up -d
195+
pnpm vitest run --config=./vitest-ci.config.ts tests/pg/neon-serverless.test.ts
196+
docker compose -f docker-neon.yml down
197+
;;
198+
199+
drizzle-orm|drizzle-kit|drizzle-zod|drizzle-seed|drizzle-typebox|drizzle-valibot|drizzle-arktype)
200+
(cd .. && pnpm test --filter ${{ matrix.shard }})
201+
;;
202+
203+
other)
204+
pnpm vitest run \
205+
--exclude tests/gel \
206+
--exclude tests/mysql/mysql-planetscale.test.ts \
207+
--exclude tests/relational/mysql.planetscale-v1.test.ts \
208+
--exclude tests/relational/mysql.planetscale.test.ts \
209+
--exclude tests/singlestore/singlestore.test.ts \
210+
--exclude tests/singlestore/singlestore-proxy.test.ts \
211+
--exclude tests/singlestore/singlestore-prefixed.test.ts \
212+
--exclude tests/singlestore/singlestore-custom.test.ts \
213+
--exclude tests/pg/neon-http.test.ts \
214+
--exclude tests/pg/neon-http-batch.test.ts \
215+
--exclude tests/pg/neon-serverless.test.ts
216+
;;
217+
218+
esac
219+
220+
attw:
221+
# only run on all pushes or pull requests from forks
222+
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository
223+
strategy:
224+
matrix:
225+
package:
226+
- drizzle-orm
227+
- drizzle-kit
228+
- drizzle-zod
229+
- drizzle-seed
230+
- drizzle-typebox
231+
- drizzle-valibot
232+
- drizzle-arktype
233+
- eslint-plugin-drizzle
234+
runs-on: ubuntu-22.04
235+
steps:
236+
- uses: actions/checkout@v4
237+
238+
- uses: actions/setup-node@v4
239+
with:
240+
node-version: '18.18'
85241
registry-url: 'https://registry.npmjs.org'
86242

87243
- uses: pnpm/action-setup@v3
@@ -108,6 +264,9 @@ jobs:
108264
- name: Install dependencies
109265
run: pnpm install
110266

267+
- name: Install Bun
268+
uses: oven-sh/setup-bun@v2
269+
111270
- name: Check preconditions
112271
id: checks
113272
shell: bash
@@ -130,57 +289,115 @@ jobs:
130289
} >> $GITHUB_OUTPUT
131290
fi
132291
292+
- name: Build Prisma client
293+
if: steps.checks.outputs.has_new_release == 'true'
294+
working-directory: drizzle-orm
295+
run: pnpm prisma generate --schema src/prisma/schema.prisma
296+
133297
- name: Build
134298
if: steps.checks.outputs.has_new_release == 'true'
299+
run: pnpm build
300+
301+
- name: Pack
302+
if: steps.checks.outputs.has_new_release == 'true'
303+
working-directory: ${{ matrix.package }}
304+
run: npm run pack
305+
306+
- name: Run @arethetypeswrong/cli
307+
if: steps.checks.outputs.has_new_release == 'true'
308+
working-directory: ${{ matrix.package }}
309+
run: bunx attw package.tgz
310+
311+
release:
312+
# only run on all pushes or pull requests from forks
313+
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository
314+
needs:
315+
- test
316+
- attw
317+
strategy:
318+
matrix:
319+
package:
320+
- drizzle-orm
321+
- drizzle-kit
322+
- drizzle-zod
323+
- drizzle-seed
324+
- drizzle-typebox
325+
- drizzle-valibot
326+
- drizzle-arktype
327+
- eslint-plugin-drizzle
328+
runs-on: ubuntu-22.04
329+
permissions:
330+
contents: read
331+
id-token: write
332+
steps:
333+
- uses: actions/checkout@v4
334+
335+
- uses: actions/setup-node@v4
336+
with:
337+
node-version: '22'
338+
registry-url: 'https://registry.npmjs.org'
339+
340+
- uses: pnpm/action-setup@v3
341+
name: Install pnpm
342+
id: pnpm-install
343+
with:
344+
version: latest
345+
run_install: false
346+
347+
- name: Get pnpm store directory
348+
id: pnpm-cache
349+
shell: bash
135350
run: |
136-
(
137-
cd drizzle-orm
138-
pnpm prisma generate --schema src/prisma/schema.prisma
139-
)
140-
pnpm build
351+
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
141352
142-
- name: Run tests
143-
if: steps.checks.outputs.has_new_release == 'true'
144-
env:
145-
PG_CONNECTION_STRING: postgres://postgres:postgres@localhost:55433/drizzle
146-
PG_VECTOR_CONNECTION_STRING: postgres://postgres:postgres@localhost:54321/drizzle
147-
PG_POSTGIS_CONNECTION_STRING: postgres://postgres:postgres@localhost:54322/drizzle
148-
MYSQL_CONNECTION_STRING: mysql://root:root@localhost:33306/drizzle
149-
PLANETSCALE_CONNECTION_STRING: ${{ secrets.PLANETSCALE_CONNECTION_STRING }}
150-
NEON_CONNECTION_STRING: ${{ secrets.NEON_CONNECTION_STRING }}
151-
TIDB_CONNECTION_STRING: ${{ secrets.TIDB_CONNECTION_STRING }}
152-
XATA_API_KEY: ${{ secrets.XATA_API_KEY }}
153-
XATA_BRANCH: ${{ secrets.XATA_BRANCH }}
154-
LIBSQL_URL: file:local.db
155-
LIBSQL_REMOTE_URL: ${{ secrets.LIBSQL_REMOTE_URL }}
156-
LIBSQL_REMOTE_TOKEN: ${{ secrets.LIBSQL_REMOTE_TOKEN }}
353+
- uses: actions/cache@v4
354+
name: Setup pnpm cache
355+
with:
356+
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
357+
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
358+
restore-keys: |
359+
${{ runner.os }}-pnpm-store-
360+
361+
- name: Install dependencies
362+
run: pnpm install
363+
364+
- name: Check preconditions
365+
id: checks
366+
shell: bash
367+
working-directory: ${{ matrix.package }}
157368
run: |
158-
if [[ ${{ github.event_name }} != "push" && "${{ github.event.pull_request.head.repo.full_name }}" != "${{ github.repository }}" ]]; then
159-
export SKIP_EXTERNAL_DB_TESTS=1
160-
fi
161-
if [[ "${{ matrix.package }}" == "drizzle-orm" ]]; then
162-
pnpm test --filter ${{ matrix.package }} --filter integration-tests
369+
old_version="$(jq -r .version package.json)"
370+
version="$old_version-$(git rev-parse --short HEAD)"
371+
npm version $version
372+
tag="${{ github.ref_name }}"
373+
is_version_published="$(npm view ${{ matrix.package }} versions --json | jq -r '.[] | select(. == "'$version'") | . == "'$version'"')"
374+
375+
if [[ "$is_version_published" == "true" ]]; then
376+
echo "\`${{ matrix.package }}$version\` already published, adding tag \`$tag\`" >> $GITHUB_STEP_SUMMARY
377+
npm dist-tag add ${{ matrix.package }}@$version $tag
163378
else
164-
pnpm test --filter ${{ matrix.package }}
379+
{
380+
echo "version=$version"
381+
echo "tag=$tag"
382+
echo "has_new_release=true"
383+
} >> $GITHUB_OUTPUT
165384
fi
166385
167-
- name: Pack
386+
- name: Build Prisma client
387+
working-directory: drizzle-orm
388+
run: pnpm prisma generate --schema src/prisma/schema.prisma
389+
390+
- name: Build
168391
if: steps.checks.outputs.has_new_release == 'true'
169-
working-directory: ${{ matrix.package }}
170-
shell: bash
171-
env:
172-
NODE_AUTH_TOKEN: ${{ secrets.NPM_ACCESS_TOKEN }}
173-
run: |
174-
npm run pack
175-
176-
- name: Run @arethetypeswrong/cli
392+
run: pnpm build
393+
394+
- name: Pack
177395
if: steps.checks.outputs.has_new_release == 'true'
178396
working-directory: ${{ matrix.package }}
179397
shell: bash
180398
env:
181399
NODE_AUTH_TOKEN: ${{ secrets.NPM_ACCESS_TOKEN }}
182-
run: |
183-
pnpm attw package.tgz
400+
run: npm run pack
184401

185402
- name: Publish
186403
if: github.event_name == 'push' && steps.checks.outputs.has_new_release == 'true'

0 commit comments

Comments
 (0)