Skip to content

Commit a1b8b59

Browse files
aleclarsonaleclarson
committed
chore: update readme
1 parent eee24ac commit a1b8b59

1 file changed

Lines changed: 25 additions & 168 deletions

File tree

‎README.md‎

Lines changed: 25 additions & 168 deletions
Original file line numberDiff line numberDiff line change
@@ -1,187 +1,44 @@
1-
# pg-schema-diff
2-
[![run_tests](https://github.com/stripe/pg-schema-diff/actions/workflows/run-tests.yaml/badge.svg)](https://github.com/stripe/pg-schema-diff/actions/workflows/run-tests.yaml)
3-
[![Go Report Card](https://goreportcard.com/badge/github.com/stripe/pg-schema-diff)](https://goreportcard.com/report/github.com/stripe/pg-schema-diff)
4-
[![Go Reference](https://pkg.go.dev/badge/github.com/stripe/pg-schema-diff.svg)](https://pkg.go.dev/github.com/stripe/pg-schema-diff)
5-
![GitHub Release](https://img.shields.io/github/v/release/stripe/pg-schema-diff?include_prereleases)
1+
# @pg-nano/pg-schema-diff
62

7-
Computes the diff(erences) between Postgres database schemas and generates the SQL required to get your database schema from point A to B with
8-
minimal downtime & locks. This enables you to take your database and migrate it to any desired schema defined in plain DDL.
3+
A cross-platform distribution of the `pg-schema-diff` binary, tailored for use with [pg-nano](https://github.com/pg-nano/pg-nano).
94

10-
The tooling attempts to use native postgres migration operations to perform online migrations and avoid locking wherever possible. Not all migrations will
11-
be lock-free and some might require downtime, but the hazards system will warn you ahead of time when that's the case.
12-
Stateful online migration techniques, like shadow tables, aren't yet supported.
5+
## Installation
136

14-
### Online index Replacement
15-
Your project's diff:
16-
```
17-
$ git diff
18-
diff --git a/schema/schema.sql b/schema/schema.sql
19-
index cc3a14b..cf4b32d 100644
20-
--- a/schema/schema.sql
21-
+++ b/schema/schema.sql
22-
@@ -2,5 +2,5 @@ CREATE TABLE foobar(
23-
created_at timestamp,
24-
message text
25-
);
26-
-CREATE INDEX message_idx ON foobar(message);
27-
+CREATE INDEX message_idx ON foobar(message, created_at);
28-
```
29-
The generated plan (*queries using `message_idx` will always have an index backing them, even while the new index is being built*):
30-
```
31-
$ pg-schema-diff plan --dsn "postgres://postgres:postgres@localhost:5432/postgres" --schema-dir ./schema
32-
################################ Generated plan ################################
33-
1. ALTER INDEX "message_idx" RENAME TO "pgschemadiff_tmpidx_message_idx_IiaKzkvPQtyA7ob9piVqiQ";
34-
-- Statement Timeout: 3s
35-
36-
2. CREATE INDEX CONCURRENTLY message_idx ON public.foobar USING btree (message, created_at);
37-
-- Statement Timeout: 20m0s
38-
-- Lock Timeout: 3s
39-
-- Hazard INDEX_BUILD: This might affect database performance. Concurrent index builds require a non-trivial amount of CPU, potentially affecting database performance. They also can take a while but do not lock out writes.
40-
41-
3. DROP INDEX CONCURRENTLY "pgschemadiff_tmpidx_message_idx_IiaKzkvPQtyA7ob9piVqiQ";
42-
-- Statement Timeout: 20m0s
43-
-- Lock Timeout: 3s
44-
-- Hazard INDEX_DROPPED: Dropping this index means queries that use this index might perform worse because they will no longer will be able to leverage it.
45-
```
46-
### Online `NOT NULL` constraint creation
47-
Your project's diff:
48-
```
49-
diff --git a/schema/schema.sql b/schema/schema.sql
50-
index cc3a14b..5a1cec2 100644
51-
--- a/schema/schema.sql
52-
+++ b/schema/schema.sql
53-
@@ -1,5 +1,5 @@
54-
CREATE TABLE foobar(
55-
- created_at timestamp,
56-
+ created_at timestamp NOT NULL,
57-
message text
58-
);
59-
CREATE INDEX message_idx ON foobar(message);
60-
```
61-
The generated plan (*leverages check constraints to eliminate the need for a long-lived access-exclusive lock on the table*):
62-
```
63-
$ pg-schema-diff plan --dsn "postgres://postgres:postgres@localhost:5432/postgres" --schema-dir ./schema
64-
################################ Generated plan ################################
65-
1. ALTER TABLE "public"."foobar" ADD CONSTRAINT "pgschemadiff_tmpnn_BCOxMXqAQwaXlKPCRXoMMg" CHECK("created_at" IS NOT NULL) NOT VALID;
66-
-- Statement Timeout: 3s
67-
68-
2. ALTER TABLE "public"."foobar" VALIDATE CONSTRAINT "pgschemadiff_tmpnn_BCOxMXqAQwaXlKPCRXoMMg";
69-
-- Statement Timeout: 3s
70-
71-
3. ALTER TABLE "public"."foobar" ALTER COLUMN "created_at" SET NOT NULL;
72-
-- Statement Timeout: 3s
73-
74-
4. ALTER TABLE "public"."foobar" DROP CONSTRAINT "pgschemadiff_tmpnn_BCOxMXqAQwaXlKPCRXoMMg";
75-
-- Statement Timeout: 3s
76-
```
77-
78-
# Key features
79-
* Declarative schema migrations
80-
* The use of postgres native operations for zero-downtime migrations wherever possible:
81-
* Concurrent index builds
82-
* Online index replacement: If some index is changed, the new version will be built before the old version is dropped, preventing a window where no index is backing queries
83-
* Online constraint builds: Constraints (check, foreign key) are added as `INVALID` before being validated, eliminating the need
84-
for a long access-exclusive lock on the table
85-
* Online `NOT NULL` constraint creation using check constraints to eliminate the need for an access-exclusive lock on the table
86-
* Prioritized index builds: Building new indexes is always prioritized over deleting old indexes
87-
* A comprehensive set of features to ensure the safety of planned migrations:
88-
* Operators warned of dangerous operations.
89-
* Migration plans are validated first against a temporary database exactly as they would be performed against the real database.
90-
* Strong support of partitions
91-
# Install
92-
## CLI
937
```bash
94-
go install github.com/stripe/pg-schema-diff/cmd/pg-schema-diff@latest
8+
pnpm add @pg-nano/pg-schema-diff
959
```
9610

97-
## Library
98-
```bash
99-
go get -u github.com/stripe/pg-schema-diff@latest
100-
```
101-
# Using CLI
102-
## 1. Apply schema to fresh database
103-
Create a directory to hold your schema files. Then, generate sql files and place them into a schema dir.
104-
```bash
105-
mkdir schema
106-
echo "CREATE TABLE foobar (id int);" > schema/foobar.sql
107-
echo "CREATE TABLE bar (id varchar(255), message TEXT NOT NULL);" > schema/bar.sql
108-
```
11+
## Usage
10912

110-
Apply the schema to a fresh database. [The connection string spec can be found here](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING).
111-
Setting the `PGPASSWORD` env var will override any password set in the connection string and is recommended.
11213
```bash
113-
pg-schema-diff apply --dsn "postgres://postgres:postgres@localhost:5432/postgres" --schema-dir schema
14+
pnpm pg-schema-diff --help
11415
```
11516

116-
## 2. Updating schema
117-
Update the SQL file(s)
118-
```bash
119-
echo "CREATE INDEX message_idx ON bar(message)" >> schema/bar.sql
120-
```
17+
You also have the option of spawning a child process to run the command:
12118

122-
Apply the schema. Any hazards in the generated plan must be approved
123-
```bash
124-
pg-schema-diff apply --dsn "postgres://postgres:postgres@localhost:5432/postgres" --schema-dir schema --allow-hazards INDEX_BUILD
19+
```js
20+
import spawn from "tinyspawn";
21+
22+
await spawn("./node_modules/.bin/pg-schema-diff --help", {
23+
stdio: "inherit",
24+
});
12525
```
12626

127-
# Using Library
128-
Docs to use the library can be found [here](https://pkg.go.dev/github.com/stripe/pg-schema-diff). Check out [the CLI](https://github.com/stripe/pg-schema-diff/tree/main/cmd/pg-schema-diff)
129-
for an example implementation with the library
27+
## Thanks
13028

131-
## 1. Generating plan
132-
```go
133-
// The tempDbFactory is used in plan generation to extract the new schema and validate the plan
134-
tempDbFactory, err := tempdb.NewOnInstanceFactory(ctx, func(ctx context.Context, dbName string) (*sql.DB, error) {
135-
copiedConfig := connConfig.Copy()
136-
copiedConfig.Database = dbName
137-
return openDbWithPgxConfig(copiedConfig)
138-
})
139-
if err != nil {
140-
panic("Generating the TempDbFactory failed")
141-
}
142-
defer tempDbFactory.Close()
143-
// Generate the migration plan
144-
plan, err := diff.Generate(ctx, connPool, diff.DDLSchemaSource(ddl),
145-
diff.WithTempDbFactory(tempDbFactory),
146-
diff.WithDataPackNewTables(),
147-
)
148-
if err != nil {
149-
panic("Generating the plan failed")
150-
}
151-
```
29+
This package wouldn't exist without these tools:
15230

153-
## 2. Applying plan
154-
We leave plan application up to the user. For example, you might want to take out a session-level advisory lock if you are
155-
concerned about concurrent migrations on your database. You might also want a second user to approve the plan
156-
before applying it.
31+
- [stripe/pg-schema-diff](https://github.com/stripe/pg-schema-diff)
32+
- [goreleaser](https://github.com/goreleaser/goreleaser)
33+
- [prantlf/grab-github-release](https://github.com/prantlf/grab-github-release)
15734

158-
Example apply:
159-
```go
160-
for _, stmt := range plan.Statements {
161-
if _, err := conn.ExecContext(ctx, fmt.Sprintf("SET SESSION statement_timeout = %d", stmt.Timeout.Milliseconds())); err != nil {
162-
panic(fmt.Sprintf("setting statement timeout: %s", err))
163-
}
164-
if _, err := conn.ExecContext(ctx, fmt.Sprintf("SET SESSION lock_timeout = %d", stmt.LockTimeout.Milliseconds())); err != nil {
165-
panic(fmt.Sprintf("setting lock timeout: %s", err))
166-
}
167-
if _, err := conn.ExecContext(ctx, stmt.ToSQL()); err != nil {
168-
panic(fmt.Sprintf("executing migration statement. the database maybe be in a dirty state: %s: %s", stmt, err))
169-
}
170-
}
171-
```
35+
## Release Process
17236

173-
# Supported Postgres versions
174-
Supported: 14, 15, 16
175-
Unsupported: <= 13 are not supported. Use at your own risk.
37+
1. Bump version with `npm version <patch|minor|major>`
38+
2. Push with `git push && git push --tags`
39+
3. Pre-compile binaries and upload with `goreleaser release --clean`
40+
4. Publish with `npm publish`
17641

177-
# Unsupported migrations
178-
An abridged list of unsupported migrations:
179-
- Views (Planned)
180-
- Privileges (Planned)
181-
- Types (Only enums are currently supported)
182-
- Renaming. The diffing library relies on names to identify the old and new versions of a table, index, etc. If you rename
183-
an object, it will be treated as a drop and an add
42+
## License
18443

185-
# Contributing
186-
This project is in its early stages. We appreciate all the feature/bug requests we receive, but we have limited cycles
187-
to review direct code contributions at this time. See [Contributing](CONTRIBUTING.md) to learn more.
44+
MIT

0 commit comments

Comments
 (0)