Skip to content

Allow empty table with allow_empty flag#37

Merged
timb07 merged 1 commit into
mainfrom
allow-empty-table-with-bigint-conversion
Jun 5, 2025
Merged

Allow empty table with allow_empty flag#37
timb07 merged 1 commit into
mainfrom
allow-empty-table-with-bigint-conversion

Conversation

@timb07
Copy link
Copy Markdown
Contributor

@timb07 timb07 commented Jun 4, 2025

Prior to this change, running psycopack on an empty table would cause the TableIsEmpty exception to be raised.

This change adds an allow_empty option to allow empty tables to be handled without raising an exception. This enables psycopack to be used (e.g. for schema changes) irrespective of whether the table is empty or not.

This PR has been changed following discussions in this thread; here's the previous description:

Prior to this change, running psycopack on an empty table would cause the TableIsEmpty exception to be raised.

This change adds a check for a schema change (initially just PK bigint conversion) before checking for an empty table. This enables psycopack to be used for schema changes irrespective of whether the table is empty or not.

@timb07 timb07 requested review from felcury and marcelofern June 4, 2025 00:51
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 4, 2025

Coverage Report Results

Name Stmts Miss Branch BrPart Cover
src/psycopack/__init__.py 6 0 0 0 100%
src/psycopack/_commands.py 109 0 8 0 100%
src/psycopack/_conn.py 5 0 0 0 100%
src/psycopack/_const.py 3 0 0 0 100%
src/psycopack/_cur.py 21 0 0 0 100%
src/psycopack/_identifiers.py 12 0 2 0 100%
src/psycopack/_introspect.py 157 0 14 0 100%
src/psycopack/_logging.py 4 0 0 0 100%
src/psycopack/_psycopg.py 5 0 0 0 100%
src/psycopack/_registry.py 58 0 6 0 100%
src/psycopack/_repack.py 349 0 116 0 100%
src/psycopack/_tracker.py 165 0 36 0 100%
tests/conftest.py 19 0 0 0 100%
tests/factories.py 33 0 4 0 100%
tests/test_cur.py 20 0 2 0 100%
tests/test_fixtures.py 5 0 0 0 100%
tests/test_package.py 3 0 0 0 100%
tests/test_repack.py 679 0 0 0 100%
TOTAL 1653 0 188 0 100%

1 empty file skipped.

Comment thread src/psycopack/_repack.py Outdated
with self.tracker.track(_tracker.Stage.PRE_VALIDATION):
if self.introspector.table_is_empty(table=self.table):
raise TableIsEmpty("No reason to repack an empty table.")
if not self.convert_pk_to_bigint:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Are there any other schema changes where it is valid to avoid an error for an empty table here? And if not are there likely to be others added? Mainly thinking if that's the case then having a dedicated function for this that can be extended if other schema changes are added rather than needing to dig through

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We expect to support other schema changes (such as adding an exclusion constraint) at some point.

For now, I think it's okay. But when we do add other schema changes, we should refactor the check into a function as you suggest.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on Suzannah's suggestion, I think it makes the code more self-documenting (i.e. this bit cares about schema changes, not bigint conversions specifically, it just so happens that the only supported schema change currently is bigint conversion)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following other discussions, I've changed this PR to add the allow_empty flag instead of inferring that empty tables should be allowed due the presence of a schema change (initially just bigint conversion). With that change, I think this discussion thread is now moot. Agree?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I like this! The allow_empty flag makes a lot of sense

Copy link
Copy Markdown

@s-cooper18 s-cooper18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense

Copy link
Copy Markdown

@felcury felcury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! 👍

Comment thread src/psycopack/_repack.py Outdated
with self.tracker.track(_tracker.Stage.PRE_VALIDATION):
if self.introspector.table_is_empty(table=self.table):
raise TableIsEmpty("No reason to repack an empty table.")
if not self.convert_pk_to_bigint:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on Suzannah's suggestion, I think it makes the code more self-documenting (i.e. this bit cares about schema changes, not bigint conversions specifically, it just so happens that the only supported schema change currently is bigint conversion)

Prior to this change, running psycopack on an empty table would cause the
TableIsEmpty exception to be raised.

This change adds an option to allow empty tables to be handled without
raising an exception. This enables psycopack to be used (e.g. for schema
changes) irrespective of whether the table is empty or not.
@timb07 timb07 force-pushed the allow-empty-table-with-bigint-conversion branch from d8c1996 to 85f64e2 Compare June 4, 2025 23:05
@timb07 timb07 changed the title Allow empty table when also doing a schema change Allow empty table with allow_empty flag Jun 4, 2025
@timb07 timb07 requested review from felcury and s-cooper18 June 4, 2025 23:11
Copy link
Copy Markdown

@s-cooper18 s-cooper18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the change - big tick from me

@timb07 timb07 merged commit d577d44 into main Jun 5, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants