chore: require an empty target database for integration tests by erichare · Pull Request #423 · datastax/astrapy

erichare · 2026-06-17T16:40:46Z

Summary

Adds a fail-fast guard for integration tests when the target database is already populated.

The guard now:

checks all non-system keyspaces, not just the configured working keyspace
looks for collections, tables, and UDTs
exits with an error locally and in CI so skipped local runs do not go unnoticed
supports TOLERATE_POPULATED_DATABASE=yes for intentional narrow local test runs
applies to both base and vectorize integration tests

This is meant to catch leftover test objects early, before a long run fails later with object-limit/conflict errors.

Test plan

make format
Verified the guard fails before tests run when the target database contains existing objects
Verified TOLERATE_POPULATED_DATABASE=yes bypasses the guard for intentional narrow local runs
Verified the empty-database path still allows integration tests to start normally

The integration tests create and drop collections, tables and keyspaces on the target database. Running them against a database that is already in use risks clobbering its data. Add an autouse, session-scoped guard fixture in tests/base/integration that lists the collections and tables in the working keyspace before any test data is provisioned. When the keyspace is not empty: - in CI (CI env var set) the run is failed hard (pytest.exit, non-zero exit code), so a non-empty database cannot produce a green-but-skipped check that lets an untested PR merge; - locally the suite is skipped, to protect the developer's data. Either way the destructive test fixtures never run against a populated DB.

sl-at-ibm · 2026-06-25T14:56:44Z

Thanks for taking a stab at this feature!
However I have a few remarks that we should discuss before calling this PR final.

I would prefer the behaviour be the same between local and CI runs. When running tests locally,
whether targeting Astra or a local HCD, there are already several tests that are skipped for various reasons.
The risk here is for a test "skipped because DB populated" to go unnoticed when doing local development.
Also an important detail is that sometimes we run tests locally restricted to few, or just one, test function. In that case
we may want to tell the test setup "I know there's table xyz and collection abc already, but trust me, my test will fit nevertheless".
So I would request for an environment variable to silence this whole check:

TOLERATE_POPULATED_DATABASE="yes" uv run pytest tests/base/integration [...]

I'm OK with this feature covering only the 'base' portion of the tests (I'm not sure why the vectorize part should not get the
same safety treatment though. I understand that one is not part of CI but I'd say it wouldn't harm to make the caller conscious about
populated DBs.)
Unfortunately, I think, the checks need to be a bit wider than what is now in the fixture. The goal of this feature, the way I see it,
is not really to avoid destroying real data (we probably don't have a production collection called id_test_collection or so), rather
to avoid wasting 15+ minutes on a test run that ends up with a sad TOO_MANY_COLLECTIONS message which could have been prevented by exiting early.
In this view, we should ensure no collections are there across all user keyspaces. Adding a similar check for tables and UDTs seems a good
addition (again, not to protect any "prod" data, but to avoid the occasional silly create-type or create-table fail 15+ minutes in
because of a leftover table that did not get cleaned up right from a previous test run).

So I would expand the check by ensuring there are no collections/UDTs/tables across all keyspaces that are not 'system keyspaces'.
This would require access to the database admin to list the keyspaces, and then a loop over keyspaces to collect existing stuff.
There is a detail here: on Astra, the Data API filters out the system keyspaces for us, whereas on HCD this must be done by our code.

Let me see how that could work (rought draft, I tested it in the basic cases):

from __future__ import annotations

import logging
import os

import pytest

from astrapy import Database

from .conftest import IS_ASTRA_DB

logger = logging.getLogger("cagnoli")


# this is effectively empty: the following system keyspaces are suppressed by the Data API itself on Astra
ASTRA_DB_SYSTEM_KEYSPACES: list[str] = [
    # "system_traces",
    # "datastax_sla"
    # "data_endpoint_auth",
    # "system_auth"
    # "system_views",
    # "system_schema"
    # "system_virtual_schema"
    # "system",
]
HCD_SYSTEM_KEYSPACES: list[str] = [
    "system_auth",
    "system_schema",
    "system_distributed",
    "system",
    "system_traces",
    "system_views",
    "system_virtual_schema",
]


@pytest.fixture(scope="module", autouse=True)
def require_empty_target_database(sync_database: Database) -> None:
    if os.environ.get("TOLERATE_POPULATED_DATABASE", "").strip() != "":
        return

    collected_items: dict[str, list[str]] = {
        "collections": [],
        "tables": [],
        "types": [],
    }
    system_keyspaces = (
        ASTRA_DB_SYSTEM_KEYSPACES if IS_ASTRA_DB else HCD_SYSTEM_KEYSPACES
    )
    keyspaces = [
        ks
        for ks in sync_database.get_database_admin().list_keyspaces()
        if ks not in system_keyspaces
    ]
    for keyspace in keyspaces:
        _db = sync_database._copy(keyspace=keyspace)
        for collection in _db.list_collection_names():
            collected_items["collections"].append(f"{keyspace}.{collection}")
        for table in _db.list_table_names():
            collected_items["tables"].append(f"{keyspace}.{table}")
        for type_ in _db.list_type_names():
            collected_items["types"].append(f"{keyspace}.{type_}")
    if any(itm_list != [] for itm_list in collected_items.values()):
        items_summary = "; ".join(
            f"{kind}: {', '.join(sorted(itm_list))}"
            for kind, itm_list in sorted(collected_items.items())
            if itm_list != []
        )
        message = f"Non-empty target database detected. Items found: {items_summary}."
        pytest.exit(message, returncode=1)

@pytest.mark.describe("Test cagnoli")
def test_cagnoli() -> None:
    pass

WDYT? Keep in mind this can be de-prioritized if we are tight on the 2.3-release things! :)

erichare · 2026-06-25T15:46:16Z

Thank you @sl-at-ibm :) I agree with you. I updated the fixture so the behavior is now consistent locally and in CI: it exits early with an error if the target database is populated, unless TOLERATE_POPULATED_DATABASE=yes is set.

i also applied it across keyspaces... it filters system keyspaces, and checks collections/tables/UDTs across all non-system keyspaces. I used with_options(keyspace=...) rather than _copy, but otherwise tried to stay faithful to your example!

I left this scoped to the base integration suite for now since that is the always-run/CI-covered suite, but I agree the same guard could be extracted/shared for vectorize tests.

In fact, heck, let's add that really quick. I was going to say it could go in a follow up, but it probably makes sense just to have it here!... Done.

By the way, the i hardcoded the list of system keyspaces... do you know of a simple data-api way to get that list dynamically? i can update it if so...

sl-at-ibm · 2026-06-25T16:08:02Z

Awesome, Eric, thank you!

By the way, the i hardcoded the list of system keyspaces... do you know of a simple data-api way to get that list dynamically? i can update it if so...

I am not aware of such a handy cool thing and I highly doubt there is one. For now, at least, we can safely keep the hardcoded list without worrying too much (it's not even user-facing stuff anyway ...)

github-actions · 2026-06-25T16:33:37Z

Coverage report

for commit: 56c13c014bf40d32a09c756072367355955fe410.
download detailed report here.

                                                File   Stmts   Miss      Cover     Delta
----------------------------------------------------------------------------------------
                                     astrapy/repl.py      77     77      0.00%     0.00%
                              astrapy/admin/admin.py     849    335     60.54%     0.00%
                                 astrapy/__init__.py      26     10     61.54%     0.00%
                               astrapy/utils/meta.py      42     15     64.29%     0.00%
                                  astrapy/results.py      50     16     68.00%     0.00%
              astrapy/exceptions/table_exceptions.py      27      8     70.37%     0.00%
         astrapy/exceptions/collection_exceptions.py      40     11     72.50%     0.00%
                  astrapy/data_types/data_api_set.py      90     23     74.44%     0.00%
                         astrapy/exceptions/utils.py      53     11     79.25%     0.00%
                  astrapy/data/cursors/pagination.py      21      4     80.95%     0.00%
                           astrapy/authentication.py     138     24     82.61%     0.00%
                        astrapy/utils/user_agents.py      18      3     83.33%     0.00%
                                   astrapy/client.py      68     11     83.82%     0.00%
               astrapy/data_types/data_api_vector.py      44      7     84.09%     0.00%
                  astrapy/data/info/database_info.py     194     30     84.54%     0.00%
          astrapy/data/info/collection_descriptor.py     222     33     85.14%     0.00%
                      astrapy/data/info/reranking.py     122     18     85.25%     0.00%
                            astrapy/utils/parsing.py       7      1     85.71%     0.00%
                  astrapy/data_types/data_api_map.py      59      8     86.44%     0.00%
 astrapy/data/info/table_descriptor/table_listing.py      49      6     87.76%     0.00%
                            astrapy/data/database.py     657     75     88.58%     0.00%
 astrapy/data/info/table_descriptor/table_indexes.py     230     26     88.70%     0.00%
            astrapy/data_types/data_api_timestamp.py      98     11     88.78%     0.00%
                      astrapy/data/info/vectorize.py     131     14     89.31%     0.00%
    astrapy/settings/definitions/definitions_data.py      38      4     89.47%     0.00%
  astrapy/data/info/table_descriptor/type_listing.py      39      4     89.74%     0.00%
             astrapy/data_types/data_api_dict_udt.py      10      1     90.00%     0.00%
                              astrapy/utils/unset.py      10      1     90.00%     0.00%
                        astrapy/utils/api_options.py     206     20     90.29%     0.00%
astrapy/data/info/table_descriptor/table_altering.py     114     11     90.35%     0.00%
             astrapy/data_types/data_api_duration.py      55      5     90.91%     0.00%
                               astrapy/data/table.py     738     65     91.19%     0.00%
                      astrapy/utils/api_commander.py     248     21     91.53%     0.00%
 astrapy/data/info/table_descriptor/table_columns.py     204     17     91.67%     0.00%
 astrapy/data/info/table_descriptor/type_altering.py      85      7     91.76%     0.00%
                 astrapy/utils/duration_std_utils.py      92      7     92.39%     0.00%
         astrapy/exceptions/devops_api_exceptions.py      79      6     92.41%     0.00%
                          astrapy/data/collection.py     752     57     92.42%     0.00%
                astrapy/data/cursors/query_engine.py     214     16     92.52%     0.00%
         astrapy/event_observers/context_managers.py      27      2     92.59%     0.00%
                 astrapy/data_types/data_api_time.py      98      7     92.86%     0.00%
                 astrapy/data_types/data_api_date.py      89      6     93.26%     0.00%
 astrapy/data/info/table_descriptor/type_creation.py      32      2     93.75%     0.00%
             astrapy/exceptions/error_descriptors.py      53      3     94.34%     0.00%
                          astrapy/admin/endpoints.py      36      2     94.44%     0.00%
   astrapy/settings/definitions/definitions_types.py      19      1     94.74%     0.00%
         astrapy/data/utils/collection_converters.py      80      4     95.00%     0.00%
              astrapy/data/utils/table_converters.py     411     20     95.13%     0.00%
                 astrapy/data/cursors/find_cursor.py     631     30     95.25%     0.00%
astrapy/data/info/table_descriptor/table_creation.py      49      2     95.92%     0.00%
                astrapy/event_observers/observers.py      30      1     96.67%     0.00%
                      astrapy/data/cursors/cursor.py      92      3     96.74%     0.00%
                 astrapy/data/cursors/farr_cursor.py     350     11     96.86%     0.00%
                           astrapy/utils/str_enum.py      32      1     96.88%     0.00%
                   astrapy/utils/duration_c_utils.py      66      2     96.97%     0.00%
                         astrapy/utils/date_utils.py      80      2     97.50%     0.00%
   astrapy/settings/definitions/definitions_admin.py      41      1     97.56%     0.00%
           astrapy/exceptions/data_api_exceptions.py      84      1     98.81%     0.00%
           astrapy/data/utils/distinct_extractors.py     104      1     99.04%     0.00%
                           astrapy/admin/__init__.py       3      0    100.00%     0.00%
                              astrapy/api_options.py       3      0    100.00%     0.00%
                               astrapy/collection.py       2      0    100.00%     0.00%
                                astrapy/constants.py       5      0    100.00%     0.00%
                                  astrapy/cursors.py       7      0    100.00%     0.00%
                            astrapy/data/__init__.py       0      0    100.00%     0.00%
                    astrapy/data/cursors/__init__.py       1      0    100.00%     0.00%
             astrapy/data/cursors/reranked_result.py       8      0    100.00%     0.00%
                      astrapy/data/utils/__init__.py       0      0    100.00%     0.00%
      astrapy/data/utils/extended_json_converters.py      28      0    100.00%     0.00%
                   astrapy/data/utils/table_types.py      36      0    100.00%     0.00%
               astrapy/data/utils/vector_coercion.py      13      0    100.00%     0.00%
                      astrapy/data_types/__init__.py      10      0    100.00%     0.00%
                                 astrapy/database.py       2      0    100.00%     0.00%
                 astrapy/event_observers/__init__.py       5      0    100.00%     0.00%
                   astrapy/event_observers/events.py      50      0    100.00%     0.00%
                                      astrapy/ids.py       5      0    100.00%     0.00%
                                     astrapy/info.py      15      0    100.00%     0.00%
                        astrapy/settings/__init__.py       0      0    100.00%     0.00%
                        astrapy/settings/defaults.py      47      0    100.00%     0.00%
            astrapy/settings/definitions/__init__.py       0      0    100.00%     0.00%
                  astrapy/settings/error_messages.py       2      0    100.00%     0.00%
                                    astrapy/table.py       2      0    100.00%     0.00%
                           astrapy/utils/__init__.py       0      0    100.00%     0.00%
                     astrapy/utils/document_paths.py      46      0    100.00%     0.00%
                     astrapy/utils/python_version.py       5      0    100.00%     0.00%
                      astrapy/utils/request_tools.py      32      0    100.00%     0.00%
                      astrapy/exceptions/__init__.py     108     18     83.33%     1.85%
----------------------------------------------------------------------------------------
                                              totals    9035   1139     87.39%     0.02%

sl-at-ibm

LGTM, thanks for doing that! (you may want to check the suggestion I left)

Data API already does this filtering

github-actions · 2026-06-26T14:11:55Z

Coverage report

for commit: 173a003776dd6e2b3613c29f29332738803e2fee.
download detailed report here.

                                                File   Stmts   Miss      Cover     Delta
----------------------------------------------------------------------------------------
                                     astrapy/repl.py      77     77      0.00%     0.00%
                              astrapy/admin/admin.py     849    335     60.54%     0.00%
                                 astrapy/__init__.py      26     10     61.54%     0.00%
                               astrapy/utils/meta.py      42     15     64.29%     0.00%
                                  astrapy/results.py      50     16     68.00%     0.00%
              astrapy/exceptions/table_exceptions.py      27      8     70.37%     0.00%
         astrapy/exceptions/collection_exceptions.py      40     11     72.50%     0.00%
                  astrapy/data_types/data_api_set.py      90     23     74.44%     0.00%
                         astrapy/exceptions/utils.py      53     11     79.25%     0.00%
                  astrapy/data/cursors/pagination.py      21      4     80.95%     0.00%
                           astrapy/authentication.py     138     24     82.61%     0.00%
                      astrapy/exceptions/__init__.py     108     18     83.33%     0.00%
                        astrapy/utils/user_agents.py      18      3     83.33%     0.00%
                                   astrapy/client.py      68     11     83.82%     0.00%
               astrapy/data_types/data_api_vector.py      44      7     84.09%     0.00%
                  astrapy/data/info/database_info.py     194     30     84.54%     0.00%
          astrapy/data/info/collection_descriptor.py     222     33     85.14%     0.00%
                      astrapy/data/info/reranking.py     122     18     85.25%     0.00%
                            astrapy/utils/parsing.py       7      1     85.71%     0.00%
                  astrapy/data_types/data_api_map.py      59      8     86.44%     0.00%
 astrapy/data/info/table_descriptor/table_listing.py      49      6     87.76%     0.00%
                            astrapy/data/database.py     657     75     88.58%     0.00%
 astrapy/data/info/table_descriptor/table_indexes.py     230     26     88.70%     0.00%
            astrapy/data_types/data_api_timestamp.py      98     11     88.78%     0.00%
                      astrapy/data/info/vectorize.py     131     14     89.31%     0.00%
    astrapy/settings/definitions/definitions_data.py      38      4     89.47%     0.00%
  astrapy/data/info/table_descriptor/type_listing.py      39      4     89.74%     0.00%
             astrapy/data_types/data_api_dict_udt.py      10      1     90.00%     0.00%
                              astrapy/utils/unset.py      10      1     90.00%     0.00%
                        astrapy/utils/api_options.py     206     20     90.29%     0.00%
astrapy/data/info/table_descriptor/table_altering.py     114     11     90.35%     0.00%
             astrapy/data_types/data_api_duration.py      55      5     90.91%     0.00%
                               astrapy/data/table.py     738     65     91.19%     0.00%
                      astrapy/utils/api_commander.py     248     21     91.53%     0.00%
 astrapy/data/info/table_descriptor/table_columns.py     204     17     91.67%     0.00%
 astrapy/data/info/table_descriptor/type_altering.py      85      7     91.76%     0.00%
                 astrapy/utils/duration_std_utils.py      92      7     92.39%     0.00%
         astrapy/exceptions/devops_api_exceptions.py      79      6     92.41%     0.00%
                          astrapy/data/collection.py     752     57     92.42%     0.00%
                astrapy/data/cursors/query_engine.py     214     16     92.52%     0.00%
         astrapy/event_observers/context_managers.py      27      2     92.59%     0.00%
                 astrapy/data_types/data_api_time.py      98      7     92.86%     0.00%
                 astrapy/data_types/data_api_date.py      89      6     93.26%     0.00%
 astrapy/data/info/table_descriptor/type_creation.py      32      2     93.75%     0.00%
             astrapy/exceptions/error_descriptors.py      53      3     94.34%     0.00%
                          astrapy/admin/endpoints.py      36      2     94.44%     0.00%
   astrapy/settings/definitions/definitions_types.py      19      1     94.74%     0.00%
         astrapy/data/utils/collection_converters.py      80      4     95.00%     0.00%
              astrapy/data/utils/table_converters.py     411     20     95.13%     0.00%
                 astrapy/data/cursors/find_cursor.py     647     30     95.36%     0.00%
astrapy/data/info/table_descriptor/table_creation.py      49      2     95.92%     0.00%
                astrapy/event_observers/observers.py      30      1     96.67%     0.00%
                      astrapy/data/cursors/cursor.py      92      3     96.74%     0.00%
                           astrapy/utils/str_enum.py      32      1     96.88%     0.00%
                 astrapy/data/cursors/farr_cursor.py     358     11     96.93%     0.00%
                   astrapy/utils/duration_c_utils.py      66      2     96.97%     0.00%
                         astrapy/utils/date_utils.py      80      2     97.50%     0.00%
   astrapy/settings/definitions/definitions_admin.py      41      1     97.56%     0.00%
           astrapy/exceptions/data_api_exceptions.py      84      1     98.81%     0.00%
           astrapy/data/utils/distinct_extractors.py     104      1     99.04%     0.00%
                           astrapy/admin/__init__.py       3      0    100.00%     0.00%
                              astrapy/api_options.py       3      0    100.00%     0.00%
                               astrapy/collection.py       2      0    100.00%     0.00%
                                astrapy/constants.py       5      0    100.00%     0.00%
                                  astrapy/cursors.py       7      0    100.00%     0.00%
                            astrapy/data/__init__.py       0      0    100.00%     0.00%
                    astrapy/data/cursors/__init__.py       1      0    100.00%     0.00%
             astrapy/data/cursors/reranked_result.py       8      0    100.00%     0.00%
                      astrapy/data/utils/__init__.py       0      0    100.00%     0.00%
      astrapy/data/utils/extended_json_converters.py      28      0    100.00%     0.00%
                   astrapy/data/utils/table_types.py      36      0    100.00%     0.00%
               astrapy/data/utils/vector_coercion.py      13      0    100.00%     0.00%
                      astrapy/data_types/__init__.py      10      0    100.00%     0.00%
                                 astrapy/database.py       2      0    100.00%     0.00%
                 astrapy/event_observers/__init__.py       5      0    100.00%     0.00%
                   astrapy/event_observers/events.py      50      0    100.00%     0.00%
                                      astrapy/ids.py       5      0    100.00%     0.00%
                                     astrapy/info.py      15      0    100.00%     0.00%
                        astrapy/settings/__init__.py       0      0    100.00%     0.00%
                        astrapy/settings/defaults.py      47      0    100.00%     0.00%
            astrapy/settings/definitions/__init__.py       0      0    100.00%     0.00%
                  astrapy/settings/error_messages.py       2      0    100.00%     0.00%
                                    astrapy/table.py       2      0    100.00%     0.00%
                           astrapy/utils/__init__.py       0      0    100.00%     0.00%
                     astrapy/utils/document_paths.py      46      0    100.00%     0.00%
                     astrapy/utils/python_version.py       5      0    100.00%     0.00%
                      astrapy/utils/request_tools.py      32      0    100.00%     0.00%
----------------------------------------------------------------------------------------
                                              totals    9059   1139     87.43%     0.00%

erichare mentioned this pull request Jun 17, 2026

chore: skip integration tests when no target DB is configured #422

Closed

erichare requested a review from sl-at-ibm June 17, 2026 16:43

erichare linked an issue Jun 17, 2026 that may be closed by this pull request

Skip integration tests if target DB is non-empty #419

Closed

erichare force-pushed the chore-skip-integration-tests-nonempty-db branch from 127e795 to 3e49c02 Compare June 17, 2026 17:00

erichare changed the title ~~chore: skip integration tests unless the target database is empty~~ chore: require an empty target database for integration tests Jun 17, 2026

erichare mentioned this pull request Jun 17, 2026

Run Astra integration tests in merge queue only #418

Open

erichare added 3 commits June 25, 2026 08:36

Merge branch 'main' into chore-skip-integration-tests-nonempty-db

a6f5a2a

fix: merge conflicts

1813b33

chore: apply extracted empty logic to vectorize

52fb79e

sl-at-ibm reviewed Jun 25, 2026

View reviewed changes

Comment thread tests/empty_database_guard.py Outdated

sl-at-ibm reviewed Jun 25, 2026

View reviewed changes

Comment thread tests/empty_database_guard.py Outdated

sl-at-ibm approved these changes Jun 25, 2026

View reviewed changes

erichare added 3 commits June 25, 2026 14:30

fix: remove the list of system keyspaces

7a0e70c

Data API already does this filtering

fix: use extended booleanize env for "yes"

0b8906b

Merge branch 'main' into chore-skip-integration-tests-nonempty-db

cca0e26

sl-at-ibm approved these changes Jun 25, 2026

View reviewed changes

erichare merged commit 391940d into main Jun 26, 2026
134 of 151 checks passed

erichare deleted the chore-skip-integration-tests-nonempty-db branch June 26, 2026 14:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: require an empty target database for integration tests#423

chore: require an empty target database for integration tests#423
erichare merged 7 commits into
mainfrom
chore-skip-integration-tests-nonempty-db

erichare commented Jun 17, 2026 •

edited

Loading

Uh oh!

sl-at-ibm commented Jun 25, 2026 •

edited

Loading

Uh oh!

erichare commented Jun 25, 2026

Uh oh!

sl-at-ibm commented Jun 25, 2026

Uh oh!

github-actions Bot commented Jun 25, 2026

Uh oh!

Uh oh!

Uh oh!

sl-at-ibm left a comment

Uh oh!

github-actions Bot commented Jun 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

erichare commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

sl-at-ibm commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erichare commented Jun 25, 2026

Uh oh!

sl-at-ibm commented Jun 25, 2026

Uh oh!

github-actions Bot commented Jun 25, 2026

Coverage report

Uh oh!

Uh oh!

Uh oh!

sl-at-ibm left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 26, 2026

Coverage report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

erichare commented Jun 17, 2026 •

edited

Loading

sl-at-ibm commented Jun 25, 2026 •

edited

Loading