fix: resolve SQLite database file locking on Windows for unittests#479
Merged
Conversation
- Add teardown_sqlalchemy() to properly dispose engine and release file locks - Fix clear_db() to dispose its engine in a finally block - Update all test teardowns to use teardown_sqlalchemy() - Add ignore_errors=True to shutil.rmtree calls as safety net - Add windows-latest to CI test matrix - Use poetry run instead of source $VENV for cross-platform CI Closes #291
Rename OHLCV CSV files that contained colons and spaces in their filenames (e.g. '08:00:00') to use dashes instead (e.g. '08-00'), matching the convention used by all other test data files. Windows does not allow colons in filenames, causing git checkout to fail on Windows CI runners.
- Call teardown_sqlalchemy() before os.remove() in initialize_storage() to release SQLite file locks on Windows (PermissionError WinError 32) - Use remove_database_if_exists=True in test setUp to prevent stale DB state between tests - Replace manual os.remove/os.rmdir loops with shutil.rmtree in web controller test teardowns - Open verbose output file with encoding='utf-8' in validate_backtest_checkpoints to fix UnicodeEncodeError on Windows (cp1252 cannot encode checkmark/cross characters)
MDUYN
force-pushed
the
291-fix-windows-unittest-database-locking
branch
from
29d3ab1 to
089b09e
Compare
StaticPool._close_connection() is a no-op, so engine.dispose() alone never closes the underlying sqlite3 connection. On Windows, this leaves mandatory file locks in place, causing PermissionError when tests try to delete the database file. Use Connection.invalidate() which bypasses the pool's _close_connection and calls dialect.do_close() directly to actually close the DBAPI connection. Also add gc.collect() in initialize_storage to ensure any lingering references are cleaned up before file removal.
…ndows RunTestBase.tearDown() was calling shutil.rmtree with ignore_errors=True but without first releasing the SQLite file lock via teardown_sqlalchemy(). On Windows, this left the database file behind, causing the next test to find a stale BITVAVO portfolio when expecting BINANCE.
MDUYN
force-pushed
the
291-fix-windows-unittest-database-locking
branch
from
235e62a to
526fb1a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes SQLite database file locking that prevents unit tests from running on Windows. On Windows, file locks are mandatory (unlike Unix/macOS), so SQLAlchemy's open connections prevent
shutil.rmtree()from deleting database files during test teardown.Changes
Core fix
teardown_sqlalchemy()function insql_alchemy.pythat properly closes all sessions, disposes the engine, and unbinds the Session — releasing the file lockclear_db()to callengine.dispose()in afinallyblock to prevent connection leaksTest cleanup updates
TestBaseandFlaskTestBaseintest_base.pyto useteardown_sqlalchemy()instead ofclose_all_sessions()teardown_sqlalchemy()calls to all standalone test files with their own teardown logicignore_errors=Trueto allshutil.rmtree()calls as a safety netCI
windows-latestto the test matrix in.github/workflows/test.ymlsource $VENVtopoetry runfor cross-platform compatibilityCloses #291