[PECOBLR-2086] Add comprehensive MST transaction metadata and edge case tests by vikrantpuppala · Pull Request #1 · vikrantpuppala/databricks-sql-python

vikrantpuppala · 2026-04-02T06:50:53Z

Summary

Adds comprehensive E2E transaction tests to tests/e2e/test_transactions.py covering gaps identified in the MST + xDBC Metadata RPCs audit
Removes @pytest.mark.skip from TestTransactions class so tests run against MST-enabled warehouses
Covers metadata operations in transactions (cursor.columns, cursor.tables, cursor.schemas, cursor.catalogs, SHOW/DESCRIBE statements)
Tests MSTCheckRule-blocked SQL statements and their error handling
Tests edge cases: empty transactions, DDL in transactions, connection close with pending txn, concurrent operations, batch operations, PreparedStatement-style parameterized queries
Renames test table to pysql_transaction_test_table to avoid collisions with JDBC driver tests
Fixes catalogOwned-preview → catalogManaged table property

Test plan

Run pytest tests/e2e/test_transactions.py against an MST-enabled warehouse
Verify all new tests pass alongside the existing tests

NO_CHANGELOG=true

This pull request was AI-assisted by Isaac.

…re (databricks#553) * Clean thrift files Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

…atabricks#544) This PR replaces the previous implementation of convert_decimals_in_arrow_table() with a more efficient approach that uses PyArrow's native casting operation instead of going through pandas conversion and array creation. - Remove conversion to pandas DataFrame via to_pandas() and apply() methods - Remove intermediate steps of creating array from decimal column and setting it back - Replace with direct type casting using PyArrow's cast() method - Build a new table with transformed columns rather than modifying the original table - Create a new schema based on the modified fields The new approach is more performant by avoiding pandas conversion overhead. The table below highlights substantial performance improvements when retrieving all rows from a table containing decimal columns, particularly when compression is disabled. Even greater gains were observed with compression enabled—showing approximately an 84% improvement (6 seconds compared to 39 seconds). Benchmarking was performed against e2-dogfood, with the client located in the us-west-2 region. ![image](https://github.com/user-attachments/assets/5407b651-8ab6-4c13-b525-cf912f503ba0) Signed-off-by: Jayant Singh <jayant.singh@databricks.com>

…cks#551)

new codeowners

…t to prevent server side resource leaks (databricks#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add

* PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>

More conditions to run github actions

* Added new table * Some more fixes

* [init] working code for corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use pytest parametrize instead of unittest subtests, switch to pytest instead of unittest primitives Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * initialisations, function calls, then assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove pytest parametrize, move back to unittest subTest to allow keeping the test inside ClientTestSuite Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove pytest import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

* Basic testing * testing examples * Basic working prototype * ttypes fix * Refractored the ttypes * nit * Added inline support * Reordered boolean to be above int * Check Working e2e tests prototype * More tests * Added unit tests * refractor * nit * nit * nit * nit

* Added change log * Update CHANGELOG.md Co-authored-by: Vikrant Puppala <vikrant.puppala@databricks.com> --------- Co-authored-by: Vikrant Puppala <vikrant.puppala@databricks.com>

nit

* Add test to check thrift field IDs --------- Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

databricks#613) * Revert "Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (databricks#554)" This reverts commit edfb283. * revert e2e

* Release version 4.0.5: Reverted cursor close handling changes to fix user errors. Updated version numbers in pyproject.toml and __init__.py. * Update CHANGELOG.md to include reference to issue databricks#613 for cursor close handling fix.

nit

…#621) * basic setup * Nit * working * moved pyjwt to code dependency * nit * nit * nit * nit * nit * nit * testing sdk * Refractor * logging * nit * nit * nit * nit * nit

* [PECO-197] Support Python 3.10 (#31) * Test with multiple python versions. * Update pyarrow to version 9.0.0 to address issue in relation to python 3.10 & a specific version of numpy being pulled in by pyarrow. Closes #26 Signed-off-by: David Black <dblack@atlassian.com> * Update changelog and bump to v2.0.4 (#34) * Update changelog and bump to v2.0.4 * Specifically thank @dbaxa for this change. Signed-off-by: Jesse Whitehouse <jesse@whitehouse.dev> * Bump to 2.0.5-dev on main (#35) Signed-off-by: Jesse Whitehouse <jesse@whitehouse.dev> * On Pypi, display the "Project Links" sidebar. (#36) Signed-off-by: Jesse Whitehouse <jesse@whitehouse.dev> * [ES-402013] Close cursors before closing connection (#38) * Add test: cursors are closed when connection closes Signed-off-by: Jesse Whitehouse <jesse@whitehouse.dev> * Bump version to 2.0.5 and improve CHANGELOG (#40) Signed-off-by: Jesse Whitehouse <jesse@whitehouse.dev> * fix dco issue Signed-off-by: Moe Derakhshani <moe.derakhshani@databricks.com> * fix dco issue Signed-off-by: Moe Derakhshani <moe.derakhshani@databricks.com> * dco tunning Signed-off-by: Moe Derakhshani <moe.derakhshani@databricks.com> * dco tunning Signed-off-by: Moe Derakhshani <moe.derakhshani@databricks.com> * Github workflows: run checks on pull requests from forks (#47) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * OAuth implementation (#15) This PR: * Adds the foundation for OAuth against Databricks account on AWS with BYOIDP. * It copies one internal module that Steve Weis @sweisdb wrote for Databricks CLI (oauth.py). Once ecosystem-dev team (Serge, Pieter) build a python sdk core we will move this code to their repo as a dependency. * the PR provides authenticators with visitor pattern format for stamping auth-token which later is intended to be moved to the repo owned by Serge @nfx and and Pieter @pietern * Automate deploys to Pypi (#48) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-205] Add functional examples (#52) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.1.0 (#54) Bump to v2.1.0 and update changelog Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [SC-110400] Enabling compression in Python SQL Connector (#49) Signed-off-by: Mohit Singla <mohit.singla@databricks.com> Co-authored-by: Moe Derakhshani <moe.derakhshani@databricks.com> * Add tests for parameter sanitisation / escaping (#46) * Refactor so we can unit test `inject_parameters` * Add unit tests for inject_parameters * Remove inaccurate comment. Per #51, spark sql does not support escaping a single quote with a second single quote. * Closes #51 and adds unit tests plus the integration test provided in #56 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Courtney Holcomb (@courtneyholcomb) Co-authored-by: @mcannamela * Bump thrift dependency to 0.16.0 (#65) Addresses https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.2.0 (#66) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Support Python 3.11 (#60) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.2.1 (#70) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Add none check on _oauth_persistence in DatabricksOAuthProvider (#71) Add none check on _oauth_persistence in DatabricksOAuthProvider to avoid app crash when _oauth_persistence is None. Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Support custom oauth client id and redirect port (#75) * Support custom oauth client id and rediret port range PySQL is used by other tools/CLIs which have own oauth client id, we need to expose oauth_client_id and oauth_redirect_port_range as the connection parameters to support this customization. Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Change oauth redirect port range to port Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Fix type check issue Signed-off-by: Jacky Hu <jacky.hu@databricks.com> Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Bump version to 2.2.2 (#76) Signed-off-by: Jacky Hu <jacky.hu@databricks.com> Signed-off-by: Jesse <jesse.whitehouse@databricks.com> * Merge staging ingestion into main (#78) Follow up to #67 and #64 * Regenerate TCLIService using latest TCLIService.thrift from DBR (#64) * SI: Implement GET, PUT, and REMOVE (#67) * Re-lock dependencies after merging `main` Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.3.0 and update changelog (#80) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Add pkgutil-style for the package (#84) Since the package is under databricks namespace. pip install this package will cause issue importing other packages under the same namespace like automl and feature store. Adding pkgutil style to resolve the issue. Signed-off-by: lu-wang-dl <lu.wang@databricks.com> * Add SQLAlchemy Dialect (#57) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump to version 2.4.0(#89) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix syntax in examples in root readme. (#92) Do this because the environment variable pulls did not have closing quotes on their string literals. * Less strict numpy and pyarrow dependencies (#90) Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Thomas Newton <thomas.w.newton@gmail.com> * Update example in docstring so query output is valid Spark SQL (#95) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.4.1 (#96) Per the sermver.org spec, updating the projects dependencies is considered a compatible change. https: //semver.org/#what-should-i-do-if-i-update-my-own-dependencies-without-changing-the-public-api Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Update CODEOWNERS (#97) * Add Andre to CODEOWNERS (#98) * Add Andre. Signed-off-by: Yunbo Deng <yunbo.deng@databricks.com> Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> * Revert the change temporarily so I can sign off. Signed-off-by: Yunbo Deng <yunbo.deng@databricks.com> Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> * Add Andre and sign off. Signed-off-by: Yunbo Deng <yunbo.deng@databricks.com> Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> * Remove redundant line Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> --------- Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> * Add external auth provider + example (#101) Signed-off-by: Andre Furlan <andre.furlan@databricks.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Retry on connection timeout (#103) A lot of the time we see the error `[Errno 110] Connection timed out`. This happens a lot in Azure, particularly. In this PR I make it a retryable error as it is safe Signed-off-by: Andre Furlan <andre.furlan@databricks.com> * [PECO-244] Make http proxies work (#81) Override thrift's proxy header encoding function. Uses the fix identified in https://github.com/apache/thrift/pull/2565 H/T @pspeter Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump to version 2.5.0 (#104) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix changelog release date for version 2.5.0 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Relax sqlalchemy requirement (#113) * Plus update docs about how to change dependency spec Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Update to version 2.5.1 (#114) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix SQLAlchemy timestamp converter + docs (#117) --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Relax pandas and alembic requirements (#119) Update dependencies for alembic and pandas per customer request Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump to version 2.5.2 (#118) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Use urllib3 for thrift transport + reuse http connections (#131) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Default socket timeout to 15 min (#137) Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Bump version to 2.6.0 (#139) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix: some thrift RPCs failed with BadStatusLine (#141) --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.6.1 (#142) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [ES-706907] Retry GetOperationStatus for http errors (#145) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.6.2 (#147) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-626] Support OAuth flow for Databricks Azure (#86) ## Summary Support OAuth flow for Databricks Azure ## Background Some OAuth endpoints (e.g. Open ID Configuration) and scopes are different between Databricks Azure and AWS. Current code only supports OAuth flow on Databricks in AWS ## What changes are proposed in this pull request? - Change `OAuthManager` to decouple Databricks AWS specific configuration from OAuth flow - Add `sql/auth/endpoint.py` that implements cloud specific OAuth endpoint configuration - Change `DatabricksOAuthProvider` to work with the OAuth configurations in different Databricks cloud (AWS, Azure) - Add the corresponding unit tests * Use a separate logger for unsafe thrift responses (#153) --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Improve e2e test development ergonomics (#155) --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Don't raise exception when closing a stale Thrift session (#159) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump to version 2.7.0 (#161) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Cloud Fetch download handler (#127) * Cloud Fetch download handler Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Issue fix: final result link compressed data has multiple LZ4 end-of-frame markers Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Addressing PR comments - Linting - Type annotations - Use response.ok - Log exception - Remove semaphore and only use threading.event - reset() flags method - Fix tests after removing semaphore - Link expiry logic should be in secs - Decompress data static function - link_expiry_buffer and static public methods - Docstrings and comments Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Changing logger.debug to remove url Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * _reset() comment to docstring Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * link_expiry_buffer -> link_expiry_buffer_secs Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> --------- Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Cloud Fetch download manager (#146) * Cloud Fetch download manager Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Bug fix: submit handler.run Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Type annotations Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Namedtuple -> dataclass Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Shutdown thread pool and clear handlers Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Docstrings and comments Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * handler.run is the correct call Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Link expiry buffer in secs Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Adding type annotations for download_handlers and downloadable_result_settings Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Move DownloadableResultSettings to downloader.py to avoid circular import Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Black linting Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Timeout is never None Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> --------- Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Cloud fetch queue and integration (#151) * Cloud fetch queue and integration Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Enable cloudfetch with direct results Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Typing and style changes Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Client-settable max_download_threads Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Docstrings and comments Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Increase default buffer size bytes to 104857600 Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Move max_download_threads to kwargs of ThriftBackend, fix unit tests Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Fix tests: staticmethod make_arrow_table mock not callable Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * cancel_futures in shutdown() only available in python >=3.9.0 Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Black linting Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Fix typing errors Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> --------- Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Cloud Fetch e2e tests (#154) * Cloud Fetch e2e tests Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Test case works for e2-dogfood shared unity catalog Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Moving test to LargeQueriesSuite and setting catalog to hive_metastore Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Align default value of buffer_size_bytes in driver tests Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Adding comment to specify what's needed to run successfully Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> --------- Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Update changelog for cloudfetch (#172) Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Improve sqlalchemy backward compatibility with 1.3.24 (#173) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * OAuth: don't override auth headers with contents of .netrc file (#122) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix proxy connection pool creation (#158) Signed-off-by: Sebastian Eckweiler <sebastian.eckweiler@mercedes-benz.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Sebastian Eckweiler <sebastian.eckweiler@mercedes-benz.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Relax pandas dependency constraint to allow ^2.0.0 (#164) Signed-off-by: Daniel Segesdi <daniel.segesdi@turbine.ai> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Use hex string version of operation ID instead of bytes (#170) --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy: fix has_table so it honours schema= argument (#174) --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix socket timeout test (#144) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Disable non_native_boolean_check_constraint (#120) --------- Signed-off-by: Bogdan Kyryliuk <b.kyryliuk@gmail.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Remove unused import for SQLAlchemy 2 compatibility (#128) Signed-off-by: William Gentry <william.barr.gentry@gmail.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.8.0 (#178) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix typo in python README quick start example (#186) --------- Co-authored-by: Jesse <jesse.whitehouse@databricks.com> * Configure autospec for mocked Client objects (#188) Resolves #187 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Use urllib3 for retries (#182) Behaviour is gated behind `enable_v3_retries` config. This will be removed and become the default behaviour in a subsequent release. Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.9.0 (#189) * Add note to changelog about using cloud_fetch Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Explicitly add urllib3 dependency (#191) Signed-off-by: Jacobus Herman <jacobus.herman@otrium.com> Co-authored-by: Jesse <jesse.whitehouse@databricks.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump to 2.9.1 (#195) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Make backwards compatible with urllib3~=1.0 (#197) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Convenience improvements to v3 retry logic (#199) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.9.2 (#201) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Github Actions Fix: poetry install fails for python 3.7 tests (#208) snok/install-poetry@v1 installs the latest version of Poetry The latest version of poetry released on 20 August 2023 (four days ago as of this commit) which drops support for Python 3.7, causing our github action to fail. Until we complete #207 we need to conditionally install the last version of poetry that supports Python 3.7 (poetry==1.5.1) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Make backwards compatible with urllib3~=1.0 [Follow up #197] (#206) * Make retry policy backwards compatible with urllib3~=1.0.0 We already implement the equivalent of backoff_max so the behaviour will be the same for urllib3==1.x and urllib3==2.x We do not implement backoff jitter so the behaviour for urllib3==1.x will NOT include backoff jitter whereas urllib3==2.x WILL include jitter. --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump version to 2.9.3 (#209) --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Add note to sqlalchemy example: IDENTITY isn't supported yet (#212) ES-842237 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-1029] Updated thrift compiler version (#216) * Updated thrift definitions Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Tried with a different thrift installation Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Reverted TCLI to previous Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Reverted to older thrift Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Updated version again Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Upgraded thrift Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Final commit Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> --------- Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * [PECO-1055] Updated thrift defs to allow Tsparkparameters (#220) Updated thrift defs to most recent versions * Update changelog to indicate that 2.9.1 and 2.9.2 have been yanked. (#222) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix changelog typo: _enable_v3_retries (#225) Closes #219 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Introduce SQLAlchemy reusable dialog tests (#125) Signed-off-by: Jim Fulton <jim.fulton@unsupervised.com> Co-Authored-By: Jesse Whitehouse <jesse.whitehouse@databricks.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-1026] Add Parameterized Query support to Python (#217) * Initial commit Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Added tsparkparam handling Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Added basic test Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Addressed comments Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Addressed missed comments Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Resolved comments --------- Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Parameterized queries: Add e2e tests for inference (#227) * [PECO-1109] Parameterized Query: add suport for inferring decimal types (#228) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: reorganise dialect files into a single directory (#231) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-1083] Updated thrift files and added check for protocol version (#229) * Updated thrift files and added check for protocol version Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Made error message more clear Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Changed name of fn Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Ran linter Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Update src/databricks/sql/client.py Co-authored-by: Jesse <jwhitehouse@airpost.net> --------- Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> Co-authored-by: Jesse <jwhitehouse@airpost.net> * [PECO-840] Port staging ingestion behaviour to new UC Volumes (#235) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Query parameters: implement support for binding NoneType parameters (#233) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: Bump dependency version and update e2e tests for existing behaviour (#236) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Revert "[PECO-1083] Updated thrift files and added check for protocol version" (#237) Reverts #229 as it causes all of our e2e tests to fail on some versions of DBR. We'll reimplement the protocol version check in a follow-up. This reverts commit 241e934a96737d506c2a1f77c7012e1ab8de967b. * SQLAlchemy 2: add type compilation for all CamelCase types (#238) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: add type compilation for uppercase types (#240) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: Stop skipping all type tests (#242) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-1134] v3 Retries: allow users to bound the number of redirects to follow (#244) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Parameters: Add type inference for BIGINT and TINYINT types (#246) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: Stop skipping some non-type tests (#247) * Stop skipping TableDDLTest and permanent skip HasIndexTest We're now in the territory of features that aren't required for sqla2 compat as of pysql==3.0.0 but we may consider adding this in the future. In this case, table comment reflection needs to be manually implemented. Index reflection would require hooking into the compiler to reflect the partition strategy. test_suite.py::HasIndexTest_databricks+databricks::test_has_index[dialect] SKIPPED (Databricks does not support indexes.) test_suite.py::HasIndexTest_databricks+databricks::test_has_index[inspector] SKIPPED (Databricks does not support indexes.) test_suite.py::HasIndexTest_databricks+databricks::test_has_index_schema[dialect] SKIPPED (Databricks does not support indexes.) test_suite.py::HasIndexTest_databricks+databricks::test_has_index_schema[inspector] SKIPPED (Databricks does not support indexes.) test_suite.py::TableDDLTest_databricks+databricks::test_add_table_comment SKIPPED (Comment reflection is possible but not implemented in this dialect.) test_suite.py::TableDDLTest_databricks+databricks::test_create_index_if_not_exists SKIPPED (Databricks does not support indexes.) test_suite.py::TableDDLTest_databricks+databricks::test_create_table PASSED test_suite.py::TableDDLTest_databricks+databricks::test_create_table_if_not_exists PASSED test_suite.py::TableDDLTest_databricks+databricks::test_create_table_schema PASSED test_suite.py::TableDDLTest_databricks+databricks::test_drop_index_if_exists SKIPPED (Databricks does not support indexes.) test_suite.py::TableDDLTest_databricks+databricks::test_drop_table PASSED test_suite.py::TableDDLTest_databricks+databricks::test_drop_table_comment SKIPPED (Comment reflection is possible but not implemented in this dialect.) test_suite.py::TableDDLTest_databricks+databricks::test_drop_table_if_exists PASSED test_suite.py::TableDDLTest_databricks+databricks::test_underscore_names PASSED Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Permanently skip QuotedNameArgumentTest with comments The fixes to DESCRIBE TABLE and visit_xxx were necessary to get to the point where I could even determine that these tests wouldn't pass. But those changes are not currently tested in the dialect. If, in the course of reviewing the remaining tests in the compliance suite, I find that these visit_xxxx methods are not tested anywhere else then we should extend test_suite.py with our own tests to confirm the behaviour for ourselves. Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Move files from base.py to _ddl.py The presence of this pytest.ini file is _required_ to establish pytest's root_path https://docs.pytest.org/en/7.1.x/reference/customize.html#finding-the-rootdir Without it, the custom pytest plugin from SQLAlchemy can't read the contents of setup.cfg which makes none of the tests runnable. Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Emit a warning for certain constructs Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Stop skipping RowFetchTest Date type work fixed this test failure Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Revise infer_types logic to never infer a TINYINT This allows these SQLAlchemy tests to pass: test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_bound_limit PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_bound_limit_offset PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_expr_limit_simple_offset PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit_expr_offset PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit_offset[cases0] PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit_offset[cases1] PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit_offset[cases2] PASSED This partially reverts the change introduced in #246 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Stop skipping FetchLimitOffsetTest I implemented our custom DatabricksStatementCompiler so we can override the default rendering of unbounded LIMIT clauses from `LIMIT -1` to `LIMIT ALL` We also explicitly skip the FETCH clause tests since Databricks doesn't support this syntax. Blacked all source code here too. Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Stop skipping FutureTableDDLTest Add meaningful skip markers for table comment reflection and indexes Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Stop skipping Identity column tests This closes https://github.com/databricks/databricks-sql-python/issues/175 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Stop skipping HasTableTest Adding the @reflection.cache decorator to has_table is necessary to pass test_has_table_cache Caching calls to has_table improves the efficiency of the connector Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Permanently skip LongNameBlowoutTest Databricks constraint names are limited to 255 characters Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Stop skipping ExceptionTest Black test_suite.py Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Permanently skip LastrowidTest Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Implement PRIMARY KEY and FOREIGN KEY reflection and enable tests Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Skip all IdentityColumnTest tests Turns out that none of these can pass for the same reason that the first two seemed un-runnable in db6f52bb329f3f43a9215b5cd46b03c3459a302a Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: implement and refactor schema reflection methods (#249) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Add GovCloud domain into AWS domains (#252) Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * SQLAlchemy 2: Refactor __init__.py into base.py (#250) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: Finish implementing all of ComponentReflectionTest (#251) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: Finish marking all tests in the suite (#253) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: Finish organising compliance test suite (#256) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: Fix failing mypy checks from development (#257) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Enable cloud fetch by default (#258) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-1137] Reintroduce protocol checking to Python test fw (#248) * Put in some unit tests, will add e2e Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Added e2e test Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Linted Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * re-bumped thrift files Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Changed structure to store protocol version as feature of connection Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Fixed parameters test Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Fixed comments Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Update src/databricks/sql/client.py Co-authored-by: Jesse <jwhitehouse@airpost.net> Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Fixed comments Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> * Removed extra indent Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> --------- Signed-off-by: nithinkdb <nithin.krishnamurthi@databricks.com> Co-authored-by: Jesse <jwhitehouse@airpost.net> * sqla2 clean-up: make sqlalchemy optional and don't mangle the user-agent (#264) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy 2: Add support for TINYINT (#265) Closes #123 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Add OAuth M2M example (#266) * Add OAuth M2M example Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Native Parameters: reintroduce INLINE approach with tests (#267) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Document behaviour of executemany (#213) Signed-off-by: Martin Rueckl <enigma@nbubu.de> * SQLAlchemy 2: Expose TIMESTAMP and TIMESTAMP_NTZ types to users (#268) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Drop Python 3.7 as a supported version (#270) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> (cherry picked from commit 8d85fa8b33a70331141c0c6556196f641d1b8ed5) * GH Workflows: remove Python 3.7 from the matrix for _all_ workflows (#274) Remove Python 3.7 from the matrix for _all_ workflows This was missed in #270 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Add README and updated example for SQLAlchemy usage (#273) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Rewrite native parameter implementation with docs and tests (#281) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Enable v3 retries by default (#282) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * security: bump pyarrow dependency to 14.0.1 (#284) pyarrow is currently compatible with Python 3.8 → Python 3.11 I also removed specifiers for when Python is 3.7 since this no longer applies. Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump package version to 3.0.0 (#285) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix docstring about default parameter approach (#287) * [PECO-1286] Add tests for complex types in query results (#293) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * sqlalchemy: fix deprecation warning for dbapi classmethod (#294) Rename `dbapi` classmethod to `import_dbapi` as required by SQLAlchemy 2 Closes #289 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-1297] sqlalchemy: fix: can't read columns for tables containing a TIMESTAMP_NTZ column (#296) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Prepared 3.0.1 release (#297) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Make contents of `__init__.py` equal across projects (#304) --------- Signed-off-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix URI construction in ThriftBackend (#303) Signed-off-by: Jessica <12jessicasmith34@gmail.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [sqlalchemy] Add table and column comment support (#329) Signed-off-by: Christophe Bornet <cbornet@hotmail.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Pin pandas and urllib3 versions to fix runtime issues in dbt-databricks (#330) Signed-off-by: Ben Cassell <ben.cassell@databricks.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * SQLAlchemy: TINYINT types didn't reflect properly (#315) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-1435] Restore `tests.py` to the test suite (#331) --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Bump to version 3.0.2 (#335) Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Update some outdated OAuth comments (#339) Signed-off-by: Jacky Hu <jacky.hu@databricks.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Redact the URL query parameters from the urllib3.connectionpool logs (#341) * Redact the URL query parameters from the urllib3.connectionpool logs Signed-off-by: Mubashir Kazia <mubashir.kazia@databricks.com> * Fix code formatting Signed-off-by: Mubashir Kazia <mubashir.kazia@databricks.com> * Add str check for the log record message arg dict values Signed-off-by: Mubashir Kazia <mubashir.kazia@databricks.com> --------- Signed-off-by: Mubashir Kazia <mubashir.kazia@databricks.com> * Bump to version 3.0.3 (#344) Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * [PECO-1411] Support Databricks OAuth on GCP (#338) * [PECO-1411] Support OAuth InHouse on GCP Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Update changelog Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> --------- Signed-off-by: Jacky Hu <jacky.hu@databricks.com> Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Co-authored-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * [PECO-1414] Support Databricks native OAuth in Azure (#351) * [PECO-1414] Support Databricks InHouse OAuth in Azure Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Prep for Test Automation (#352) Getting ready for test automation Signed-off-by: Ben Cassell <ben.cassell@databricks.com> * Update code owners (#345) * update owners Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> * update owners Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> * update owners Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> --------- Signed-off-by: yunbodeng-db <104732431+yunbodeng-db@users.noreply.github.com> * Reverting retry behavior on 429s/503s to how it worked in 2.9.3 (#349) Signed-off-by: Ben Cassell <ben.cassell@databricks.com> * Bump to version 3.1.0 (#358) Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * [PECO-1440] Expose current query id on cursor object (#364) * [PECO-1440] Expose current query id on cursor object Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Clear `active_op_handle` when closing the cursor Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Add a default for retry after (#371) * Add a default for retry after Signed-off-by: Ben Cassell <ben.cassell@databricks.com> * Applied black formatter Signed-off-by: Ben Cassell <ben.cassell@databricks.com> * Fix boolean literals (#357) Set supports_native_boolean to True Signed-off-by: Alex Holyoke <alexander.holyoke@growthloop.com> * Don't retry network requests that fail with code 403 (#373) * Don't retry requests that fail with 404 Signed-off-by: Jesse Whitehouse <jesse@whitehouse.dev> * Fix lint error Signed-off-by: Jesse Whitehouse <jesse@whitehouse.dev> --------- Signed-off-by: Jesse Whitehouse <jesse@whitehouse.dev> * Bump to 3.1.1 (#374) * bump to 3.1.1 Signed-off-by: Ben Cassell <ben.cassell@databricks.com> * Fix cookie setting (#379) * fix cookie setting Signed-off-by: Ben Cassell <ben.cassell@databricks.com> * Removing cookie code Signed-off-by: Ben Cassell <ben.cassell@databricks.com> --------- Signed-off-by: Ben Cassell <ben.cassell@databricks.com> * Fixing a couple type problems: how I would address most of #381 (#382) * Create py.typed Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * add -> Connection annotation Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * massage the code to appease the particular version of the project's mypy deps Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * fix circular import problem Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> --------- Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * fix the return types of the classes' __enter__ functions (#384) fix the return types of the classes' __enter__ functions so that the type information is preserved in context managers eg with-as blocks Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * Add Kravets Levko to codeowners (#386) Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Prepare for 3.1.2 (#387) Signed-off-by: Ben Cassell <ben.cassell@databricks.com> * Update the proxy authentication (#354) changed authentication for proxy * Fix failing tests (#392) Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Relax `pyarrow` pin (#389) * Relax `pyarrow` pin Signed-off-by: Dave Hirschfeld <dave.hirschfeld@gmail.com> * Allow `pyarrow` 16 Signed-off-by: Dave Hirschfeld <dave.hirschfeld@gmail.com> * Update `poetry.lock` Signed-off-by: Dave Hirschfeld <dave.hirschfeld@gmail.com> --------- Signed-off-by: Dave Hirschfeld <dave.hirschfeld@gmail.com> * Fix log error in oauth.py (#269) * Duplicate of applicable change from #93 Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Update changelog Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> * Fix after merge Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> Signed-off-by: Levko Kravets <levko.ne@gmail.com> Co-authored-by: Levko Kravets <levko.ne@gmail.com> * Enable `delta.feature.allowColumnDefaults` for all tables (#343) * Enable `delta.feature.allowColumnDefaults` for all tables * Code style Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Levko Kravets <levko.ne@gmail.com> Co-authored-by: Levko Kravets <levko.ne@gmail.com> * Fix SQLAlchemy tests (#393) Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Add more debug logging for CloudFetch (#395) Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Update Thrift package (#397) Signed-off-by: Milan Lukac <milan@lukac.online> * Prepare release 3.2.0 (#396) * Prepare release 3.2.0 Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Update changelog Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Levko Kravets <levko.ne@gmail.com> * move py.typed to correct places (#403) * move py.typed to correct places https://peps.python.org/pep-0561/ says 'For namespace packages (see PEP 420), the py.typed file should be in the submodules of the namespace, to avoid conflicts and for clarity.'. Previously, when I added the py.typed file to this project, https://github.com/databricks/databricks-sql-python/pull/382 , I was unaware this was a namespace package (although, curiously, it seems I had done it right initially and then changed to the wrong way). As PEP 561 warns us, this does create conflicts; other libraries in the databricks namespace package (such as, in my case, databricks-vectorsearch) are then treated as though they are typed, which they are not. This commit moves the py.typed file to the correct places, the submodule folders, fixing that problem. Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * change target of mypy to src/databricks instead of src. I think this might fix the CI code-quality checks failure, but unfortunately I can't replicate that failure locally and the error message is unhelpful Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * Possible workaround for bad error message 'error: --install-types failed (no mypy cache directory)'; see https://github.com/python/mypy/issues/10768#issuecomment-2178450153 Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * fix invalid yaml syntax Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * Best fix (#3) Fixes the problem by cding and supplying a flag to mypy (that mypy needs this flag is seemingly fixed/changed in later versions of mypy; but that's another pr altogether...). Also fixes a type error that was somehow in the arguments of the program (?!) (I guess this is because you guys are still using implicit optional) --------- Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * return the old result_links default (#5) Return the old result_links default, make the type optional, & I'm pretty sure the original problem is that add_file_links can't take a None, so these statements should be in the body of the if-statement that ensures it is not None Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * Update src/databricks/sql/utils.py "self.download_manager is unconditionally used later, so must be created. Looks this part of code is totally not covered with tests 🤔" Co-authored-by: Levko Kravets <levko.ne@gmail.com> Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> --------- Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> Co-authored-by: Levko Kravets <levko.ne@gmail.com> * Upgrade mypy (#406) * Upgrade mypy This commit removes the flag (and cd step) from https://github.com/databricks/databricks-sql-python/commit/f53aa37a34dc37026d430e71b5e0d1b871bc5ac1 which we added to get mypy to treat namespaces correctly. This was apparently a bug in mypy, or behavior they decided to change. To get the new behavior, we must upgrade mypy. (This also allows us to remove a couple `# type: ignore` comment that are no longer needed.) This commit runs changes the version of mypy and runs `poetry lock`. It also conforms the whitespace of files in this project to the expectations of various tools and standard (namely: removing trailing whitespace as expected by git and enforcing the existence of one and only one newline at the end of a file as expected by unix and github.) It also uses https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade codebase due to a change in mypy behavior. For a similar reason, it also fixes a new type (or otherwise) errors: * "Return type 'Retry' of 'new' incompatible with return type 'DatabricksRetryPolicy' in supertype 'Retry'" * databricks/sql/auth/retry.py:225: error: object has no attribute update [attr-defined] * /test_param_escaper.py:31: DeprecationWarning: invalid escape sequence \) [as it happens, I think it was also wrong for the string not to be raw, because I'm pretty sure it wants all of its backslashed single-quotes to appear literally with the backslashes, which wasn't happening until now] * ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject [this is like a numpy version thing, which I fixed by being stricter about numpy version] --------- Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * Incorporate suggestion. I decided the most expedient way of dealing with this type error was just adding the type ignore comment back in, but with a `[attr-defined]` specifier this time. I mean, otherwise I would have to restructure the code or figure out the proper types for a TypedDict for the dict and I don't think that's worth it at the moment. Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> --------- Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com> * Do not retry failing requests with status code 401 (#408) - Raises NonRecoverableNetworkError when request results in 401 status code Signed-off-by: Tor Hødnebø <thodnebo@gmail.com> Signed-off-by: Tor Hødnebø <tor.hodnebo@gjensidige.no> * [PECO-1715] Remove username/password (BasicAuth) auth option (#409) Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * [PECO-1751] Refactor CloudFetch downloader: handle files sequentially (#405) * [PECO-1751] Refactor CloudFetch downloader: handle files sequentially; utilize Futures Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Retry failed CloudFetch downloads Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Update tests Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Fix CloudFetch retry policy to be compatible with all `urllib3` versions we support (#412) Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Disable SSL verification for CloudFetch links (#414) * Disable SSL verification for CloudFetch links Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Use existing `_tls_no_verify` option in CloudFetch downloader Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Update tests Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Prepare relese 3.3.0 (#415) * Prepare relese 3.3.0 Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Remove @arikfr from CODEOWNERS Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Fix pandas 2.2.2 support (#416) * Support pandas 2.2.2 See release note numpy 2.2.2: https://pandas.pydata.org/docs/dev/whatsnew/v2.2.0.html#to-numpy-for-numpy-nullable-and-arrow-types-converts-to-suitable-numpy-dtype * Allow pandas 2.2.2 in pyproject.toml * Update poetry.lock, poetry lock --no-update * Code style Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Levko Kravets <levko.ne@gmail.com> Co-authored-by: Levko Kravets <levko.ne@gmail.com> * [PECO-1801] Make OAuth as the default authenticator if no authentication setting is provided (#419) * [PECO-1801] Make OAuth as the default authenticator if no authentication setting is provided Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * [PECO-1857] Use SSL options with HTTPS connection pool (#425) * [PECO-1857] Use SSL options with HTTPS connection pool Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Some cleanup Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Resolve circular dependencies Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Update existing tests Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Fix MyPy issues Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Fix `_tls_no_verify` handling Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Add tests Signed-off-by: Levko Kravets <levko.ne@gmail.com> --------- Signed-off-by: Levko Kravets <levko.ne@gmail.com> * Prepare release v3.4.0 (#430) Prepare release 3.4.0 Signed-off-by: Levko Kravets <levko.ne@gmail.com> * [PECO-1926] Create a non pyarrow flow to handle small results for the column set (#440) * Implemented the columnar flow for non arrow users * Minor fixes * Introduced the Column Table structure * Added test for the new column table * Minor fix * Removed unnecessory fikes * [PECO-1961] On non-retryable error, ensure PySQL includes useful information in error (#447) * added error info on non-retryable error * Reformatted all the files using black (#448) Reformatted the files using black * Prepare release v3.5.0 (#457) Prepare release 3.5.0 Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * [PECO-2051] Add custom auth headers into cloud fetch request (#460) Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Prepare release 3.6.0 (#461) Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * [ PECO - 1768 ] PySQL: adjust HTTP retry logic to align with Go and Nodejs drivers (#467) * Added the exponential backoff code * Added the exponential backoff algorithm and refractored the code * Added jitter and added unit tests * Reformatted * Fixed the test_retry_exponential_backoff integration test * [ PECO-2065 ] Create the async execution flow for the PySQL Connector (#463) * Built the basic flow for the async pipeline - testing is remaining * Implemented the flow for the get_execution_result, but the problem of invalid operation handle still persists * Missed adding some files in previous commit * Working prototype of execute_async, get_query_state and get_execution_result * Added integration tests for execute_async * add docs for functions * Refractored the async code * Fixed java doc * Reformatted * Fix for check_types github action failing (#472) Fixed the chekc_types failing * Remove upper caps on dependencies (#452) * Remove upper caps on numpy and pyarrow versions * Updated the doc to specify native parameters in PUT operation is not supported from >=3.x connector (#477) Added doc update * Incorrect rows in inline fetch result (#479) * Raised error when incorrect Row offset it returned * Changed error type * grammar fix * Added unit tests and modified the code * Updated error message * Updated the non retying to only inline case * Updated fix * Changed the flow * Minor update * Updated the retryable condition * Minor test fix * Added extra space * Bumped up to version 3.7.0 (#482) * bumped up version * Updated to version 3.7.0 * Grammar fix * Minor fix * PySQL Connector split into connector and sqlalchemy (#444) * Modified the gitignore file to not have .idea file * [PECO-1803] Splitting the PySql connector into the core and the non core part (#417) * Implemented ColumnQueue to test the fetchall without pyarrow Removed token removed token * order of fields in row corrected * Changed the folder structure and tested the basic setup to work * Refractored the code to make connector to work * Basic Setup of connector, core and sqlalchemy is working * Basic integration of core, connect and sqlalchemy is working * Setup working dynamic change from ColumnQueue to ArrowQueue * Refractored the test code and moved to respective folders * Added the unit test for column_queue Fixed __version__ Fix * venv_main added to git ignore * Added code for merging columnar table * Merging code for columnar * Fixed the retry_close sesssion test issue with logging * Fixed the databricks_sqlalchemy tests and introduced pytest.ini for the sqla_testing * Added pyarrow_test mark on pytest * Fixed databricks.sqlalchemy to databricks_sqlalchemy imports * Added poetry.lock * Added dist folder * Changed the pyproject.toml * Minor Fix * Added the pyarrow skip tag on unit tests and tested their working * Fixed the Decimal and timestamp conversion issue in non arrow pipeline * Removed not required files and reformatted * Fixed test_retry error * Changed the folder structure to src / databricks * Removed the columnar non arrow flow to another PR * Moved the README to the root * removed columnQueue instance * Revmoved databricks_sqlalchemy dependency in core * Changed the pysql_supports_arrow predicate, introduced changes in the pyproject.toml * Ran the black formatter with the original version * Extra .py removed from all the __init__.py files names * Undo formatting check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * BIG UPDATE * Refeactor code * Refractor * Fixed versioning * Minor refractoring * Minor refractoring * Changed the folder structure such that sqlalchemy has not reference here * Fixed README.md and CONTRIBUTING.md * Added manual publish * On push trigger added * Manually setting the publish step * Changed versioning in pyproject.toml * Bumped up the version to 4.0.0.b3 and also changed the structure to have pyarrow as optional * Removed the sqlalchemy tests from integration.yml file * [PECO-1803] Print warning message if pyarrow is not installed (#468) Print warning message if pyarrow is not installed Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * [PECO-1803] Remove sqlalchemy and update README.md (#469) Remove sqlalchemy and update README.md Signed-off-by: Jacky Hu <jacky.hu@databricks.com> * Removed all sqlalchemy related stuff * generated the lock file * Fixed failing tests * removed poetry.lock * Updated the lock file * Fixed poetry numpy 2.2.2 issue * Workflow fixes --------- Signed-off-by: Jacky Hu <jacky.hu@databricks.com> Co-authored-by: Jacky Hu <jacky.hu@databricks.com> * Removed CI CD for python3.8 (#490) * Removed python3.8 support * Minor fix * Added CI CD upto python 3.12 (#491) Support for Py till 3.12 * Merging changes from v3.7.1 release (#488) * Increased the number of retry attempts allowed (#486) Updated the number of attempts allowed * bump version to 3.7.1 (#487) bumped up version * Refractore * Minor change * Bumped up to version 4.0.0 (#493) bumped up the version * Updated action's version (#455) Updated actions version. Signed-off-by: Arata Hatori <newwingbird@gmail.com> * Support Python 3.13 and update deps (#510) * Remove upper caps on dependencies (#452) * Remove upper caps on numpy and pyarrow versions Signed-off-by: David Black <dblack@atlassian.com> * Added CI CD upto python 3.13 Signed-off-by: David Black <dblack@atlassian.com> * Specify pandas 2.2.3 as the lower bound for python 3.13 Signed-off-by: David Black <dblack@atlassian.com> * Specify pyarrow 18.0.0 as the lower bound for python 3.13 Signed-off-by: David Black <dblack@atlassian.com> * Move `numpy` to dev dependencies Signed-off-by: Dave Hirschfeld <dave.hirschfeld@gmail.com> * Updated lockfile Signed-off-by: Dave Hirschfeld <dave.hirschfeld@gmail.com> --------- Signed-off-by: David Black <dblack@atlassian.com> Signed-off-by: Dave Hirschfeld <dave.hirschfeld@gmail.com> Co-authored-by: David Black <dblack@atlassian.com> * Improve debugging + fix PR review template (#514) * Improve debugging + add PR review template * case sensitivity of PR template * Forward porting all changes into 4.x.x. uptil v3.7.3 (#529) * Base changes * Black formatter * Cache version fix * Added the changed test_retry.py file * retry_test_mixins changes * Updated the CODEOWNERS (#531) Updated the codeowners * Add version check for urllib3 in backoff calculation (#526) Signed-off-by: Shivam Raj <shivam.raj@databricks.com> * [ES-1372353] make user_agent_header part of public API (#530) * make user_agent_header part of public API * removed user_agent_entry from list of internal params * add backward compatibility * Updates runner used to run DCO check to use databricks-protected-runner (#521) * commit 1 Signed-off-by: Madhav Sainanee <madhav.sainanee@databricks.com> * commit 1 Signed-off-by: Madhav Sainanee <madhav.sainanee@databricks.com> * updates runner for dco check Signed-off-by: Madhav Sainanee <madhav.sainanee@databricks.com> * removes contributing file changes Signed-off-by: Madhav Sainanee <madhav.sainanee@databricks.com> --------- Signed-off-by: Madhav Sainanee <madhav.sainanee@databricks.com> * Support multiple timestamp formats in non arrow flow (#533) * Added check for 2 formats * Wrote unit tests * Added more supporting formats * Added the T format datetime * Added more timestamp formats * Added python-dateutil library * prepare release for v4.0.1 (#534) Signed-off-by: Shivam Raj <shivam.raj@databricks.com> * Relaxed bound for python-dateutil (#538) Changed bound for python-datetutil * Bumped up the version for 4.0.2 (#539) * Added example for async execute query (#537) Added examples and fixed the async execute not working without pyarrow * Added urllib3 version check (#547) * Added version check * Removed packaging * Bump version to 4.0.3 (#549) Updated the version to 4.0.3 * Cleanup fields as they might be deprecated/removed/change in the future (#553) * Clean thrift files Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com> * Refactor decimal conversion in PyArrow tables to use direct casting (#544) This PR replaces the previous implementation of convert_decimals_in_arrow_table() with a more efficient approach that uses PyArrow's native casting operation instead of going through pandas conversion and array creation. - Remove conversion to pandas DataFrame via to_pandas() and apply() methods - Remove intermediate steps of creating array from decimal column and setting it back - Replace with direct type casting using PyArrow's cast() method - Build a new table with transformed columns rather than modifying the original table - Create a new schema based on the modified fields The new approach is more performant by avoiding pandas conversion overhead. The table below highlights substantial performance improvements when retrieving all rows from a table containing decimal columns, particularly when compression is disabled. Even greater gains were observed with compression enabled—showing approximately an 84% improvement (6 seconds compared to 39 seconds). Benchmarking was performed against e2-dogfood, with the client located in the us-west-2 region. ![image](https://github.com/user-attachments/assets/5407b651-8ab6-4c13-b525-cf912f503ba0) Signed-off-by: Jayant Singh <jayant.singh@databricks.com> * [PECOBLR-361] convert column table to arrow if arrow present (#551) * Update CODEOWNERS (#562) new codeowners * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Update github actions run conditions (#569) More conditions to run github actions * Added classes required for telemetry (#572) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added classes required for telemetry Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed TelemetryHelper Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * [PECOBLR-361] convert column table to arrow if arrow present (#551) Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Update github actions run conditions (#569) More conditions to run github actions Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Added classes required for telemetry Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed example Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * chan…

* add close() for Queue, add ResultSet invocation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue closure to finally: block to ensure client-side cleanup regardless of server side state Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add unit test assertions to ensure Queue closure Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move results closure to try block Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

* Separate Session related functionality from Connection class (databricks#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. databricks@c676f9b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b19. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (databricks#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (databricks#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (databricks#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec5, reversing changes made to 7192f11. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb8381. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (databricks#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (databricks#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use lazy logging Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * replace getters with property tag Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove repetition from Session.__init__ Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * mention that if catalog / schema name is None, we fetch across all Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * mention fetching across all tables if null table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove lazy import of ThriftResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clarified role of cursor in docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

* Minor fix * Perf update * more * test fix

* send telemetry to unauth endpoint in case of connection/auth errors Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added unit test for send_connection_error_telemetry Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * retry errors Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Add functionality for export of latency logs via telemetry (databricks#608) * added functionality for export of failure logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed logger.error to logger.debug in exc.py Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Fix telemetry loss during Python shutdown Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * unit tests for export_failure_log Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * try-catch blocks to make telemetry failures non-blocking for connector operations Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed redundant try/catch blocks, added try/catch block to initialize and get telemetry client Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * skip null fields in telemetry request Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed dup import, renamed func, changed a filter_null_values to lamda Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed unnecassary class variable and a redundant try/except block Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * public functions defined at interface level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed export_event and flush to private functions Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed connection_uuid to thread local in thrift backend Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * made errors more specific Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * revert change to connection_uuid Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * reverting change in close in telemetry client Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * JsonSerializableMixin Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * isdataclass check in JsonSerializableMixin Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * convert TelemetryClientFactory to module-level functions, replace NoopTelemetryClient class with NOOP_TELEMETRY_CLIENT singleton, updated tests accordingly Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * renamed connection_uuid as session_id_hex Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added NotImplementedError to abstract class, added unit tests Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added PEP-249 link, changed NoopTelemetryClient implementation Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed unused import Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * made telemetry client close a module-level function Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * unit tests verbose Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * debug logs in unit tests Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * debug logs in unit tests Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed ABC from mixin, added try/catch block around executor shutdown Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * checking stuff Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * finding out * finding out more * more more finding out more nice * locks are useless anyways * haha * normal * := looks like walrus horizontally * one more * walrus again * old stuff without walrus seems to fail * manually do the walrussing * change 3.13t, v2 Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting, added walrus Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed walrus, removed test before stalling test Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed order of stalling test Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed debugging, added TelemetryClientFactory Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * remove more debugging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * latency logs funcitionality Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed type of return value in get_session_id_hex() in thrift backend Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * debug on TelemetryClientFactory lock Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * type notation for _waiters Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * called connection.close() in test_arraysize_buffer_size_passthrough Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * run all unit tests Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * more debugging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed the connection.close() from that test, put debug statement before and after TelemetryClientFactory lock Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * more debug Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * more more more Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * why Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * whywhy Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * thread name Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added teardown to all tests except finalizer test (gc collect) Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added the get_attribute functions to the classes Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed tearDown, added connection.close() to first test Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * finally Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * remove debugging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added test for export_latency_log, made mock of thrift backend with retry policy Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added multi threaded tests Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added TelemetryExtractor, removed multithreaded tests Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixes in test Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fix in telemetry extractor Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added doc strings to latency_logger, abstracted export_telemetry_log Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * statement type, unit test fix Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * unit test fix Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * statement type changes Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * test_fetches fix Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added mocks to resolve the errors caused by log_latency decorator in tests Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed function in test_fetches cuz it is only used once Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * added _safe_call which returns None in case of errors in the get functions Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed the changes in test_client and test_fetches Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed the changes in test_fetches Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * test_telemetry Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed test Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * MaxRetryDurationError Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * main changes Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * formatting Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * import json Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * without the max retry errors Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * unauth telemetry client Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * remove duplicate code setting telemetry_enabled Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed unused errors Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * merge with main changes Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * test Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * without try/catch block Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * - Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * error log for auth provider, ThriftDatabricksClient Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * error log for session.open Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * retry tests fix Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * test connection failure log Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * check types fix Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * test Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * rephrase import Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>

* Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add newline at EOF Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce `SeaDatabricksClient` (Session Implementation) (#582) * [squashed from prev branch] introduce sea client with session open and close functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidental changes to workflows (merge artifacts) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass test_input to get_protocol_version instead of session_id to maintain previous API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black + line gaps after multi-line pydocs) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use factory for backend instantiation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce models for requests and responses Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove http client and test script to prevent diff from showing up post http-client merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add newline at EOF Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * CustomHttpClient -> SeaHttpClient Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment in backend client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * regex for warehouse_id instead of .split, remove excess imports and behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [nit] reduce nested code Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line gap after multi-line pydoc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move sea backend and models into separate sea/ dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move http client into separate sea/ dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change commands to include ones in docs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add link to sql-ref-parameters for session-confs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add client side filtering for session confs, add note on warehouses over endoints Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test unimplemented methods and max_download_threads prop Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Normalise Execution Response (clean backend interfaces) (#587) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * maintain log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * frmatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better mocked backend naming Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move back to old table types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce models for `SeaDatabricksClient` (#595) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce preliminary SEA Result Set (#588) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * maintain log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * frmatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better mocked backend naming Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move back to old table types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align SeaResultSet with new structure Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct sea res set tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type errors with arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * spaces after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate queue init (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove newlines) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 anyway Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "remove un-necessary changes" This reverts commit a70a6cee277db44d6951604e890f91cae9f92f32. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * b"" -> None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid ExecuteResponse import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement SeaDatabricksClient (Complete Execution Spec) (#590) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add more unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * streamline unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for manifest in _extract_description Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary column skipping Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove parsing in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: convert sea statement id to CommandId type Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make polling interval a separate constant Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align state checking with Thrift implementation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests according to changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add unit tests for added methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add spec to description extraction docstring, add strong typing to params Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for backend parameters arg Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use lazy logging Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * replace getters with property tag Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate test, correct active_command_id attribute Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * SeaDatabricksClient: Add Metadata Commands (#593) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add metadata commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add metadata command unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'sea-migration' into exec-models-sea" This reverts commit 8bd12d829ea13abf8fc1507fff8cb21751001c67, reversing changes made to 030edf8df3db487b7af8d910ee51240d1339229e. * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment…

* telemetry retry Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * shifted tests to unit test, removed unused imports Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * tests Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>

* Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add newline at EOF Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce `SeaDatabricksClient` (Session Implementation) (#582) * [squashed from prev branch] introduce sea client with session open and close functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidental changes to workflows (merge artifacts) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass test_input to get_protocol_version instead of session_id to maintain previous API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black + line gaps after multi-line pydocs) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use factory for backend instantiation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce models for requests and responses Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove http client and test script to prevent diff from showing up post http-client merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add newline at EOF Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * CustomHttpClient -> SeaHttpClient Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment in backend client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * regex for warehouse_id instead of .split, remove excess imports and behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [nit] reduce nested code Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line gap after multi-line pydoc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move sea backend and models into separate sea/ dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move http client into separate sea/ dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change commands to include ones in docs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add link to sql-ref-parameters for session-confs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add client side filtering for session confs, add note on warehouses over endoints Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test unimplemented methods and max_download_threads prop Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Normalise Execution Response (clean backend interfaces) (#587) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * maintain log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * frmatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better mocked backend naming Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move back to old table types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce models for `SeaDatabricksClient` (#595) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce preliminary SEA Result Set (#588) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * maintain log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * frmatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better mocked backend naming Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move back to old table types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align SeaResultSet with new structure Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct sea res set tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type errors with arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * spaces after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate queue init (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove newlines) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 anyway Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "remove un-necessary changes" This reverts commit a70a6cee277db44d6951604e890f91cae9f92f32. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * b"" -> None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid ExecuteResponse import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement SeaDatabricksClient (Complete Execution Spec) (#590) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add more unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * streamline unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for manifest in _extract_description Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary column skipping Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove parsing in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: convert sea statement id to CommandId type Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make polling interval a separate constant Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align state checking with Thrift implementation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests according to changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add unit tests for added methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add spec to description extraction docstring, add strong typing to params Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for backend parameters arg Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use lazy logging Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * replace getters with property tag Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate test, correct active_command_id attribute Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * SeaDatabricksClient: Add Metadata Commands (#593) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add metadata commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add metadata command unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'sea-migration' into exec-models-sea" This reverts commit 8bd12d829ea13abf8fc1507fff8cb21751001c67, reversing changes made to 030edf8df3db487b7af8d910ee51240d1339229e. * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comments S…

…bricks#622) configurable telemetry batch size, time based flush Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>

* Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add newline at EOF Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce `SeaDatabricksClient` (Session Implementation) (#582) * [squashed from prev branch] introduce sea client with session open and close functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidental changes to workflows (merge artifacts) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass test_input to get_protocol_version instead of session_id to maintain previous API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black + line gaps after multi-line pydocs) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use factory for backend instantiation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce models for requests and responses Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove http client and test script to prevent diff from showing up post http-client merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add newline at EOF Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * CustomHttpClient -> SeaHttpClient Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment in backend client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * regex for warehouse_id instead of .split, remove excess imports and behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [nit] reduce nested code Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line gap after multi-line pydoc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move sea backend and models into separate sea/ dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move http client into separate sea/ dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change commands to include ones in docs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add link to sql-ref-parameters for session-confs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add client side filtering for session confs, add note on warehouses over endoints Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test unimplemented methods and max_download_threads prop Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Normalise Execution Response (clean backend interfaces) (#587) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * maintain log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * frmatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better mocked backend naming Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move back to old table types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce models for `SeaDatabricksClient` (#595) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce preliminary SEA Result Set (#588) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * maintain log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * frmatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better mocked backend naming Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move back to old table types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align SeaResultSet with new structure Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct sea res set tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type errors with arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * spaces after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate queue init (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove newlines) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 anyway Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "remove un-necessary changes" This reverts commit a70a6cee277db44d6951604e890f91cae9f92f32. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * b"" -> None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid ExecuteResponse import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement SeaDatabricksClient (Complete Execution Spec) (#590) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add more unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * streamline unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for manifest in _extract_description Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary column skipping Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove parsing in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: convert sea statement id to CommandId type Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make polling interval a separate constant Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align state checking with Thrift implementation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests according to changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add unit tests for added methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add spec to description extraction docstring, add strong typing to params Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for backend parameters arg Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use lazy logging Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * replace getters with property tag Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate test, correct active_command_id attribute Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * SeaDatabricksClient: Add Metadata Commands (#593) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add metadata commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add metadata command unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'sea-migration' into exec-models-sea" This reverts commit 8bd12d829ea13abf8fc1507fff8cb21751001c67, reversing changes made to 030edf8df3db487b7af8d910ee51240d1339229e. * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comments …

* e2e test telemetry Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * assert session id, statement id Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * minor changes, added checks on server response Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * finally block Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * removed setup clean up Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * finally in test_complex_types Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>

Implement PEP 249-compliant transaction control with extensions for manual commit/rollback operations. This enables atomic multi-table operations with REPEATABLE_READ isolation semantics. Core API additions: - connection.autocommit property for enabling/disabling auto-commit mode - connection.commit() to commit active transactions - connection.rollback() to rollback active transactions - connection.get_transaction_isolation() returns current isolation level - connection.set_transaction_isolation() validates isolation level - TransactionError exception for transaction-specific failures Implementation details: - Added autocommit state caching in Session with optional server query - Added TRANSACTION_ISOLATION_LEVEL_REPEATABLE_READ constant - All transaction operations include proper error handling and telemetry - Supports fetch_autocommit_from_server connection parameter Testing: - Unit tests covering all transaction methods and error scenarios - e2e integration tests validating transaction behavior including multi-table atomicity, sequential transactions, and isolation semantics Documentation: - Comprehensive TRANSACTIONS.md guide with examples and best practices - Updated README.md with basic usage and reference to detailed docs Requires MST-enabled Databricks SQL warehouse and Delta tables with 'delta.feature.catalogOwned-preview' table property. --------- Signed-off-by: Jayant Singh <jayant.singh@databricks.com>

Bump to version 4.2.0 Signed-off-by: Jayant Singh <jayant.singh@databricks.com>

…abricks#711) Introduces a new `ignore_transactions` configuration parameter (default: True) to control transaction-related behavior in the Connection class. When ignore_transactions=True (default): - commit(): no-op, returns immediately - rollback(): raises NotSupportedError with message "Transactions are not supported on Databricks" - autocommit setter: no-op, returns immediately When ignore_transactions=False: - All transaction methods execute normally Changes: - Added ignore_transactions parameter to Connection.__init__() with default value True - Modified commit(), rollback(), and autocommit setter to check ignore_transactions flag - Updated unit tests to pass ignore_transactions=False when testing transaction functionality - Updated e2e transaction tests to pass ignore_transactions=False - Added three new unit tests to verify ignore_transactions

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

This changes the default value of use_hybrid_disposition from True to False in the SEA backend, disabling hybrid disposition by default.

* Added driver connection params Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * Added model fields for chunk/result latency Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed linting issues Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * lint issue fixing Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * circuit breaker changes using pybreaker Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * Added interface layer top of http client to use circuit rbeaker Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * Added test cases to validate ciruit breaker Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixing broken tests Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed linting issues Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed failing test cases Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed urllib3 issue Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * added more test cases for telemetry Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * simplified CB config Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * poetry lock Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fix minor issues & improvement Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * improved circuit breaker for handling only 429/503 Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * linting issue fixed Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * raise CB only for 429/503 Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fix broken test cases Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed untyped references Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * added more test to verify the changes Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * description changed Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * remove cb congig class to constants Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * removed mocked reponse and use a new exlucded exception in CB Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed broken test Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * added e2e test to verify circuit breaker Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * lower log level for telemetry Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed broken test, removed tests on log assertions Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * modified unit to reduce the noise and follow dry principle Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> --------- Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com>

…ks#715) perf: Optimize telemetry latency logging to reduce overhead Optimizations implemented: 1. Eliminated extractor pattern - replaced wrapper classes with direct attribute access functions, removing object creation overhead 2. Added feature flag early exit - checks cached telemetry_enabled flag to skip heavy work when telemetry is disabled 3. Simplified code structure with early returns for better readability Signed-off-by: Samikshya Chand <samikshya.chand@databricks.com>

* basic e2e test for force telemetry verification Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * Added more integration test scenarios Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * default on telemetry + logs to investigate failing test Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed linting issue Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * added more logs to identify server side flag evaluation Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * remove unused logs Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fix broken test case for default enable telemetry Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * redcude test length and made more reusable code Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * removed telemetry e2e to daily single run Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> --------- Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com>

…databricks#718) * feat: Implement host-level telemetry batching to reduce rate limiting Changes telemetry client architecture from per-session to per-host batching, matching the JDBC driver implementation. This reduces the number of HTTP requests to the telemetry endpoint and prevents rate limiting in test environments. Key changes: - Add _TelemetryClientHolder with reference counting for shared clients - Change TelemetryClientFactory to key clients by host_url instead of session_id - Add getHostUrlSafely() helper for defensive null handling - Update all callers (client.py, exc.py, latency_logger.py) to pass host_url Before: 100 connections to same host = 100 separate TelemetryClients After: 100 connections to same host = 1 shared TelemetryClient (refcount=100) This fixes rate limiting issues seen in e2e tests where 300+ parallel connections were overwhelming the telemetry endpoint with 429 errors. * chore: Change all telemetry logging to DEBUG level Reduces log noise by changing all telemetry-related log statements (info, warning, error) to debug level. Telemetry operations are background tasks and should not clutter logs with operational messages. Changes: - Circuit breaker state changes: info/warning -> debug - Telemetry send failures: error -> debug - All telemetry operations now consistently use debug level * chore: Fix remaining telemetry warning log to debug Changes remaining logger.warning in telemetry_push_client.py to debug level for consistency with other telemetry logging. * fix: Update tests to use host_url instead of session_id_hex - Update circuit breaker test to check logger.debug instead of logger.info - Replace all session_id_hex test parameters with host_url - Apply Black formatting to exc.py and telemetry_client.py This fixes test failures caused by the signature change from session_id_hex to host_url in the Error class and TelemetryClientFactory. * fix: Revert session_id_hex in tests for functions that still use it Only Error classes changed from session_id_hex to host_url. Other classes (TelemetryClient, ResultSetDownloadHandler, etc.) still use session_id_hex. Reverted: - test_telemetry.py: TelemetryClient and initialize_telemetry_client - test_downloader.py: ResultSetDownloadHandler - test_download_manager.py: ResultFileDownloadManager Kept as host_url: - test_client.py: Error class instantiation * fix: Update all Error raises and test calls to use host_url Changes: 1. client.py: Changed all error raises from session_id_hex to host_url - Connection class: session_id_hex=self.get_session_id_hex() -> host_url=self.session.host - Cursor class: session_id_hex=self.connection.get_session_id_hex() -> host_url=self.connection.session.host 2. test_telemetry.py: Updated get_telemetry_client() and close() calls - get_telemetry_client(session_id) -> get_telemetry_client(host_url) - close(session_id) -> close(host_url=host_url) 3. test_telemetry_push_client.py: Changed logger.warning to logger.debug - Updated test assertion to match debug logging level These changes complete the migration from session-level to host-level telemetry client management. * fix: Update thrift_backend.py to use host_url instead of session_id_hex Changes: 1. Added self._host attribute to store server_hostname 2. Updated all error raises to use host_url=self._host 3. Changed method signatures from session_id_hex to host_url: - _check_response_for_error - _hive_schema_to_arrow_schema - _col_to_description - _hive_schema_to_description - _check_direct_results_for_error 4. Updated all method calls to pass self._host instead of self._session_id_hex This completes the migration from session-level to host-level error reporting. * Fix Black formatting by adjusting fmt directive placement Moved the `# fmt: on` directive to the except block level instead of inside the if statement to resolve Black parsing confusion. * Fix telemetry feature flag tests to set mock session host The tests were failing because they called get_telemetry_client("test") but the mock session didn't have .host set, so the telemetry client was registered under a different key (likely None or MagicMock). This caused the factory to return NoopTelemetryClient instead of the expected client. Fixed by setting mock_session_instance.host = "test" in all three tests. * Add teardown_method to clear telemetry factory state between tests Without this cleanup, tests were sharing telemetry clients because they all used the same host key ("test"), causing test pollution. The first test would create an enabled client, and subsequent tests would reuse it even when they expected a disabled client. * Clear feature flag context cache in teardown to fix test pollution The FeatureFlagsContextFactory caches feature flag contexts per session, causing tests to share the same feature flag state. This resulted in the first test creating a context with telemetry enabled, and subsequent tests incorrectly reusing that enabled state even when they expected disabled. * fix: Access actual client from holder in flush worker The flush worker was calling _flush() on _TelemetryClientHolder objects instead of the actual TelemetryClient. Fixed by accessing holder.client before calling _flush(). Fixes AttributeError in e2e tests: '_TelemetryClientHolder' object has no attribute '_flush' * Clear telemetry client cache in e2e test teardown Added _clients.clear() to the teardown fixture to prevent telemetry clients from persisting across e2e tests, which was causing session ID pollution in test_concurrent_queries_sends_telemetry. * Pass session_id parameter to telemetry export methods With host-level telemetry batching, multiple connections share one TelemetryClient. Each client stores session_id_hex from the first connection that created it. This caused all subsequent connections' telemetry events to use the wrong session ID. Changes: - Modified telemetry export method signatures to accept optional session_id - Updated Connection.export_initial_telemetry_log() to pass session_id - Updated latency_logger.py export_latency_log() to pass session_id - Updated Error.__init__() to accept optional session_id_hex and pass it - Updated all error raises in Connection and Cursor to pass session_id_hex 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix Black formatting in telemetry_client.py * Use 'test-host' instead of 'test' for mock host in telemetry tests * Replace test-session-id with test-host in test_client.py * Fix telemetry client lookup to use test-host in tests * Make session_id_hex keyword-only parameter in Error.__init__ --------- Co-authored-by: Claude <noreply@anthropic.com>

* Prepare for a release with telemetry on by default Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com> * Make edits Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com> * Update version Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com> * Fix CHANGELOG formatting to match previous style Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com> * Fix telemetry e2e tests for default-enabled behavior - Update test expectations to reflect telemetry being enabled by default - Add feature flags cache cleanup in teardown to prevent state leakage between tests - This ensures each test runs with fresh feature flag state * Add wait after connection close for async telemetry submission * Remove debug logging from telemetry tests * Mark telemetry e2e tests as serial - must not run in parallel Root cause: Telemetry tests share host-level client across pytest-xdist workers, causing test isolation issues with patches. Tests pass serially but fail with -n auto. Solution: Add @pytest.mark.serial marker. CI needs to run these separately without -n auto. * Split test execution to run serial tests separately Telemetry e2e tests must run serially due to shared host-level telemetry client across pytest-xdist workers. Running with -n auto causes test isolation issues where futures aren't properly captured. Changes: - Run parallel tests with -m 'not serial' -n auto - Run serial tests with -m 'serial' without parallelization - Use --cov-append for serial tests to combine coverage - Mark telemetry e2e tests with @pytest.mark.serial - Update test expectations for default telemetry behavior - Add feature flags cache cleanup in test teardown * Mark telemetry e2e tests as serial - must not run in parallel The concurrent telemetry e2e test globally patches telemetry methods to capture events. When run in parallel with other tests via pytest-xdist, it captures telemetry events from other concurrent tests, causing assertion failures (expected 60 events, got 88). All telemetry e2e tests must run serially to avoid cross-test interference with the shared host-level telemetry client. --------- Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>

* added pandas 2.3.3 support and tests for py 3.14 Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> * generated poetry.lock Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> * lz4 version update for py 3.14 Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> * dependency selection based on py version Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> * pyarrow version update for py 3.14 Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> * poetry.lock with latest poetry version Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> --------- Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

* pandas 2.3.3 support for py < 3.14 Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> * poetry lock Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> --------- Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

…icks#723) Fixed the exception handler calls close() on _TelemetryClientHolder objects instead of accessing the client inside them.

…ks#724) * created util method to normalise http protocol in http path Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * Added impacted files using util method Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * Fixed linting issues Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * fixed broken test with mock host string Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * mocked http client Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * made case sensitive check in url utils Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * linting issue resolved Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * removed unnecessary md files Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * made test readbale Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> * changes done in auth util as well as sea http Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com> --------- Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com>

New minor version release

* query tags telemetry Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> * code linting fix Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> --------- Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

…heck failures in the repo (databricks#735) * Fix 60 seconds delay in gov cloud connections * keep it simple :) * Add fix for krb error * pin poetry * Pin for publish flow too * Fix failing tests * Edit order for pypi * One last fix : pls work

…ycle management (databricks#734) * Fix databricks#729 and databricks#731: Telemetry lifecycle management Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * Address review comments: revert timeout and telemetry_enabled changes Per reviewer feedback on PR databricks#734: 1. Revert timeout from 30s back to 900s (line 299) - Reviewer noted that with wait=False, timeout is not critical - The async nature and wait=False handle the exit speed 2. Revert telemetry_enabled parameter back to True (line 734) - Reviewer noted this is redundant given the early return - If enable_telemetry=False, we return early (line 729) - Line 734 only executes when enable_telemetry=True - Therefore using the parameter here is unnecessary These changes address the reviewer's valid technical concerns while keeping the core fixes intact: - wait=False for non-blocking shutdown (critical for Issue databricks#729) - Early return when enable_telemetry=False (critical for Issue databricks#729) - All Issue databricks#731 fixes (null-safety, __del__, documentation) Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * Fix Black formatting violations Apply Black formatting to files modified in previous commits: - src/databricks/sql/common/unified_http_client.py - src/databricks/sql/telemetry/telemetry_client.py Changes are purely cosmetic (quote style consistency). Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * Fix CI test failure: Prevent parallel execution of telemetry tests Add @pytest.mark.xdist_group to telemetry test classes to ensure they run sequentially on the same worker when using pytest-xdist (-n auto). Root cause: Tests marked @pytest.mark.serial were still being parallelized in CI because pytest-xdist doesn't respect custom markers by default. With host-level telemetry batching (PR databricks#718), tests running in parallel would share the same TelemetryClient and interfere with each other's event counting, causing test_concurrent_queries_sends_telemetry to see 88 events instead of the expected 60. The xdist_group marker ensures all tests in the "serial_telemetry" group run on the same worker sequentially, preventing state interference. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Fix telemetry test fixtures: Clean up state before AND after tests Modified telemetry_setup_teardown fixtures to clean up TelemetryClientFactory state both BEFORE and AFTER each test, not just after. This prevents leftover state from previous tests (pending events, active executors) from interfering with the current test. Root cause: In CI with sequential execution on the same worker, if a previous test left pending telemetry events in the executor, those events could be captured by the next test's mock, causing inflated event counts (88 instead of 60). Now ensures complete isolation between tests by resetting all shared state before each test starts. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Fix CI test failure: Clear _flush_event between tests The _flush_event threading.Event was never cleared after stopping the flush thread, remaining in "set" state. This caused timing issues in subsequent tests where the Event was already signaled, triggering unexpected flush behavior and causing extra telemetry events to be captured (88 instead of 60). Now explicitly clear the _flush_event flag in both setup (before test) and teardown (after test) to ensure clean state isolation between tests. This explains why CI consistently got 88 events - the flush_event from previous tests triggered additional flushes during test execution. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Add debug workflow and output to diagnose CI test failure 1. Created new workflow 'test-telemetry-only.yml' that runs only the failing telemetry test with -n auto, mimicking real CI but much faster 2. Added debug output to test showing: - Client-side captured events - Number of futures/batches - Number of server responses - Server-reported successful events This will help identify why CI gets 88 events vs local 60 events. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Fix workflow: Add krb5 system dependency The workflow was failing during poetry install due to missing krb5 system libraries needed for kerberos dependencies. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Fix xdist_group: Add --dist=loadgroup to pytest commands The @pytest.mark.xdist_group markers were being ignored because pytest-xdist uses --dist=load by default, which doesn't respect groups. With --dist=loadgroup, tests in the same xdist_group run sequentially on the same worker, preventing telemetry state interference between tests. This is the ROOT CAUSE of the 88 vs 60 events issue - tests were running in parallel across workers instead of sequentially on one worker as intended. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Add aggressive flush before test to prevent event interference CI shows 72 events instead of 60. Debug output reveals: - Client captured: 60 events (correct) - Server received: 72 events across 2 batches The 12 extra events accumulate in the timing window between fixture cleanup and mock setup. Other tests (like circuit breaker tests not in our xdist_group) may be sending telemetry concurrently. Solution: Add an explicit flush+shutdown RIGHT BEFORE setting up the mock to ensure a completely clean slate with zero buffered events. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Split workflow: Isolate telemetry tests in separate job To prevent interference from other e2e tests, split into two jobs: Job 1 (run-non-telemetry-tests): - Runs all e2e tests EXCEPT telemetry tests - Uses -n auto for parallel execution Job 2 (run-telemetry-tests): - Runs ONLY telemetry tests - Depends on Job 1 completing (needs: run-non-telemetry-tests) - Fresh Python process = complete isolation - No ambient telemetry from other tests This eliminates the 68 vs 60 event discrepancy by ensuring telemetry tests run in a clean environment with zero interference. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Fix workflows: Add krb5 deps and cleanup debug code Changes across multiple workflows: 1. integration.yml: - Add krb5 system dependency to telemetry job - Fixes: krb5-config command not found error during poetry install 2. code-coverage.yml: - Add krb5 system dependency - Split telemetry tests into separate step for isolation - Maintains coverage accumulation with --cov-append 3. publish-test.yml: - Add krb5 system dependency for consistent builds 4. test_concurrent_telemetry.py: - Remove debug print statements 5. Delete test-telemetry-only.yml: - Remove temporary debug workflow All workflows now have proper telemetry test isolation and required system dependencies for kerberos packages. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Fix publish-test.yml: Update Python 3.9 -> 3.10 Poetry 2.3.2 installation fails with Python 3.9: Installing Poetry (2.3.2): An error occurred. Other workflows use Python 3.10 and work fine. Updating to match ensures consistency and avoids Poetry installation issues. Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com> * Fix integration workflow: Remove --dist=loadgroup from non-telemetry tests - Remove --dist=loadgroup from non-telemetry job (only needed for telemetry) - Remove test_telemetry_e2e.py from telemetry job (was skipped before) - This should fix test_uc_volume_life_cycle failure caused by changed test distribution * Fix code-coverage workflow: Remove test_telemetry_e2e.py from coverage tests - Only run test_concurrent_telemetry.py in isolated telemetry step - test_telemetry_e2e.py was excluded in original workflow, keep it excluded * Fix publish-test workflow: Remove cache conditional - Always run poetry install (not just on cache miss) - Ensures fresh install with system dependencies (krb5) - Matches pattern used in integration.yml * Fix publish-test.yml: Remove duplicate krb5 install, restore cache conditional - Remove duplicate system dependencies step - Restore cache conditional to match main branch - Keep Python 3.10 (our change from 3.9) * Fix code-coverage: Remove serial tests step - All serial tests are telemetry tests (test_concurrent_telemetry.py and test_telemetry_e2e.py) - They're already run in the isolated telemetry step - Running -m serial with --ignore on both files results in 0 tests (exit code 5) --------- Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

Signed-off-by: Jayant Singh <jayant.singh@databricks.com>

* Add statement level query tag support by introducing it as a parameter on execute* methods Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com> * Add query_tags support to executemany method - Added query_tags parameter to executemany() method - Query tags are applied to all queries in the batch - Updated example to demonstrate executemany usage with query_tags - All tests pass (122/122 client tests) Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com> * add example that doesn't have tag Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com> * fix presubmit errors Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com> * another lint Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com> * address review comments Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com> --------- Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com>

…tion (databricks#749) * Allow specifiying query tags as a dict upon connection creation Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com> * fix comment Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com> --------- Signed-off-by: Jiabin Hu <jiabin.hu@databricks.com>

databricks#742) * Fix float inference to use DoubleParameter (64-bit) instead of FloatParameter (32-bit) Signed-off-by: Shubhambhusate <bhusates6@gmail.com> * Add DoubleParameter with Primitive.DOUBLE to test_inference coverage --------- Signed-off-by: Shubhambhusate <bhusates6@gmail.com>

… default (databricks#751) Updated pyarrow-concat

* Add statement-level query_tags support for SEA backend Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> * Simplify None handling in query_tags serialization Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com> --------- Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

Adds 42 new E2E tests and 3 unit tests covering gaps identified in MST + xDBC metadata audit: E2E tests (test_transactions.py): - Metadata operations (cursor.columns/tables/schemas/catalogs) in txn - MSTCheckRule-blocked SQL (SHOW/DESCRIBE/information_schema) in MST - Allowed operations (USE CATALOG/SCHEMA) in MST - DML variants: executemany, parameterized, DELETE, mixed DML in txn - Concurrent transactions: write conflict, write skew, repeatable reads - Multiple cursors in single transaction - Connection close with pending transaction (implicit rollback) - DDL behavior in transactions (CREATE/DROP/ALTER TABLE) - Edge cases: empty txn, read-only, atomicity, rollback recovery, MERGE Unit tests (test_client.py): - executemany error propagation in transaction context - Cursor close does not affect transaction state - Autocommit toggle consistency Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

vikrantpuppala and others added 30 commits May 9, 2025 23:02

Cleanup fields as they might be deprecated/removed/change in the futu…

bdb5154

…re (databricks#553) * Clean thrift files Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

[PECOBLR-361] convert column table to arrow if arrow present (databri…

7b51c6e

…cks#551)

Update CODEOWNERS (databricks#562)

f9936d7

new codeowners

Update github actions run conditions (databricks#569)

eaa542d

More conditions to run github actions

New Complex type test table + Github Action changes (databricks#575)

0947b9a

* Added new table * Some more fixes

Bump version to 4.0.4 (databricks#599)

0271b28

* Added change log * Update CHANGELOG.md Co-authored-by: Vikrant Puppala <vikrant.puppala@databricks.com> --------- Co-authored-by: Vikrant Puppala <vikrant.puppala@databricks.com>

Updated the changelog format (databricks#600)

c123af3

nit

Updated tests (databricks#614)

8b841c7

Add test to check thrift field IDs (databricks#602)

4cebc36

* Add test to check thrift field IDs --------- Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

Revert "Enhance Cursor close handling and context manager exception m… (

b3a6f55

databricks#613) * Revert "Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (databricks#554)" This reverts commit edfb283. * revert e2e

Bump version to 4.0.5 (databricks#615)

7696316

* Release version 4.0.5: Reverted cursor close handling changes to fix user errors. Updated version numbers in pyproject.toml and __init__.py. * Update CHANGELOG.md to include reference to issue databricks#613 for cursor close handling fix.

Removed Codeowners (databricks#623)

90f0ac1

nit

[PECOBLR-587] Azure Service Principal Credential Provider (databricks…

e50e86d

…#621) * basic setup * Nit * working * moved pyjwt to code dependency * nit * nit * nit * nit * nit * nit * testing sdk * Refractor * logging * nit * nit * nit * nit * nit

Arrow performance optimizations (databricks#638)

e0ca049

* Minor fix * Perf update * more * test fix

added logs for cloud fetch speed (databricks#654)

59d28b0

Make telemetry batch size configurable and add time-based flush (data…

a0d7cd1

…bricks#622) configurable telemetry batch size, time based flush Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>

jayantsing-db and others added 27 commits November 14, 2025 02:45

Bump to version 4.2.0 (databricks#707)

cca421b

Bump to version 4.2.0 Signed-off-by: Jayant Singh <jayant.singh@databricks.com>

Ready for 4.2.1 release (databricks#713)

ad227ca

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

Change default use_hybrid_disposition to False (databricks#714)

b8494ff

This changes the default value of use_hybrid_disposition from True to False in the SEA backend, disabling hybrid disposition by default.

New minor release (databricks#722)

ce55e7b

Fixed the exception handler close() on _TelemetryClientHolder (databr…

9b4e577

…icks#723) Fixed the exception handler calls close() on _TelemetryClientHolder objects instead of accessing the client inside them.

New minor version release 4.2.4 (databricks#725)

03eb369

New minor version release

Bump to version 4.2.5 (databricks#737)

9fe7356

Signed-off-by: Jayant Singh <jayant.singh@databricks.com>

Updated the PyArrow concatenation of tables to use promote_options as…

36fb376

… default (databricks#751) Updated pyarrow-concat

Rename test table to avoid collisions with JDBC driver tests

cf92d96

Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

vikrantpuppala had a problem deploying to azure-prod April 2, 2026 06:50 — with GitHub Actions Failure

vikrantpuppala closed this Apr 2, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PECOBLR-2086] Add comprehensive MST transaction metadata and edge case tests#1

[PECOBLR-2086] Add comprehensive MST transaction metadata and edge case tests#1
vikrantpuppala wants to merge 81 commits intomainfrom
add-mst-transaction-tests

vikrantpuppala commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

Conversation

vikrantpuppala commented Apr 2, 2026

Summary

Test plan

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants