Allow passing parameters into `to_pandas` call

[j-bennet]

When creating a Connection, currently it's possible to configure some aspects of pandas / arrow, such as _use_arrow_native_complex_types and _use_arrow_native_timestamps, but _convert_arrow_table uses a hardcoded type mapper that's impossible to change:

databricks-sql-python/src/databricks/sql/client.py

Lines 1349 to 1362 in 0947b9a

	dtype_mapping = {
	pyarrow.int8(): pandas.Int8Dtype(),
	pyarrow.int16(): pandas.Int16Dtype(),
	pyarrow.int32(): pandas.Int32Dtype(),
	pyarrow.int64(): pandas.Int64Dtype(),
	pyarrow.uint8(): pandas.UInt8Dtype(),
	pyarrow.uint16(): pandas.UInt16Dtype(),
	pyarrow.uint32(): pandas.UInt32Dtype(),
	pyarrow.uint64(): pandas.UInt64Dtype(),
	pyarrow.bool_(): pandas.BooleanDtype(),
	pyarrow.float32(): pandas.Float32Dtype(),
	pyarrow.float64(): pandas.Float64Dtype(),
	pyarrow.string(): pandas.StringDtype(),
	}

Similarly, there's no way to pass other parameters to the to_pandas call here:

databricks-sql-python/src/databricks/sql/client.py

Lines 1366 to 1370 in 0947b9a

	df = table_renamed.to_pandas(
	types_mapper=dtype_mapping.get,
	date_as_object=True,
	timestamp_as_object=True,
	)

Please consider extending the connection parameters to add more flexibility.

[madhav-db]

Hey @j-bennet , I've added a PR to extend this functionality (#579), you should be able to use these once merged.