fix: preserve column names with spaces in wr.redshift.copy()

[hirenkumar-n-dholariya]

Problem

wr.redshift.copy() silently renames columns with spaces (e.g. "my col" → "my_col")
because the internal s3.to_parquet call defaults to pyarrow flavor='spark',
which sanitizes column names.

Fix

Explicitly pass pyarrow_additional_kwargs={"flavor": None} in the internal
s3.to_parquet call to preserve original column names.

Fixes #3293

[kukushking]

AWS CodeBuild CI Report

• CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
• Commit ID: fdccae4
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[kukushking]

AWS CodeBuild CI Report

• CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
• Commit ID: fdccae4
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[hirenkumar-dholariya]

AWS CodeBuild CI Report

• CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
• Commit ID: fdccae4
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@kukushking Could you confirm this failure is pre-existing and unrelated to the fix? Happy to address any other feedback!
The CI failure is unrelated to this fix. The GitHubDistributedCodeBuild failure is caused by a pre-existing incompatibility between modin==0.37.1 and pandas==3.0.1 — pandas.read_gbq was removed in pandas 3.x, which causes an AttributeError when loading modin.

The GitHubCodeBuild (non-distributed) pipeline passed successfully.
This issue exists independently in the main branch and is not introduced by this PR.

[hirenkumar-n-dholariya]

@kukushking Could you confirm this failure is pre-existing and unrelated to the fix? Happy to address any other feedback! The CI failure is unrelated to this fix. The GitHubDistributedCodeBuild failure is caused by a pre-existing incompatibility between modin==0.37.1 and pandas==3.0.1 — pandas.read_gbq was removed in pandas 3.x, which causes an AttributeError when loading modin.

The GitHubCodeBuild (non-distributed) pipeline passed successfully. This issue exists independently in the main branch and is not introduced by this PR.

@kukushking Could you confirm this failure is pre-existing and unrelated to the fix? Happy to address any other feedback!
The CI failure is unrelated to this fix. The GitHubDistributedCodeBuild failure is caused by a pre-existing incompatibility between modin==0.37.1 and pandas==3.0.1 — pandas.read_gbq was removed in pandas 3.x, which causes an AttributeError when loading modin.

The GitHubCodeBuild (non-distributed) pipeline passed successfully.
This issue exists independently in the main branch and is not introduced by this PR.

[hirenkumar-n-dholariya]

@kukushking Could you confirm this failure is pre-existing and unrelated to the fix? Happy to address any other feedback! The CI failure is unrelated to this fix. The GitHubDistributedCodeBuild failure is caused by a pre-existing incompatibility between modin==0.37.1 and pandas==3.0.1 — pandas.read_gbq was removed in pandas 3.x, which causes an AttributeError when loading modin.
The GitHubCodeBuild (non-distributed) pipeline passed successfully. This issue exists independently in the main branch and is not introduced by this PR.

@kukushking Could you confirm this failure is pre-existing and unrelated to the fix? Happy to address any other feedback! The CI failure is unrelated to this fix. The GitHubDistributedCodeBuild failure is caused by a pre-existing incompatibility between modin==0.37.1 and pandas==3.0.1 — pandas.read_gbq was removed in pandas 3.x, which causes an AttributeError when loading modin.

The GitHubCodeBuild (non-distributed) pipeline passed successfully. This issue exists independently in the main branch and is not introduced by this PR.

Hi @kukushking
Hope you are doing well. Could you please take a look at the comments and help to share your feedback/approval on the PR. Thank you so much in advance for your time.

[kukushking]

AWS CodeBuild CI Report

• CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
• Commit ID: 857326e
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[hirenkumar-n-dholariya]

@kukushking Could you please take a look at this PR when you get a chance?

Quick summary:

• The fix passes flavor=None to the internal s3.to_parquet call to preserve column names with spaces in wr.redshift.copy()
• The GitHubCodeBuild pipeline passed
• The GitHubDistributedCodeBuild failure is a pre-existing issue caused by modin==0.37.1 incompatibility with pandas==3.0.1 (pandas.read_gbq` was removed in pandas 3.x) -> unrelated to this fix

Would appreciate your review!

[kukushking]

AWS CodeBuild CI Report

• CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
• Commit ID: 857326e
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[kukushking]

Hi @hirenkumar-n-dholariya yes - the failure you are referring to is pre-existing, no reason to worry about.

With regards to your change, this is a breaking change of default behavior for any current redshift user will be impacted. We will consider this for the next major version.

[hirenkumar-n-dholariya]

@kukushking Thank you for the feedback! That's a fair point about the breaking change concern.

Would it make sense to make the behavior configurable via an optional parameter, so existing users are not impacted by default?

For example:

def copy(
    df,
    ...
    sanitize_column_names: bool = True,  # preserves backward compatibility
):

This way:

• Existing users are unaffected (default = True keeps current behavior)
• Users who need to preserve column names with spaces can opt in with sanitize_column_names=False

Happy to implement this if it sounds like a good direction.

[kukushking]

@hirenkumar-n-dholariya yes, that would makes sense! Please also consider adding a test case to test the new behavior. Thank you!

[hirenkumar-n-dholariya]

@kukushking Both files are now formatted correctly.
The sanitize_column_names parameter added with default=True for backward compatibility, test case added, and ruff
formatting fixed.

Could you please review, give your approval/feeedback to proceed with merge.