(IcebergIO) Support PartitionSpec/SortOrder on dynamic table creation via IcebergIO

[clairemcginty]

#38269 added SortOrder support to Iceberg writes via Managed IO, but it's not accessible through IcebergIO directly. This PR adds support for setting SortOrder+PartitionSpec through the IcebergIO.writeRows builder.

It does make serialization of the transform more complex which might be antithetical to the goal of keeping new transforms as portable as possible -- cc @ahmedabu98, let me know if not exposing these params via IcebergIO was by design and if I should just switch to ManagedIO if I need to specify.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[ahmedabu98]

Sorry was OOO for some time and just seeing this now. I think this PR is a good addition, and doesn’t harm the portable case at all. Just had one nit regarding naming but it LGTM! Feel free to convert it so we can run some tests against it

Also I’m curious to know if the ManagedIO route is causing any friction for your use case?

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[clairemcginty]

Sorry was OOO for some time and just seeing this now. I think this PR is a good addition, and doesn’t harm the portable case at all. Just had one nit regarding naming but it LGTM! Feel free to convert it so we can run some tests against it

Also I’m curious to know if the ManagedIO route is causing any friction for your use case?

sounds good! Just renamed the method and undrafted the PR.

No, so far so good! The iceberg write API in Scio goes through ManagedIO and works well. I've just been parity testing a lot of iceberg IO features on both managedIO + icebergIO and caught this particular discrepancy!