Support serialization and deserialization for GoogleCloudStorageReadOptions

[shunping]

This PR fixes a bug where annotating GoogleCloudStorageReadOptions with @JsonIgnore caused custom GCS read options (such as fadvise) to be dropped during JSON serialization, forcing remote workers to fall back to default settings.

Additionally, in gcs-connector v3, GoogleCloudStorageReadOptions.DEFAULT changed its default fadvise from SEQUENTIAL to AUTO .

• v.2.2.33: https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v2.2.33/gcsio/src/main/java/com/google/cloud/hadoop/gcsio/GoogleCloudStorageReadOptions.java#L47
• v3.1.16: https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v3.1.16/gcsio/src/main/java/com/google/cloud/hadoop/gcsio/GoogleCloudStorageReadOptions.java#L50

Beam workloads default to SEQUENTIAL in GcsReadOptionsFactory to preserve expected sequential read throughput and caching behavior.

Going forward, users can configure GoogleCloudStorageReadOptions so that they successfully take effect on workers using a code snippet like the following:

        // Configure GCS Read Options with AUTO fadvise
        GoogleCloudStorageReadOptions gcsReadOpt = GoogleCloudStorageReadOptions.DEFAULT.toBuilder()
                .setFadvise(GoogleCloudStorageReadOptions.Fadvise.AUTO)
                .build();
        options.as(GcsOptions.class).setGoogleCloudStorageReadOptions(gcsReadOpt);

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[shunping]

r: @Abacn

[github-actions]

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

[Abacn]

The size of this fix is non-trivial. There are a few concerns

• reflection based, generally fragile

• It also increases serialized size of pipeline options (seen in Dataflow UI, internal objects, etc).

Given that get/setGoogleCloudStorageReadOptions are deprecated and is only effective in GcsUtilV1, I would prefer a smaller fix:

• Only json-serialize-deserialize options that we want to explicitly expose (Fadvise, etc). On serialization, when it's found not the default, write to encoding. In this way we can avoid reflection and largely reduce the serialization size

• If possible, do not leak jackson classes to ApiSurface

[shunping]

If our goal is solely to fix the regression caused by the default value of fadvise, we wouldn't need to modify the serialization and deserialization logic for GoogleCloudStorageReadOptions.

However, the broader intent of this fix is to address the underlying issue: the inability to propagate GoogleCloudStorageReadOptions to remote workers via pipeline options.

Regarding the use of reflection, it was chosen to future-proof the implementation. If we explicitly serialize fields instead of using reflection, we will be forced to update our code every time the upstream gcs-connector modifies or adds fields to this class.

[shunping]

I made the change to support a subset of options.

However, Jackson classes must be included in the ApiSurface. This is because GcsOptions is a public interface with methods annotated with @JsonSerialize and @JsonDeserialize. The JsonDeserialize annotation declares methods returning Class<? extends JsonDeserializer>, which pulls com.fasterxml.jackson.databind into the exposed API surface graph. JsonDeserializer, in turn, references JsonParser and other classes from com.fasterxml.jackson.core.

[shunping]

@Abacn