Describe the bug

Hello!

I'm having some trouble using the operator (Python bindings) to write a pyarrow table to S3. I'm using operator.open (so an opendal.File) as a context manager and passing this object to pyarrow.parquet.ParquetWriter along with my table to be persisted to S3. When calling write_table on the writer object in a unit test, the operation hangs indefinitely (no failure).

Reading a table from S3 works as expected, it's writing that's causing us trouble.

Steps to Reproduce

from pathlib import Path

import boto3
import pyarrow as pa
from moto import mock_aws
from moto.moto_server.threaded_moto_server import ThreadedMotoServer
from opendal import Operator


def test_s3_operator_write() -> None:
    with mock_aws():
        server = ThreadedMotoServer(port=0)
        server.start()
        _, port = server.get_host_and_port()
        endpoint_url = f"http://localhost:{port}"

        conn = boto3.resource("s3", endpoint_url=endpoint_url)
        conn.create_bucket(Bucket="test-bucket")

        handler = Operator(
            "s3",
            bucket="test-bucket",
            region="auto",
            endpoint=endpoint_url  # Add the mock endpoint URL
        )

        table = pa.Table.from_pydict({"A": [2]})
        dest = Path("prefix/table.parquet")
        with handler.open(dest, mode="wb") as f:
            with pa.parquet.ParquetWriter(f) as writer:
                writer.write(table)

        server.stop()

Expected Behavior

The test would actually execute the write to S3, and then I would verify this by reading using the operator.

Additional Context

I tried wrapping the operator in pyarrow.PythonFile and passing it to ParquetWriter, but to no avail.

I wonder if I'm doing something incorrectly, or if this is even expected behaviour given this comment about file-like operations being supported only for fs mode?

Would appreciate your help, thanks!

Are you willing to submit a PR to fix this bug?

• Yes, I would like to submit a PR.