HDDS-14194. Make streaming write SYNC configurable

[ivandika3]

What changes were proposed in this pull request?

See BlockDataStreamOutput#needSync, currently SYNC is never used and therefore we might not be sure when the data is actually flushed to the disk (see StreamDataChannelBase#force which will invoke FileChannel#force(false) to flush the data to the underlying storage)

private boolean needSync(long position) {
  if (SYNC_SIZE > 0) {
    // TODO: or position >= fileLength
    if (position - syncPosition >= SYNC_SIZE) {
      syncPosition = position;
      return true;
    }
  }
  return false;
} 

We can create a configuration to allow SYNC, which is disabled by default to preserve compatibility.

We also might want to revisit whether we should make SYNC to be a client-side or server-side. The concern is if a client is misconfigured, it might cause a lot of sync on the DN which will cause higher latencies.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14194

How was this patch tested?

[rich7420]

thanks for the patch @ivandika3 !

[szetszwo]

+1 the change looks good.

[ivandika3]

Thanks @szetszwo @rich7420 for the reviews.

In the future, we might either change the SYNC boundary to PutBlock level or only during close similar to HDDS-14246.