Support configuring SDK batch size in config using items or bytes

[iblancasa]

Fixes #5068

Changes

Add in-development guidance for configuring SDK batch size using items or bytes, including declarative configuration support via batch_size_unit.

Similar to what we currently support in exporter helper from OpenTelemetry Collector https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/exporterhelper

Prototype for go: open-telemetry/opentelemetry-go#8321

• Links to the prototypes (when adding or changing features)
• CHANGELOG.md file updated for non-trivial changes
• Declarative config data model is updated if SDK config surface is changed

[dashpole]

I think first you need to define the SDK surface that actually allows it to be configured with a batch size in units other than spans/logs/data-points.

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#batching-processor has a maxExportBatchSize, which is defined in terms of the number of spans.

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/sdk.md#batching-processor has a maxExportBatchSize, which is defined in terms of the number of log records.

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#periodic-exporting-metricreader has (development) maxExportBatchSize, which is defined in terms of the number of metric data points.

Then we can add a declarative configuration surface for it as well. I think it should be possible to add a unit without it being a breaking change, but we need to check and make sure implementations didn't do something like name it "maxLogRecordBatchSize", or something that would make it harder to use for bytes.

I don't think library guidelines are the right place to put the specification for how SDKs handle batching.

[jmacd]

Please have a look at open-telemetry/opentelemetry-collector#15132: the same discussion is happening for collector exporterhelper.

[github-actions]

This PR was marked stale. It will be closed in 14 days without additional activity.

[jmacd]

please see https://gist.github.com/braydonk/78ec0cfe9079ab8b611f52c49830116c, recommend closing this PR. @braydonk

[braydonk]

For future readers: The project from the gist has been opened as a proper project proposal in the community repo. open-telemetry/community#3477