Description:
As part of the changes introduced in #5434, the OTEL protobuf specification was updated. While the system remains backward compatible when using Protobuf, issues arise when the data is sent in JSON format.
When sample JSON data includes instrumentationLibraryMetrics with an empty instrumentationLibrary and a list of metrics like this:
"instrumentationLibraryMetrics": [
{
"instrumentationLibrary": {},
"metrics": [
{
"name": "counter-int"
}
]
}
]The following exception is thrown:
org.opensearch.dataprepper.GrpcRequestExceptionHandler - Unexpected exception handling gRPC request
com.google.protobuf.InvalidProtocolBufferException: Expected start of object, got: [
at org.curioswitch.common.protobuf.json.TypeSpecificMarshaller.mergeValue(TypeSpecificMarshaller.java:66) ~[protobuf-jackson-2.5.0.jar:?]
Expected Behavior:
Data Prepper should handle empty instrumentationLibrary objects gracefully in JSON mode, without throwing an exception. At a minimum, a more user-friendly error message would help in identifying and correcting malformed input.
Steps to Reproduce:
- Send OTEL metrics in JSON format with an empty
instrumentationLibrary object.
- Observe the thrown
InvalidProtocolBufferException.
Sample pipeline configuration:
test-pipeline:
source:
otel_metrics_source:
ssl: false
port: 4317
path: /log/ingest
unframed_requests: true
sink:
- stdout:
curl -X POST http://localhost:4317/log/ingest \
-H "Content-Type: application/json" \
-u admin:securepass \
-d '{
"resourceMetrics": [
{
"instrumentationLibraryMetrics": [
{
"instrumentationLibrary": {},
"metrics": [
{
"name": "cpu_usage",
"sum": {
"dataPoints": [
{
"asDouble": 75.5
}
]
}
}
]
}
]
}
]
}'
Environment:
Suggested Fix:
Investigate JSON deserialization logic for instrumentationLibrary and handle cases where the field is present but empty.
Description:
As part of the changes introduced in #5434, the OTEL protobuf specification was updated. While the system remains backward compatible when using Protobuf, issues arise when the data is sent in JSON format.
When sample JSON data includes
instrumentationLibraryMetricswith an emptyinstrumentationLibraryand a list of metrics like this:The following exception is thrown:
Expected Behavior:
Data Prepper should handle empty
instrumentationLibraryobjects gracefully in JSON mode, without throwing an exception. At a minimum, a more user-friendly error message would help in identifying and correcting malformed input.Steps to Reproduce:
instrumentationLibraryobject.InvalidProtocolBufferException.Sample pipeline configuration:
Environment:
protobuf-jackson-2.5.0Suggested Fix:
Investigate JSON deserialization logic for
instrumentationLibraryand handle cases where the field is present but empty.