-
Notifications
You must be signed in to change notification settings - Fork 891
Logs API - initialize LogRecordData.Timestamp with MinValue #7045
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
833135a
2f945ba
6ca310e
dad842c
c6c9a55
e90c7c8
48d6a07
8bff6b4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -179,9 +179,25 @@ internal static int WriteLogRecord(byte[] buffer, int writePosition, SdkLimitOpt | |
| var logRecordLengthPosition = otlpTagWriterState.WritePosition; | ||
| otlpTagWriterState.WritePosition += ReserveSizeForLength; | ||
|
|
||
| var timestamp = (ulong)logRecord.Timestamp.ToUnixTimeNanoseconds(); | ||
| otlpTagWriterState.WritePosition = ProtobufSerializer.WriteFixed64WithTag(otlpTagWriterState.Buffer, otlpTagWriterState.WritePosition, ProtobufOtlpLogFieldNumberConstants.LogRecord_Time_Unix_Nano, timestamp); | ||
| otlpTagWriterState.WritePosition = ProtobufSerializer.WriteFixed64WithTag(otlpTagWriterState.Buffer, otlpTagWriterState.WritePosition, ProtobufOtlpLogFieldNumberConstants.LogRecord_Observed_Time_Unix_Nano, timestamp); | ||
| var logTimestamp = logRecord.Timestamp; | ||
| ulong timeUnixNano; | ||
| ulong observedTimeUnixNano; | ||
| if (logTimestamp != DateTime.MinValue) | ||
| { | ||
| // Timestamp was explicitly set: use it for both fields. | ||
| timeUnixNano = (ulong)logTimestamp.ToUnixTimeNanoseconds(); | ||
| observedTimeUnixNano = timeUnixNano; | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not always setting
I am not sure what would be the value of setting the same time for ObservedTimestamp and Timestamp. Here is what we do in OTel Go: https://github.com/open-telemetry/opentelemetry-go/blob/3157d0a7d9d22c3343838da6b1a691557676072f/sdk/log/logger.go#L116-L119
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Setting
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Julius is right, it is intermittent state and should be changed in the short cadence in #6973. |
||
| } | ||
| else | ||
| { | ||
| // Timestamp not set -> time_unix_nano = 0 ("unknown or missing" per OTLP spec). | ||
| // observed_time_unix_nano MUST still be populated (proto spec requirement). | ||
| timeUnixNano = 0; | ||
| observedTimeUnixNano = (ulong)DateTime.UtcNow.ToUnixTimeNanoseconds(); | ||
| } | ||
|
Comment on lines
+185
to
+197
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Have you considered changing
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, but we should handle timestamp and observed timespan differently, so I would keep as is. |
||
|
|
||
| otlpTagWriterState.WritePosition = ProtobufSerializer.WriteFixed64WithTag(otlpTagWriterState.Buffer, otlpTagWriterState.WritePosition, ProtobufOtlpLogFieldNumberConstants.LogRecord_Time_Unix_Nano, timeUnixNano); | ||
| otlpTagWriterState.WritePosition = ProtobufSerializer.WriteFixed64WithTag(otlpTagWriterState.Buffer, otlpTagWriterState.WritePosition, ProtobufOtlpLogFieldNumberConstants.LogRecord_Observed_Time_Unix_Nano, observedTimeUnixNano); | ||
|
|
||
| otlpTagWriterState.WritePosition = ProtobufSerializer.WriteEnumWithTag(otlpTagWriterState.Buffer, otlpTagWriterState.WritePosition, ProtobufOtlpLogFieldNumberConstants.LogRecord_Severity_Number, logRecord.Severity.HasValue ? (int)logRecord.Severity : 0); | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My read of the API spec is that timestamp is an optional parameter, but the spec does not lend guidance for what the default value should be. I think one could argue that our current behavior of defaulting to UtcNow is acceptable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
However if we were to change the default value, why
DateTime.MinValue? Wouldn't a default value of 1/1/1970 00:00 be better? Someone correct me if I'm wrong, but I believe that's what Java and Go SDKs default to.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could clarify it in the spec. The intention is that the default is "0" when the value is exported via OTLP if no value was explicitly set.
From https://github.com/open-telemetry/opentelemetry-proto/blob/62498ba4f8e268c40e9420ed912e3cc32e11f487/opentelemetry/proto/logs/v1/logs.proto#L139-L142:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alan lets consider this:
https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-timestamp
and https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-observedtimestamp
The first one is marked as fully optional (what is more proto treats its as a nullable, sets to 0).Does not say anything about default.
Observed timestamp is the moment when the OTel sees it first time, and it should be set to the UtcNow.
We are still missing ObservedTimestamp, it will be added in scope of #6979.
Sdk already exposes LogRecord as DateTime non nullable this value (see changes in this file). To do not break this, I considered choosing one value as null/0/equivalent. IMO DateTime.MinValue is the most obvious choice. It was discussed earlier with @pellared, and the Go-lang is doing mostly the same.
The following part is prepared by the Codex, but as I understand the code, it confirms what I stated about go-lang.
Short version: this branch behavior matches Java and Go.
OpenTelemetry defines
Timestampas optional when the source/event time is unknown.ObservedTimestampshould be set once OpenTelemetry observes the event. In OTLP protobuf,time_unix_nano = 0means unknown or missing timestamp.Java
Java
v1.61.0keeps the source timestamp unset when the caller does not provide one. The SDK storestimestampEpochNanosas0by default, whileobservedTimestampEpochNanosis filled from the SDK clock if absent.Go
Go
v1.43.0/ logsv0.19.0behaves similarly. A zero-valuelog.Recordhas no source timestamp. The SDK preserves that zero timestamp, fillsObservedTimestampwithtime.Now()if it is unset, and the OTLP transform maps a zero timestamp to0.Conclusion
This change makes .NET follow the same model:
DateTime.MinValuerepresents an unset source timestamp.time_unix_nano = 0.observed_time_unix_nanois still populated when OpenTelemetry observes/serializes the record.Sources
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think a clarification in the logs API spec would be beneficial. Both
time_unix_nanoandobserved_time_unix_nanoare marked as optional parameters which could lead someone to believe the intent is that they both remain unset if they're not provided. Only if you go read the data model spec can you begin to try and infer what the intended default behavior should be.Seemingly silly question, but I think it's important for us to think through: What if the timestamp is set to
DateTime.MinValue.AddDays(1);?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I should note, I'm supportive of changing the default value and I favor matching our behavior to what Java and Go are doing, but the fact we're using a
DateTimeadds a bit of a wrinkle we just need to think through...There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I created open-telemetry/opentelemetry-specification#5075