Logs API - initialize LogRecordData.Timestamp with MinValue#7045
Logs API - initialize LogRecordData.Timestamp with MinValue#7045Kielek wants to merge 8 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7045 +/- ##
==========================================
- Coverage 89.86% 89.81% -0.06%
==========================================
Files 273 273
Lines 13767 13773 +6
==========================================
- Hits 12372 12370 -2
- Misses 1395 1403 +8
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
@juliuskoval, could you please review? |
| if (logTimestamp != DateTime.MinValue) | ||
| { | ||
| // Timestamp was explicitly set: use it for both fields. | ||
| timeUnixNano = (ulong)logTimestamp.ToUnixTimeNanoseconds(); | ||
| observedTimeUnixNano = timeUnixNano; | ||
| } | ||
| else | ||
| { | ||
| // Timestamp not set -> time_unix_nano = 0 ("unknown or missing" per OTLP spec). | ||
| // observed_time_unix_nano MUST still be populated (proto spec requirement). | ||
| timeUnixNano = 0; | ||
| observedTimeUnixNano = (ulong)DateTime.UtcNow.ToUnixTimeNanoseconds(); | ||
| } |
There was a problem hiding this comment.
Have you considered changing ToUnixTimeNanoseconds (and other extension methods) to return 0 when datetime is min?
There was a problem hiding this comment.
Yes, but we should handle timestamp and observed timespan differently, so I would keep as is.
| { | ||
| // Timestamp was explicitly set: use it for both fields. | ||
| timeUnixNano = (ulong)logTimestamp.ToUnixTimeNanoseconds(); | ||
| observedTimeUnixNano = timeUnixNano; |
There was a problem hiding this comment.
Why not always setting observedTimeUnixNano = (ulong)DateTime.UtcNow.ToUnixTimeNanoseconds();?
This field SHOULD be set once the event is observed by OpenTelemetry.
I am not sure what would be the value of setting the same time for ObservedTimestamp and Timestamp.
Here is what we do in OTel Go: https://github.com/open-telemetry/opentelemetry-go/blob/3157d0a7d9d22c3343838da6b1a691557676072f/sdk/log/logger.go#L116-L119
There was a problem hiding this comment.
Setting ObservedTimestamp is addressed in #6979; this PR only addresses nullability of Timestamp if I understand it correctly. Changes from the other PR would allow users to set ObservedTimestamp in the API/SDK.
There was a problem hiding this comment.
Julius is right, it is intermittent state and should be changed in the short cadence in #6973.
I hope that both will be merged together. For now, I am keeping existing behavior as much as possible.
You didn't add changelogs for |
|
This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or pushing will instruct the bot to automatically remove the label. This bot runs once per day. |
| #pragma warning restore CA1815 // Override equals and operator equals on value types | ||
| { | ||
| internal DateTime TimestampBacking = DateTime.UtcNow; | ||
| internal DateTime TimestampBacking = DateTime.MinValue; |
There was a problem hiding this comment.
My read of the API spec is that timestamp is an optional parameter, but the spec does not lend guidance for what the default value should be. I think one could argue that our current behavior of defaulting to UtcNow is acceptable.
There was a problem hiding this comment.
However if we were to change the default value, why DateTime.MinValue? Wouldn't a default value of 1/1/1970 00:00 be better? Someone correct me if I'm wrong, but I believe that's what Java and Go SDKs default to.
There was a problem hiding this comment.
My read of the API spec is that timestamp is an optional parameter, but the spec does not lend guidance for what the default value should be.
We could clarify it in the spec. The intention is that the default is "0" when the value is exported via OTLP if no value was explicitly set.
From https://github.com/open-telemetry/opentelemetry-proto/blob/62498ba4f8e268c40e9420ed912e3cc32e11f487/opentelemetry/proto/logs/v1/logs.proto#L139-L142:
// time_unix_nano is the time when the event occurred.
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
// Value of 0 indicates unknown or missing timestamp.
fixed64 time_unix_nano = 1;There was a problem hiding this comment.
Alan lets consider this:
https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-timestamp
and https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-observedtimestamp
The first one is marked as fully optional (what is more proto treats its as a nullable, sets to 0).Does not say anything about default.
Observed timestamp is the moment when the OTel sees it first time, and it should be set to the UtcNow.
We are still missing ObservedTimestamp, it will be added in scope of #6979.
Sdk already exposes LogRecord as DateTime non nullable this value (see changes in this file). To do not break this, I considered choosing one value as null/0/equivalent. IMO DateTime.MinValue is the most obvious choice. It was discussed earlier with @pellared, and the Go-lang is doing mostly the same.
The following part is prepared by the Codex, but as I understand the code, it confirms what I stated about go-lang.
Short version: this branch behavior matches Java and Go.
OpenTelemetry defines Timestamp as optional when the source/event time is unknown. ObservedTimestamp should be set once OpenTelemetry observes the event. In OTLP protobuf, time_unix_nano = 0 means unknown or missing timestamp.
Java
Java v1.61.0 keeps the source timestamp unset when the caller does not provide one. The SDK stores timestampEpochNanos as 0 by default, while observedTimestampEpochNanos is filled from the SDK clock if absent.
Go
Go v1.43.0 / logs v0.19.0 behaves similarly. A zero-value log.Record has no source timestamp. The SDK preserves that zero timestamp, fills ObservedTimestamp with time.Now() if it is unset, and the OTLP transform maps a zero timestamp to 0.
Conclusion
This change makes .NET follow the same model:
DateTime.MinValuerepresents an unset source timestamp.- OTLP exports that as
time_unix_nano = 0. observed_time_unix_nanois still populated when OpenTelemetry observes/serializes the record.
Sources
There was a problem hiding this comment.
We could clarify it in the spec. The intention is that the default is "0" when the value is exported via OTLP if no value was explicitly set.
I think a clarification in the logs API spec would be beneficial. Both time_unix_nano and observed_time_unix_nano are marked as optional parameters which could lead someone to believe the intent is that they both remain unset if they're not provided. Only if you go read the data model spec can you begin to try and infer what the intended default behavior should be.
OTLP exports that as time_unix_nano = 0
Seemingly silly question, but I think it's important for us to think through: What if the timestamp is set to DateTime.MinValue.AddDays(1);?
- What should the OTLP exporter export? The timestamp from the OTLP data model is an unsigned 64 bit integer.
- What do we recommend other exporters do like the console export or a custom exporter? Should they export 1/2/0001, or would we recommend a behavior that matches OTLP?
There was a problem hiding this comment.
I should note, I'm supportive of changing the default value and I favor matching our behavior to what Java and Go are doing, but the fact we're using a DateTime adds a bit of a wrinkle we just need to think through...
There was a problem hiding this comment.
|
If it's supposed to mean |
I think that we are able to bring resonable backward-behavioral compatibility. I do not see possibility for such things for the public contract on the SDK level (already declared as stable). |
|
I also think that using |
|
This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or pushing will instruct the bot to automatically remove the label. This bot runs once per day. |
Handles discussions from #6979 related to nullability.
Changes
Logs API - initialize LogRecordData.Timestamp with MinValue
MinValue is equivalent of 0/null value from OTel Spec.
Merge requirement checklist
CHANGELOG.mdfiles updated for non-trivial changes[ ] Changes in public API reviewed (if applicable)