-
Notifications
You must be signed in to change notification settings - Fork 976
First cut at Entity API + SDK specification. #4565
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
d79fb72
d20b378
246867b
371cb0e
8c87d42
8b16faf
a392bb7
1079fad
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| <!--- Hugo front matter used to generate the website version of this page: | ||
| linkTitle: API | ||
| ---> | ||
|
|
||
| # Entities API | ||
|
|
||
| **Status**: [Development](../document-status.md) | ||
|
|
||
| <!-- toc --> | ||
|
|
||
| - [Overview](#overview) | ||
| - [ResourceProvider](#resourceprovider) | ||
| * [ResourceProvider operations](#resourceprovider-operations) | ||
| + [Get the active Resource](#get-the-active-resource) | ||
| * [Resource](#resource) | ||
| * [Attach an Entity](#attach-an-entity) | ||
|
|
||
| <!-- tocstop --> | ||
|
|
||
| ## Overview | ||
|
|
||
| The Entities API is provided for instrumentation authors to build | ||
| [Entities](./data-model.md#entity-data-model) and report them | ||
| against the OpenTelemetry [Resource](../resource/README.md) for all signals. | ||
|
|
||
| This detection of the environment and its entities is expected | ||
| to happen early / immediately in the lifespan of a system being | ||
| observed, however, the Entities are allowed to change over the | ||
| lifespan of the Resource. | ||
|
|
||
| The Entities API consists of these main components: | ||
|
|
||
| - [ResourceProvider](#resourceprovider) | ||
| - [Resource](#resource) | ||
|
|
||
| ## ResourceProvider | ||
|
|
||
| `Resource`s can be accessed with a `ResourceProvider`. | ||
|
|
||
| Normally, the `ResourceProvider` is expected to be accessed from a central place. | ||
| Thus, the API SHOULD provide a way to set/register and access a global default | ||
| `ResourceProvider`. | ||
|
|
||
| ### ResourceProvider operations | ||
|
|
||
| The `ResourceProvider` MUST provide the following functions: | ||
|
|
||
| * Get the active `Resource` | ||
|
|
||
| #### Get the active Resource | ||
|
|
||
| This API MUST return the current `Resource` for which Telemetry | ||
| is being reported. | ||
|
|
||
| ### Resource | ||
|
|
||
| The `Resource` is responsible for emitting `Entity`s and tracking | ||
| the current set of entities attached to the `Resource`. | ||
|
|
||
| The `Resource` MUST provide a function to: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this resource equivalent to the SDK's current representation of a resource? I.e. can the API caller access the resource to get its attributes, even if it doesn't intend to attach an entity? This idea of whether the resource should be in any way accessible to instrumentation or distributions via getters in the API or SDK has come up a number of times in opentelemetry-java. I've resisted adding such accessors because 1. couldn't get a solid use case that couldn't be solved more idiomatically through different means 2. the spec doesn't mention these types of accessors.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This is an option for the SDK. Due to binary compatibility reasons, the approach I took in the Java prototype was to have an Regarding solid use cases for read in the API path, I do not see a need either and this draft does not propose that.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👍 I do think the reuse of a I get that the goal is symmetry. Just as TracerProvider provides Tracers, MeterProvider provides Meters, LoggerProvider provides Loggers, it follows that a ResourceProvider would provide a Resource. And here, the Resource provided by ResourceProvider is an interface that instrumentations use to update entities. But Resource is now overloaded. Its now both a sort of record / pojo / struct style dumb data carrier, and the entity analog of a Meter / Logger / Tracer. I'm trying to brainstorm for alternative words that describe this concept.
Maybe something like "Resourcer"? The "er" suffix seems weird, but follows the pattern meter / tracer / logger, and avoid collision with the "Resource" SDK concept.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can use
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Well naming aside, I think these things make sense conceptually |
||
|
|
||
| - [Attach an Entity](#attach-an-entity) | ||
|
|
||
| ### Attach an Entity | ||
|
|
||
| The effect of calling this API is to attach (or update) an Entity | ||
| to the current `Resource`. | ||
|
Comment on lines
+64
to
+67
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we should be a lot more specific about the update behavior. Is it a full replacement or a merge? If it is a merge, is there a way to remove descriptive attributes? What is done with invalid entries (e.g. different identity but same entity type, which iirc is not allowed).
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was specific in the SDK but not the API. Agree we need to be explicit, I can move that detail to API, but in other areas of the spec, those details are left to the SDK specification. In practice, I think an implementor of the API would want those details in the API definition, so I'm fine having it defined in the API specification on expected behavior.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess it could be left to the SDK. It depends if we want to allow third party SDKs to have different behavior. The actual logic is implemented in the SDK so I think that's probably the right place for it. I talked myself out of my own comment. I'd leave it in SDK spec. |
||
|
|
||
| The API MUST accept the following parameters: | ||
|
|
||
| - `entity_type`: A string that uniquely identifies the type of Entity. | ||
| See [entity data model](./data-model.md#entity-data-model) for details. | ||
| - `identity`: Specifies the attributes which identify the entity. | ||
| This API MUST be structured to accept a variable number of | ||
| attributes, but must see at least one attribute. | ||
| - `description` (optional): Specifies the attributes which describe the | ||
| entity. This API MUST be structured to accept a variable number | ||
| of attributes, including none. | ||
| - `schema_url` (optional): Specifies the Schema URL that should be recorded in | ||
| the emitted telemetry. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| <!--- Hugo front matter used to generate the website version of this page: | ||
| linkTitle: SDK | ||
| ---> | ||
|
|
||
| # Entities SDK | ||
|
|
||
| **Status**: [Development](../document-status.md) | ||
|
|
||
| <!-- toc --> | ||
|
|
||
| - [Overview](#overview) | ||
| - [ResourceProvider](#resourceprovider) | ||
| * [ResourceProvider Creation](#resourceprovider-creation) | ||
| * [Active Resource](#active-resource) | ||
| * [Configuration](#configuration) | ||
| * [Shutdown](#shutdown) | ||
| * [ForceFlush](#forceflush) | ||
| - [Resource](#resource) | ||
| * [Attach an Entity](#attach-an-entity) | ||
| * [Flattened Resource](#flattened-resource) | ||
| * [Resolve conflicts in Schema URL](#resolve-conflicts-in-schema-url) | ||
|
|
||
| <!-- tocstop --> | ||
|
|
||
| ## Overview | ||
|
|
||
| Users of OpenTelemetry need a way for instrumentation interactions with the | ||
| OpenTelemetry API to actually produce telemetry. The OpenTelemetry Entities SDK | ||
| (henceforth referred to as the SDK) is an implementation of the OpenTelemetry | ||
| API that provides users with this functionally. | ||
|
|
||
| All language implementations of OpenTelemetry MUST provide an SDK. | ||
|
|
||
| The Entities SDK consists of these main components: | ||
|
|
||
| - [ResourceProvider](#resourceprovider) | ||
| - [Resource](#resource) | ||
|
|
||
| ## ResourceProvider | ||
|
|
||
| The ResourceProvider MUST provide a way to retrieve the current | ||
| [Resource](../resource/sdk.md) for use in there parts of the OpenTelemetry SDK. | ||
| This `Resource` MUST include all registered `Entity`s. | ||
|
|
||
| ### ResourceProvider Creation | ||
|
|
||
| The SDK SHOULD allow the creation of multiple independent `ResourceProvider`s. | ||
|
|
||
| ### Active Resource | ||
|
|
||
| This SDK MUST implement the | ||
| [Get the active Resource API](api.md#get-the-active-resource). | ||
|
|
||
| The SDK MUST provide exactly one `Resource` per `ResourceProvider` via this API. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If a
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. -- my updated thinking below --
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I need to formulate my thoughts more for a response. High level - We want entities not attached to the "active" resource. I.e. we may have another mechanism to report other resource (which should not be associated with telemetry from this SDK), but may report So this limitation of only one resource applies to the "active" resource for which the SDK is reporting telemetry against. |
||
|
|
||
| ### Configuration | ||
|
|
||
| TBD | ||
|
|
||
| ### Shutdown | ||
|
|
||
| TBD | ||
|
|
||
| ### ForceFlush | ||
|
|
||
| TBD | ||
|
|
||
| ## Resource | ||
|
|
||
| The SDK `Resource` is responsible for managing the current set of associated | ||
| `Entity`s on the `Resource. | ||
|
|
||
| The SDK MUST track associated `Entity`s on this `Resource`. | ||
|
|
||
| ### Attach an Entity | ||
|
|
||
| The `Resource` MUST implement the [Attach an Entity](api.md#attach-an-entity) | ||
| operation. This operation will try to associate a new `Entity` on the | ||
| `Resource`. | ||
|
|
||
| If the incoming `Entity`'s `entity_type` property is not found in the current | ||
| set of `Entity`s on `Resource`, then the new `Entity` is added to the set. | ||
|
|
||
| If the incoming `Entity`'s `entity_type` property matches an existing `Entity` | ||
| in the current set AND the `identity` attributes for these `Entity`s are | ||
| different, then the SDK MUST ignore the new entity. | ||
|
Comment on lines
+84
to
+86
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How would we handle the hw namespace where we would either have 1 hw entity definition or possibly 1 per hw.type if we could have fixed attributes? It gets trickier as we could have multiple of the 1 hw type ie 4 disks in a raid array.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Two things:
|
||
|
|
||
| If the incoming `Entity`'s `entity_type` property matches an existing `Entity` | ||
| in the current set AND the `identity` attributes for these `Entity`s are | ||
| the same but the `schema_url` is different, then the SDK MUST ignore the new | ||
| entity. | ||
|
|
||
| If the incoming `Entity`'s `entity_type` property matches an existing `Entity` | ||
| in the current set AND the `identity` attributes for these `Entity`s are | ||
| the same AND the `schema_url` is the same, then the SDK MUST add any new | ||
| attributes found in the `description` to the existing entity. Any new | ||
| `description` attributes with the same keys as existing `description` | ||
| attributes SHOULD replace previous values. | ||
|
|
||
| ### Flattened Resource | ||
|
|
||
| The SDK MUST return a flattened `Resource` for usage in exporters. This | ||
| flattened resource MUST provide the fields expected in the `Resource` | ||
| data model: | ||
|
|
||
| - `attributes`: The complete set of all attributes | ||
| - `schema_url`: The Schema URL that should be recorded for this resource. | ||
| See [resolve conflicts in schema url](#resolve-conflicts-in-schema-url). | ||
| - `entity_refs`: References to the `Entity` data model in `Resource`. | ||
| These each include: | ||
| - `type`: The type of the entity. | ||
| - `schema_url`: The Scheam URL that was recorded for this Entity. | ||
| - `id_keys`: The attribute keys for the Entity identity. Values are found | ||
| in the `Resource.attributes` property. | ||
| - `description_keys`: The attribute keys for the Entity description. Values | ||
| are found in the `Resource.attributes` property. | ||
|
|
||
| ### Resolve conflicts in Schema URL | ||
|
dyladan marked this conversation as resolved.
|
||
|
|
||
| If all detected entities and initial `Resource` have the same URL, then | ||
|
dyladan marked this conversation as resolved.
|
||
| this is chosen for the resulting `Resource`. | ||
|
|
||
| Otherwise, the Schema URL of the resulting `Resource` SHOULD be empty. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this always return the same resource? If so, what is the reason to have a
ResourceProvider? We may want to consider the case where there are multiple resources in an application (different per-signal or per-TracerProvider, etc).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now, yes.
I see a parallel in having multiple resources. Today if you want multiple resources, you have multiple
{Signal}Provider. You'd do the same forResourceProvider, and you register these with each{Signal}Provider}so It's a pairing.In Java prototype, I accomplished this via the
OpenTelemetryAPI -> see https://github.com/open-telemetry/opentelemetry-java/pull/7434/files#diff-fae9ff5f52df2c944624272d9bd01cf361c8bea7a172f6536f74c260231d47ffR76