Skip to content

fields.asciidoc

Latest commit

 

History

History
515 lines (257 loc) · 9.64 KB

fields.asciidoc

File metadata and controls

515 lines (257 loc) · 9.64 KB

Exported Fields

This document describes the fields that are exported by App-Logs. They are grouped in the following categories:

Summary

This is a specification of the "app-logs beat" message type that we use to store "application logs" written via a proper logging infrastructure implementation.

Note
 For Java applications running on CF, this is our logging support library, see https://github.com/SAP/cf-java-logging-support

Field names follow the convention that we’re not using camel case names, but rather use "_" as a delimiter between parts.

For any field that actually represents a measure (or metric for that matter) the name includes a descriptive suffix mnemonic that helps identifying the unit, e.g.:

  • "_ms" → milliseconds

  • "_b" → bytes

  • "_p" → percent

  • "_at" → date with time

  • "_ts" → timestamp

Context Fields

This group contains context information for the given application log message such that analytics can slice and dice information based on these criteria.

written_at

type: date

required: True

The human-readable date when the message was written.

written_ts

type: long

example: 1456820553816849408

required: True

The timestamp in nano-second precision when the message was written.

correlation_id

type: string

example: db2d002e-2702-41ec-66f5-c002a80a3d3f

required: True

A unique identifier that can be used to correlate multiple messages into a logical unit.

Currently, we use a UUID for that matter, but future versions may use a different key for this.

"-" indicates that no proper value could be provided.

Note
 As these type of messages can be written at any time and outside a "request context" it may happen that this correlation id does not match with any other. Logging implementations may provide a way to inherit correlation ids from request processing layers further up in the processing stack. If so, that id should be used.

request_id

example: e24a5963-95eb-4568-b1ae-81b67c41e99a

required: False

A unique identifier that can be used to correlate multiple messages to a request.

w3c_traceparent

type: string

example: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

required: False

The content of the W3C traceparent header as defined in https://www.w3.org/TR/trace-context/#traceparent-header. The traceparent allows correlation of logs to the request.

sap_passport

type: string

example: 2a54482a0300e60000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002a54482a

required: False

SAP Passport is an end to end tracing technology used in many SAP products. It is a binary format encoded in hex notation. Applications can add the whole SAP Passport in this field or give its constituents in the respective fields.

sap_passport_Action

type: string

required: False

This field can contain the action of an SAP Passport.

sap_passport_ActionType

type: string

required: False

This field can contain the action type of an SAP Passport.

sap_passport_ClientNumber

type: string

required: False

This field can contain the client number of an SAP Passport.

sap_passport_ConnectionCounter

type: integer

required: False

This field can contain the connection counter of an SAP Passport.

sap_passport_ConnectionId

type: string

required: False

This field can contain the connection id type of an SAP Passport.

sap_passport_ComponentName

type: string

required: False

This field can contain the component name of an SAP Passport.

sap_passport_ComponentType

type: string

required: False

This field can contain the component type of an SAP Passport.

sap_passport_PreviousComponentName

type: string

required: False

This field can contain the previous component name of an SAP Passport.

sap_passport_TraceFlags

type: string

required: False

This field can contain the trace flags of an SAP Passport.

sap_passport_TransactionId

type: string

required: False

This field can contain the transaction id of an SAP Passport.

sap_passport_RootContextId

type: string

required: False

This field can contain the root context id of an SAP Passport.

sap_passport_UserId

type: string

required: False

This field can contain the user id of an SAP Passport.

tenant_id

example: e24a5963-95eb-4568-b1ae-81b67c41e99a

required: False

A unique identifier that can be used to correlate multiple messages to a tenant.

tenant_subdomain

example: acme-inc

required: False

A unique identifier that represents the subdomain of the current tenant specific request url, e.g. acme-inc in https://acme-inc.eu10.cloud.alm.sap/home

component_id

type: string

example: 9e6f3ecf-def0-4baf-8fac-9339e61d5645

required: True

A unique identifier of the software component that has written the message. It may either be a "service" or an "application" that can be deployed to a container or VM, but it may not yet fully identify the actual "running instance" of that "component".

Note
 For applications in CF, this is the "application_id", but we may have more than one instance of that running.

component_name

type: string

example: my-fancy-component

required: False

A human-friendly, not necessarily unique, name representing the software component.

component_type

type: string

example: application

required: True

The type of the software component that has written the message.

Either "application" or "service"

component_instance

type: string

example: 7

required: True

CF allows us to scale out horizontally by running several instances of the same "component" in one container, this is identifying the instance. Defaults to "0".

container_id

type: string

example: ???

format: ???

required: False

The unique identifier of the container on which the software component instance is running.

Note
 If security/privacy is not an issue, an IP address would be ok.
Note
 CF sets an environment variable "INSTANCE_IP" that is currently used for that purpose

organization_id

type: string

example: 280437b3-dd8b-40b1-bbab-1f05a44345f8

required: True

The unique identifier of the Cloud Foundry organization to which the software component belongs.

"-" indicates that no proper value could be provided.

organization_name

type: string

example: acme

required: True

The human-readable name of the Cloud Foundry organization.

Note
 Most likely NOT unique!

"-" indicates that no proper value could be provided.

space_id

type: string

example: 280437b3-dd8b-40b1-bbab-1f05a44345f8

required: True

The unique identifier of the Cloud Foundry space to which the software component belongs.

"-" indicates that no proper value could be provided.

space_name

type: string

example: test

required: True

The human-readable name of the Cloud Foundry space.

Note
 Most likely NOT unique!

layer

type: string

example: [JAVA:BusinessLogicController]

required: True

The execution layer in the overall processing chain that has written the message.

type

type: string

required: True

The type of the message. To make the message self-contained this is the type tag. As we’re talking about log messages, it’s always "logs".

Application log message fields Fields

The individual fields of a log message. Should be familiar to anyone who ever used a logging library in Java…​

logger

type: string

example: com.sap.demo.shine.OrderController

required: True

The actual application entity that wrote the log message, e.g. the Java class name.

Note
 In Java, this is the name that was passed to the LoggerFactory when instantiating a Logger.

thread

type: string

example: http-nio-4655

required: False

The execution thread in which the log message has been written.

"-" indicates that no proper value could be provided.

level

type: string

example: INFO

required: True

The log "level" indicating the severity of the log message.

msg

type: string

example: This is a log messasge

required: True

The original log message that has been written by the application.

exception_type

type: string

example: java.lang.NullPointerException

required: False

The Java class name of the logged exception when available.

exception_message

type: string

example: Something went wrong

required: False

The message of the logged exception when available.

stacktrace

type: array

example: stacktrace: ["java.lang.NullPointerException","at com.sap.demo.exception.NullPointerException"]

required: False

Stacktrace information in case the message is reporting an exception.

Note
 This MUST NOT contain NEWLINE or TAB characters!

categories

type: string

required: False

A list of names to further categorize this log message.

#cf

type: object

example: "#cf": { "string": [ {"k":"some_key", "v":"some_value", "i": 0}, {"k":"other_key", "v":"other_value", "i": 1} ] }

required: False

An object containing collections of non-standard fields. The field "string" contains custom fields with key "k", value "v" and an index "i". The index can be used for field order during parsing.

Note
 As this is "custom" there are no predefined fields here!