diff --git a/.gitignore b/.gitignore index 1bc36e76d64c..6225d6722751 100644 --- a/.gitignore +++ b/.gitignore @@ -29,3 +29,4 @@ OBSDOCS-*_PLAN.md jira-token pull-jira-backlog.sh .jira-backlog/ +_work/ diff --git a/_topic_maps/_topic_map.yml b/_topic_maps/_topic_map.yml index 51fa8b40755f..8d49d6f8b02d 100644 --- a/_topic_maps/_topic_map.yml +++ b/_topic_maps/_topic_map.yml @@ -42,8 +42,12 @@ Name: Configuring logging Dir: configuring Distros: openshift-logging Topics: -- Name: Configuring the logging collector - File: cluster-logging-collector +- Name: Configure the collector + File: configure-the-collector +- Name: Configure collector network policies + File: configure-collector-network-policies +- Name: Monitor collector metrics + File: monitor-collector-metrics - Name: Configuring log forwarding File: configuring-log-forwarding - Name: Configuring inputs @@ -58,22 +62,38 @@ Topics: File: advanced-log-forwarding-configuration - Name: Forwarding to third-party systems File: forwarding-to-third-party-systems +- Name: Forwarding from pods and projects + File: forwarding-from-pods-and-projects +- Name: Forwarding logs over HTTP + File: forwarding-logs-over-http - Name: Forwarding to Google Cloud File: forwarding-to-google-cloud - Name: Forwarding to Splunk File: forwarding-to-splunk - Name: Forwarding to Kafka File: forwarding-to-kafka -- Name: Forwarding to Azure Monitor +- Name: About forwarding to Azure Monitor + File: about-forwarding-to-azure +- Name: Forward to Azure Monitor File: forwarding-to-azure - Name: Forwarding to Amazon CloudWatch File: forwarding-to-amazon-cloudwatch +- Name: Forward to Amazon CloudWatch from STS-enabled clusters + File: forwarding-to-amazon-cloudwatch-sts - Name: Collecting Kubernetes events File: collecting-kubernetes-events -- Name: Configuring the log store +- Name: About the LokiStack log store + File: about-the-lokistack-log-store +- Name: Selecting an object storage provider for Loki + File: selecting-loki-storage-provider +- Name: Configure the log store File: configuring-the-log-store -- Name: Tuning the log store - File: tuning-the-log-store +- Name: About log store tuning + File: about-log-store-tuning +- Name: Tuning Loki for performance and scalability + File: tuning-loki-for-performance-and-scalability +- Name: Configure Loki network policies + File: configuring-loki-network-policies - Name: Configuring LokiStack for OTLP File: configuring-lokistack-otlp - Name: OpenTelemetry data model @@ -171,6 +191,8 @@ Topics: File: default-logging-alerts - Name: Custom logging alerts File: custom-logging-alerts +- Name: Configuring Loki log-based alerts + File: configuring-loki-log-based-alerts #--- #Name: Performance and reliability tuning #Dir: performance_reliability @@ -196,6 +218,8 @@ Distros: openshift-logging Topics: - Name: Troubleshoot the installation File: troubleshoot-installation +- Name: Troubleshoot the collector + File: troubleshoot-the-collector - Name: Troubleshoot log forwarding File: troubleshoot-log-forwarding - Name: Troubleshoot logging alerts diff --git a/configuring/about-forwarding-to-azure.adoc b/configuring/about-forwarding-to-azure.adoc new file mode 100644 index 000000000000..e90e733c2ad3 --- /dev/null +++ b/configuring/about-forwarding-to-azure.adoc @@ -0,0 +1,36 @@ +:_mod-docs-content-type: ASSEMBLY +[id="about-forwarding-to-azure"] += About forwarding to Azure Monitor +include::_attributes/common-attributes.adoc[] +:context: about-forwarding-to-azure + +toc::[] + +[role="_abstract"] +Understand Azure Monitor Logs Ingestion API requirements, Data Collection Rules, authentication methods, and output format before configuring log forwarding. + +include::modules/about-azure-logs-ingestion-api.adoc[leveloffset=+1] + +include::modules/azure-logs-ingestion-dcr-requirements.adoc[leveloffset=+1] + +include::modules/azure-dcr-components.adoc[leveloffset=+2] + +include::modules/azure-dcr-required-information.adoc[leveloffset=+2] + +include::modules/azure-dcr-authentication-permissions.adoc[leveloffset=+2] + +include::modules/azure-logs-ingestion-authentication.adoc[leveloffset=+1] + +include::modules/azure-auth-client-secret.adoc[leveloffset=+2] + +include::modules/azure-auth-workload-identity.adoc[leveloffset=+2] + +include::modules/azure-auth-permissions.adoc[leveloffset=+2] + +include::modules/azure-logs-ingestion-output-format.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/forwarding-to-azure.adoc#forwarding-to-azure[Forward to Azure Monitor] +* link:https://learn.microsoft.com/en-us/azure/azure-monitor/logs/tutorial-logs-ingestion-portal[Create a data collection rule in Azure Monitor] diff --git a/configuring/about-log-store-tuning.adoc b/configuring/about-log-store-tuning.adoc new file mode 100644 index 000000000000..cbf246b229f1 --- /dev/null +++ b/configuring/about-log-store-tuning.adoc @@ -0,0 +1,23 @@ +:_mod-docs-content-type: ASSEMBLY +include::_attributes/common-attributes.adoc[] +[id="about-log-store-tuning"] += About log store tuning + +:context: about-log-store-tuning + +toc::[] + +[role="_abstract"] +You can configure advanced settings for Loki to optimize reliability, performance, scalability, security, and alerting in production environments. + +== Prerequisites + +* You have created a `LokiStack` custom resource. + +include::modules/understanding-lokistack-dashboards.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/tuning-loki-for-performance-and-scalability.adoc#tuning-loki-for-performance-and-scalability[Tuning Loki for performance and scalability] +* xref:../configuring/configuring-loki-network-policies.adoc#configuring-loki-network-policies[Configure Loki network policies] diff --git a/configuring/about-the-lokistack-log-store.adoc b/configuring/about-the-lokistack-log-store.adoc new file mode 100644 index 000000000000..dfdef18c4857 --- /dev/null +++ b/configuring/about-the-lokistack-log-store.adoc @@ -0,0 +1,26 @@ +:_mod-docs-content-type: ASSEMBLY +include::_attributes/common-attributes.adoc[] +[id="about-the-lokistack-log-store"] += About the LokiStack log store + +:context: about-the-lokistack-log-store + +toc::[] + +[role="_abstract"] +Loki is a horizontally scalable, highly available, multitenant log aggregation system that you can use to store and query logs from your {product-title} cluster. + +include::snippets/loki-statement-snip.adoc[leveloffset=+1] + +include::modules/loki-output-format.adoc[leveloffset=+1] + +include::modules/loki-sizing.adoc[leveloffset=+1] + +include::modules/logging-identity-federation.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/selecting-loki-storage-provider.adoc#selecting-loki-storage-provider[Selecting an object storage provider for Loki] +* xref:../configuring/configuring-the-log-store.adoc#configuring-the-log-store[Configuring the log store] +* xref:../configuring/about-log-store-tuning.adoc#about-log-store-tuning[About log store tuning] diff --git a/configuring/advanced-log-forwarding-configuration.adoc b/configuring/advanced-log-forwarding-configuration.adoc index 337802cd047a..e0d33b6ab6bc 100644 --- a/configuring/advanced-log-forwarding-configuration.adoc +++ b/configuring/advanced-log-forwarding-configuration.adoc @@ -18,3 +18,9 @@ include::modules/setting-collector-log-level.adoc[leveloffset=+1] include::modules/setting-collector-resource-limits.adoc[leveloffset=+1] include::modules/creating-logfilesmetricexporter.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/configuring-log-forwarding.adoc#configuring-log-forwarding[Configuring log forwarding] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] diff --git a/configuring/configure-collector-network-policies.adoc b/configuring/configure-collector-network-policies.adoc new file mode 100644 index 000000000000..f9f946907c4d --- /dev/null +++ b/configuring/configure-collector-network-policies.adoc @@ -0,0 +1,28 @@ +:_mod-docs-content-type: ASSEMBLY +[id="configure-collector-network-policies"] += Configure collector network policies +include::_attributes/common-attributes.adoc[] +:context: configure-collector-network-policies + +toc::[] + +[role="_abstract"] +You can configure network policies to enable the collector to operate in clusters with restrictive network configurations. + +include::modules/collector-network-policy.adoc[leveloffset=+1] + +include::modules/configuring-network-policy-rule-set-for-a-collector.adoc[leveloffset=+1] + +include::modules/logfilemetricexporter-network-policy.adoc[leveloffset=+1] + +include::modules/configuring-network-policy-rule-set-for-logfilemetricexporter.adoc[leveloffset=+1] + +include::modules/creating-an-adminnetworkpolicy-rule-for-collector-network-policy.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/configure-the-collector.adoc#configure-the-collector[Configure the collector] +* xref:../configuring/monitor-collector-metrics.adoc#monitor-collector-metrics[Monitor collector metrics] +* link:https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html-single/network_security/index#network-policy-apis[Understanding network policy APIs] +* link:https://docs.redhat.com/en/documentation/openshift_container_platform/4.19/html-single/network_security/index#ovn-k-anp[OVN-Kubernetes AdminNetworkPolicy] diff --git a/configuring/configure-the-collector.adoc b/configuring/configure-the-collector.adoc new file mode 100644 index 000000000000..7575831be484 --- /dev/null +++ b/configuring/configure-the-collector.adoc @@ -0,0 +1,50 @@ +:_mod-docs-content-type: ASSEMBLY +[id="configure-the-collector"] += Configure the collector +include::_attributes/common-attributes.adoc[] +:context: configure-the-collector + +toc::[] + +[role="_abstract"] +You can configure the log collector to receive logs from external sources, manage pod rollout strategy, and set resource limits. + +[id="cluster-logging-collector-input-receivers_{context}"] +== Configure input receivers + +The {clo} deploys a service for each configured input receiver so that clients can write to the collector. This service exposes the port specified for the input receiver. For log forwarder `ClusterLogForwarder` CR deployments, the service name is in the `-` format. + +[IMPORTANT] +==== +Receiver inputs are only supported when logs originate from a Red{nbsp}Hat-supported product that is installed on the same cluster as the {CLO}, or when logging is installed on {hcp}. +==== + +The collector supports two types of receiver inputs: + +HTTP receiver:: +Receives and parses audit logs. Use this receiver when: ++ +* Logging is installed on {hcp}. +* Logs originate from a Red{nbsp}Hat-supported product that is installed on the same cluster as the {CLO}. For example, {VirtProductName}. + +Syslog receiver:: +Receives and parses infrastructure logs in journal format. Use this receiver when: ++ +* Logs originate from a Red{nbsp}Hat-supported product that is installed on the same cluster as the {CLO}. For example: +** Red Hat OpenStack Services on OpenShift (RHOSO) +** {VirtProductName} + +include::modules/configuring-the-collector-to-receive-audit-logs-as-an-http-server.adoc[leveloffset=+2] + +include::modules/configuring-the-collector-to-listen-for-connections-as-a-syslog-server.adoc[leveloffset=+2] + +include::modules/configuring-pod-rollout-strategy.adoc[leveloffset=+1] + +include::modules/cluster-logging-collector-limits.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/configure-collector-network-policies.adoc#configure-collector-network-policies[Configure collector network policies] +* xref:../configuring/monitor-collector-metrics.adoc#monitor-collector-metrics[Monitor collector metrics] +* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] diff --git a/configuring/configuring-filters.adoc b/configuring/configuring-filters.adoc index fba373a9bcfd..8f0733c2e141 100644 --- a/configuring/configuring-filters.adoc +++ b/configuring/configuring-filters.adoc @@ -22,3 +22,13 @@ include::modules/adding-a-prune-filter.adoc[leveloffset=+2] include::modules/about-multi-line-exceptions.adoc[leveloffset=+1] include::modules/enabling-multi-line-exception-detection.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/configuring-log-forwarding.adoc#configuring-log-forwarding[Configuring log forwarding] +* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] +* xref:../configuring/configuring-outputs.adoc#configuring-outputs[Configuring outputs] +* xref:../configuring/configuring-pipelines.adoc#configuring-pipelines[Configuring pipelines] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] +* link:https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/#audit-policy[Kubernetes audit policy] diff --git a/configuring/configuring-inputs.adoc b/configuring/configuring-inputs.adoc index 89146795f4c4..a74deb4a16f1 100644 --- a/configuring/configuring-inputs.adoc +++ b/configuring/configuring-inputs.adoc @@ -18,3 +18,12 @@ include::modules/input-spec-filter-labels-expressions.adoc[leveloffset=+1] include::modules/input-spec-filter-audit-infrastructure.adoc[leveloffset=+1] include::modules/adding-audit-log-collection.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/configuring-log-forwarding.adoc#configuring-log-forwarding[Configuring log forwarding] +* xref:../configuring/configuring-filters.adoc#configuring-filters[Configuring filters] +* xref:../configuring/configuring-outputs.adoc#configuring-outputs[Configuring outputs] +* xref:../configuring/configuring-pipelines.adoc#configuring-pipelines[Configuring pipelines] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] diff --git a/configuring/configuring-log-forwarding.adoc b/configuring/configuring-log-forwarding.adoc index 864a2af3070c..ebdab8b5dc18 100644 --- a/configuring/configuring-log-forwarding.adoc +++ b/configuring/configuring-log-forwarding.adoc @@ -30,151 +30,14 @@ include::modules/clf-about-status-conditions.adoc[leveloffset=+2] include::modules/verifying-log-collection.adoc[leveloffset=+2] -// --- Phase 4: Configuring the Pipeline (I/O Flow) --- - -[id="configuring-inputs_{context}"] -== Configuring inputs - -include::modules/clf-inputs-reference.adoc[leveloffset=+2] - -include::modules/input-spec-filter-namespace-container.adoc[leveloffset=+2] - -include::modules/input-spec-filter-labels-expressions.adoc[leveloffset=+2] - -include::modules/input-spec-filter-audit-infrastructure.adoc[leveloffset=+2] - -include::modules/adding-audit-log-collection.adoc[leveloffset=+2] - -[id="configuring-filters_{context}"] -== Configuring filters - -include::modules/clf-filters-reference.adoc[leveloffset=+2] - -include::modules/logging-audit-log-filtering.adoc[leveloffset=+2] - -include::modules/filtering-collected-logs.adoc[leveloffset=+2] - -include::modules/adding-a-drop-filter.adoc[leveloffset=+3] - -include::modules/adding-a-prune-filter.adoc[leveloffset=+3] - -include::modules/about-multi-line-exceptions.adoc[leveloffset=+2] - -include::modules/enabling-multi-line-exception-detection.adoc[leveloffset=+2] - -[id="configuring-outputs_{context}"] -== Configuring outputs - -include::modules/clf-outputs-reference.adoc[leveloffset=+2] - -[id="configuring-pipelines_{context}"] -== Configuring pipelines - -include::modules/clf-pipelines-reference.adoc[leveloffset=+2] - -// --- Phase 5: Advanced Configuration --- - -[id="advanced-configuration_{context}"] -== Advanced configuration - -include::modules/configuring-rate-limits.adoc[leveloffset=+2] - -include::modules/logging-delivery-tuning.adoc[leveloffset=+2] - -include::modules/setting-collector-log-level.adoc[leveloffset=+2] - -include::modules/setting-collector-resource-limits.adoc[leveloffset=+2] - -include::modules/creating-logfilesmetricexporter.adoc[leveloffset=+2] - -// --- Phase 6: Troubleshooting --- - -include::modules/troubleshooting-log-collection.adoc[leveloffset=+1] - -// --- Phase 7: Third-Party Output Configuration (Detailed Procedures) --- - -[id="forwarding-to-third-party-destinations_{context}"] -== Forwarding to third-party destinations - -include::modules/logging-external-destination-compatibility.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forwarding-about.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-gcp.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-gcp-wif.adoc[leveloffset=+2] - -include::modules/configuring-otlp-output.adoc[leveloffset=+2] - -[id="forwarding-logs-to-splunk_{context}"] -=== Forwarding logs to Splunk - -include::modules/logging-forward-splunk.adoc[leveloffset=+3] - -include::modules/default-splunk-metadata-key-values.adoc[leveloffset=+3] - -include::modules/logging-http-forward.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-loki.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-kafka.adoc[leveloffset=+2] - -include::modules/about-azure-logs-ingestion-api.adoc[leveloffset=+2] - -include::modules/azure-logs-ingestion-dcr-requirements.adoc[leveloffset=+2] - -include::modules/azure-dcr-components.adoc[leveloffset=+3] - -include::modules/azure-dcr-required-information.adoc[leveloffset=+3] - -include::modules/azure-dcr-authentication-permissions.adoc[leveloffset=+3] - -include::modules/azure-logs-ingestion-authentication.adoc[leveloffset=+2] - -include::modules/azure-auth-client-secret.adoc[leveloffset=+3] - -include::modules/azure-auth-workload-identity.adoc[leveloffset=+3] - -include::modules/azure-auth-permissions.adoc[leveloffset=+3] - -include::modules/logging-forwarding-azure-logs-ingestion.adoc[leveloffset=+2] - -include::modules/logging-forwarding-azure.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-project.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-logs-from-application-pods.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-syslog.adoc[leveloffset=+3] - -include::modules/cluster-logging-collector-log-forward-syslog-rfc5424-defaults.adoc[leveloffset=+3] - -include::modules/cluster-logging-collector-log-forward-syslog-log-source.adoc[leveloffset=+3] - -include::modules/cluster-logging-collector-log-forward-cloudwatch.adoc[leveloffset=+2] - -include::modules/forwarding-logs-to-amazon-s3-endpoint.adoc[leveloffset=+2] - -[id="forwarding-logs-to-amazon-cloudwatch-from-sts-enabled-clusters_{context}"] -=== Forwarding logs to Amazon CloudWatch from STS-enabled clusters - -include::modules/creating-an-aws-role.adoc[leveloffset=+3] - -include::modules/cluster-logging-collector-log-forward-secret-cloudwatch.adoc[leveloffset=+3] - -include::modules/cluster-logging-collector-log-forward-sts-cloudwatch.adoc[leveloffset=+3] - -include::modules/forwarding-logs-to-amazon-s3-endpoint-from-sts-enabled-clusters.adoc[leveloffset=+3] - -include::modules/cross-account-log-forwarding-with-assumerole.adoc[leveloffset=+3] - -include::modules/configuring-aws-iam-for-cross-account-access.adoc[leveloffset=+3] - -include::modules/example-clusterlogforwarder-configuration-for-cross-account-forwarding.adoc[leveloffset=+3] - -include::modules/cross-account-log-forwarding-troubleshooting.adoc[leveloffset=+3] - [role="_additional-resources"] .Additional resources +* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] +* xref:../configuring/configuring-filters.adoc#configuring-filters[Configuring filters] +* xref:../configuring/configuring-outputs.adoc#configuring-outputs[Configuring outputs] +* xref:../configuring/configuring-pipelines.adoc#configuring-pipelines[Configuring pipelines] +* xref:../configuring/advanced-log-forwarding-configuration.adoc#advanced-log-forwarding-configuration[Advanced log forwarding configuration] +* xref:../configuring/forwarding-to-third-party-systems.adoc#forwarding-to-third-party-systems[Forwarding to third-party systems] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] * xref:../configuring/opentelemetry-data-model.adoc#opentelemetry-data-model[OpenTelemetry data model] diff --git a/configuring/configuring-loki-network-policies.adoc b/configuring/configuring-loki-network-policies.adoc new file mode 100644 index 000000000000..8a2fcdfdfaed --- /dev/null +++ b/configuring/configuring-loki-network-policies.adoc @@ -0,0 +1,25 @@ +:_mod-docs-content-type: ASSEMBLY +include::_attributes/common-attributes.adoc[] +[id="configuring-loki-network-policies"] += Configure Loki network policies + +:context: configuring-loki-network-policies + +toc::[] + +[role="_abstract"] +You can deploy and manage network policies that restrict communications to and from Loki components to enhance security. + +include::modules/about-loki-network-security.adoc[leveloffset=+1] + +include::modules/configuring-a-network-policy-for-loki.adoc[leveloffset=+2] + +include::modules/loki-networkpolicy-resources.adoc[leveloffset=+2] + +include::modules/integrating-loki-network-policy-with-external-systems.adoc[leveloffset=+2] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/about-log-store-tuning.adoc#about-log-store-tuning[About log store tuning] +* xref:../configuring/tuning-loki-for-performance-and-scalability.adoc#tuning-loki-for-performance-and-scalability[Tuning Loki for performance and scalability] diff --git a/configuring/configuring-outputs.adoc b/configuring/configuring-outputs.adoc index 18f6f92b766b..bba13e5cb32c 100644 --- a/configuring/configuring-outputs.adoc +++ b/configuring/configuring-outputs.adoc @@ -10,3 +10,13 @@ toc::[] Outputs define where logs are sent after collection and filtering. Each output represents a destination system such as LokiStack, CloudWatch, Kafka, or Splunk. Configure outputs to match your destination's authentication requirements, URL structure, and protocol specifications. include::modules/clf-outputs-reference.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/configuring-log-forwarding.adoc#configuring-log-forwarding[Configuring log forwarding] +* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] +* xref:../configuring/configuring-filters.adoc#configuring-filters[Configuring filters] +* xref:../configuring/configuring-pipelines.adoc#configuring-pipelines[Configuring pipelines] +* xref:../configuring/forwarding-to-third-party-systems.adoc#forwarding-to-third-party-systems[Forwarding to third-party systems] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] diff --git a/configuring/configuring-pipelines.adoc b/configuring/configuring-pipelines.adoc index beac483f61a3..0c4d97b48183 100644 --- a/configuring/configuring-pipelines.adoc +++ b/configuring/configuring-pipelines.adoc @@ -10,3 +10,12 @@ toc::[] Pipelines connect inputs to outputs, optionally applying filters in between. Each pipeline defines a routing path for specific log types through your logging infrastructure. Configure pipelines to route application logs to one destination, infrastructure logs to another, and audit logs to a compliance system. include::modules/clf-pipelines-reference.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/configuring-log-forwarding.adoc#configuring-log-forwarding[Configuring log forwarding] +* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] +* xref:../configuring/configuring-filters.adoc#configuring-filters[Configuring filters] +* xref:../configuring/configuring-outputs.adoc#configuring-outputs[Configuring outputs] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] diff --git a/configuring/configuring-the-log-store.adoc b/configuring/configuring-the-log-store.adoc index ba16a7b19733..c369df1567dd 100644 --- a/configuring/configuring-the-log-store.adoc +++ b/configuring/configuring-the-log-store.adoc @@ -1,101 +1,32 @@ :_mod-docs-content-type: ASSEMBLY include::_attributes/common-attributes.adoc[] -[id="configuring-lokistack-storage"] -= Configuring the log store +[id="configuring-the-log-store"] += Configure the log store :context: configuring-the-log-store toc::[] +[role="_abstract"] You can configure a `LokiStack` custom resource (CR) to store application, audit, and infrastructure-related logs. -include::snippets/loki-statement-snip.adoc[leveloffset=+1] +.Prerequisites -include::modules/loki-output-format.adoc[leveloffset=+1] +* You have configured object storage for Loki. -// Loki sizing -include::modules/loki-sizing.adoc[leveloffset=+1] - - -// Loki object storage -include::modules/logging-loki-storage.adoc[leveloffset=+1] - -// create object storage -include::modules/logging-loki-storage-aws.adoc[leveloffset=+2] - -include::modules/logging-loki-storage-azure.adoc[leveloffset=+2] - -include::modules/logging-loki-storage-azure-entra.adoc[leveloffset=+2] - -include::modules/logging-loki-storage-gcp.adoc[leveloffset=+2] - -include::modules/logging-loki-storage-s3-compatible.adoc[leveloffset=+2] - -include::modules/logging-loki-storage-odf.adoc[leveloffset=+2] - -include::modules/logging-loki-storage-swift.adoc[leveloffset=+2] - -//Loki storage STS include::modules/about-loki-storage-sts.adoc[leveloffset=+1] -include::modules/logging-identity-federation.adoc[leveloffset=+1] - include::modules/logging-create-loki-cr-console.adoc[leveloffset=+1,tag=!pre-5.9] -include::modules/loki-create-object-storage-secret-cli.adoc[leveloffset=+1] +include::modules/creating-log-forwarding-secret-cli.adoc[leveloffset=+1] include::modules/logging-loki-log-access.adoc[leveloffset=+2,tag=!NetObservMode] include::modules/logging-creating-new-group-cluster-admin-user-role.adoc[leveloffset=+2] -//Enhanced reliability and performance -[id="performance_{context}"] -== Enhanced reliability and performance for Loki - -Use the following configurations to ensure reliability and efficiency of Loki in production. - -include::modules/loki-pod-placement.adoc[leveloffset=+2] - -include::modules/logging-loki-reliability-hardening.adoc[leveloffset=+2] - -include::modules/logging-loki-retention.adoc[leveloffset=+2] - -include::modules/loki-memberlist-ip.adoc[leveloffset=+2] - -include::modules/loki-restart-hardening.adoc[leveloffset=+2] -//include::modules/enabling-automatic-stream-sharding.adoc[leveloffset=+2] - -//Advanced deployment and scalability -[id="advanced_{context}"] -== Advanced deployment and scalability for Loki - -You can configure high availability, scalability, and error handling for Loki. - -include::modules/loki-zone-aware-replication.adoc[leveloffset=+2] - -include::modules/loki-zone-fail-recovery.adoc[leveloffset=+2] - -include::modules/troubleshoot-loki-rate-limit-errors.adoc[leveloffset=+2] - -[id="loki-network-policies-for-added-security_{context}"] -== Loki network policies for added security - -{loki-op} can deploy and manage a set of network policies that restrict the communications to and from the Loki components to enhance security. - -include::modules/loki-network-policies.adoc[leveloffset=+2] - -include::modules/configuring-a-network-policy-for-loki.adoc[leveloffset=+2] - -include::modules/loki-networkpolicy-resources.adoc[leveloffset=+2] - -include::modules/integrating-loki-network-policy-with-external-systems.adoc[leveloffset=+2] - -//log based alerts -[id="log-based-alerts_{context}"] -== Log-based alerts for Loki - -You can configure log-based alerts for Loki by creating an `AlertingRule` custom resource (CR). - -include::modules/loki-rbac-rules-permissions.adoc[leveloffset=+2] +[role="_additional-resources"] +.Additional resources -include::modules/enabling-loki-alerts.adoc[leveloffset=+2] +* xref:../configuring/about-the-lokistack-log-store.adoc#about-the-lokistack-log-store[About the LokiStack log store] +* xref:../configuring/selecting-loki-storage-provider.adoc#selecting-loki-storage-provider[Selecting an object storage provider for Loki] +* xref:../configuring/about-log-store-tuning.adoc#about-log-store-tuning[About log store tuning] diff --git a/configuring/forwarding-from-pods-and-projects.adoc b/configuring/forwarding-from-pods-and-projects.adoc new file mode 100644 index 000000000000..20bed466403a --- /dev/null +++ b/configuring/forwarding-from-pods-and-projects.adoc @@ -0,0 +1,30 @@ +:_mod-docs-content-type: ASSEMBLY +[id="forwarding-from-pods-and-projects"] += Forwarding from pods and projects +include::_attributes/common-attributes.adoc[] +:context: forwarding-from-pods-and-projects + +toc::[] + +[role="_abstract"] +You can configure log forwarding to send logs from specific pods or projects, allowing you to route logs based on application labels or namespace ownership. + +include::modules/cluster-logging-collector-log-forward-logs-from-application-pods.adoc[leveloffset=+1] + +include::modules/cluster-logging-collector-log-forward-syslog.adoc[leveloffset=+2] + +include::modules/cluster-logging-collector-log-forward-syslog-rfc5424-defaults.adoc[leveloffset=+2] + +include::modules/syslog-output-format.adoc[leveloffset=+2] + +include::modules/cluster-logging-collector-log-forward-syslog-log-source.adoc[leveloffset=+2] + +include::modules/cluster-logging-collector-log-forward-project.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/forwarding-to-third-party-systems.adoc#forwarding-to-third-party-systems[Forwarding to third party systems] +* xref:../configuring/forwarding-logs-over-http.adoc#forwarding-logs-over-http[Forwarding logs over HTTP] +* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] diff --git a/configuring/forwarding-logs-over-http.adoc b/configuring/forwarding-logs-over-http.adoc new file mode 100644 index 000000000000..f91c856b4367 --- /dev/null +++ b/configuring/forwarding-logs-over-http.adoc @@ -0,0 +1,21 @@ +:_mod-docs-content-type: ASSEMBLY +[id="forwarding-logs-over-http"] += Forwarding logs over HTTP +include::_attributes/common-attributes.adoc[] +:context: forwarding-logs-over-http + +toc::[] + +[role="_abstract"] +You can forward logs to generic HTTP endpoints by using the HTTP output type, which supports custom authentication, headers, and batch configuration. + +include::modules/logging-http-forward.adoc[leveloffset=+1] + +include::modules/http-output-format.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/forwarding-to-third-party-systems.adoc#forwarding-to-third-party-systems[Forwarding to third party systems] +* xref:../configuring/configuring-outputs.adoc#configuring-outputs[Configuring outputs] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] diff --git a/configuring/forwarding-to-amazon-cloudwatch-sts.adoc b/configuring/forwarding-to-amazon-cloudwatch-sts.adoc new file mode 100644 index 000000000000..cc69c5b2b17b --- /dev/null +++ b/configuring/forwarding-to-amazon-cloudwatch-sts.adoc @@ -0,0 +1,31 @@ +:_mod-docs-content-type: ASSEMBLY +[id="forwarding-to-amazon-cloudwatch-sts"] += Forward to Amazon CloudWatch from STS-enabled clusters +include::_attributes/common-attributes.adoc[] +:context: forwarding-to-amazon-cloudwatch-sts + +toc::[] + +[role="_abstract"] +You can forward logs to Amazon CloudWatch and Amazon S3 from STS-enabled clusters, including cross-account configurations using AssumeRole. + +include::modules/creating-an-aws-role.adoc[leveloffset=+1] + +include::modules/cluster-logging-collector-log-forward-secret-cloudwatch.adoc[leveloffset=+1] + +include::modules/cluster-logging-collector-log-forward-sts-cloudwatch.adoc[leveloffset=+1] + +include::modules/forwarding-logs-to-amazon-s3-endpoint-from-sts-enabled-clusters.adoc[leveloffset=+1] + +include::modules/cross-account-log-forwarding-with-assumerole.adoc[leveloffset=+1] + +include::modules/configuring-aws-iam-for-cross-account-access.adoc[leveloffset=+1] + +include::modules/example-clusterlogforwarder-configuration-for-cross-account-forwarding.adoc[leveloffset=+1] + +include::modules/cross-account-log-forwarding-troubleshooting.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/forwarding-to-amazon-cloudwatch.adoc#forwarding-to-amazon-cloudwatch[Forwarding to Amazon CloudWatch] diff --git a/configuring/forwarding-to-amazon-cloudwatch.adoc b/configuring/forwarding-to-amazon-cloudwatch.adoc index 031987476d83..9b13f87a65fa 100644 --- a/configuring/forwarding-to-amazon-cloudwatch.adoc +++ b/configuring/forwarding-to-amazon-cloudwatch.adoc @@ -7,7 +7,7 @@ include::_attributes/common-attributes.adoc[] toc::[] [role="_abstract"] -You can forward logs to Amazon CloudWatch and Amazon S3, including from STS-enabled clusters and cross-account configurations. +You can forward logs to Amazon CloudWatch and Amazon S3 using AWS authentication credentials. include::modules/cluster-logging-collector-log-forward-cloudwatch.adoc[leveloffset=+1] @@ -15,21 +15,7 @@ include::modules/cloudwatch-output-format.adoc[leveloffset=+1] include::modules/forwarding-logs-to-amazon-s3-endpoint.adoc[leveloffset=+1] -[id="forwarding-logs-to-amazon-cloudwatch-from-sts-enabled-clusters_{context}"] -== Forwarding logs to Amazon CloudWatch from STS-enabled clusters +[role="_additional-resources"] +.Additional resources -include::modules/creating-an-aws-role.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-secret-cloudwatch.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-sts-cloudwatch.adoc[leveloffset=+2] - -include::modules/forwarding-logs-to-amazon-s3-endpoint-from-sts-enabled-clusters.adoc[leveloffset=+2] - -include::modules/cross-account-log-forwarding-with-assumerole.adoc[leveloffset=+2] - -include::modules/configuring-aws-iam-for-cross-account-access.adoc[leveloffset=+2] - -include::modules/example-clusterlogforwarder-configuration-for-cross-account-forwarding.adoc[leveloffset=+2] - -include::modules/cross-account-log-forwarding-troubleshooting.adoc[leveloffset=+2] +* xref:../configuring/forwarding-to-amazon-cloudwatch-sts.adoc#forwarding-to-amazon-cloudwatch-sts[Forward to Amazon CloudWatch from STS-enabled clusters] diff --git a/configuring/forwarding-to-azure.adoc b/configuring/forwarding-to-azure.adoc index 70605132f172..e53f124f88b5 100644 --- a/configuring/forwarding-to-azure.adoc +++ b/configuring/forwarding-to-azure.adoc @@ -1,6 +1,6 @@ :_mod-docs-content-type: ASSEMBLY [id="forwarding-to-azure"] -= Forwarding to Azure Monitor += Forward to Azure Monitor include::_attributes/common-attributes.adoc[] :context: forwarding-to-azure @@ -9,26 +9,11 @@ toc::[] [role="_abstract"] You can forward logs to Azure Monitor by using the Logs Ingestion API or the deprecated Data Collector API. -include::modules/about-azure-logs-ingestion-api.adoc[leveloffset=+1] - -include::modules/azure-logs-ingestion-dcr-requirements.adoc[leveloffset=+1] - -include::modules/azure-dcr-components.adoc[leveloffset=+2] - -include::modules/azure-dcr-required-information.adoc[leveloffset=+2] - -include::modules/azure-dcr-authentication-permissions.adoc[leveloffset=+2] - -include::modules/azure-logs-ingestion-authentication.adoc[leveloffset=+1] - -include::modules/azure-auth-client-secret.adoc[leveloffset=+2] - -include::modules/azure-auth-workload-identity.adoc[leveloffset=+2] - -include::modules/azure-auth-permissions.adoc[leveloffset=+2] - include::modules/logging-forwarding-azure-logs-ingestion.adoc[leveloffset=+1] -include::modules/azure-logs-ingestion-output-format.adoc[leveloffset=+1] - include::modules/logging-forwarding-azure.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/about-forwarding-to-azure.adoc#about-forwarding-to-azure[About forwarding to Azure Monitor] diff --git a/configuring/forwarding-to-third-party-systems.adoc b/configuring/forwarding-to-third-party-systems.adoc index 6b886df42a76..f12a054c7d8b 100644 --- a/configuring/forwarding-to-third-party-systems.adoc +++ b/configuring/forwarding-to-third-party-systems.adoc @@ -1,6 +1,6 @@ :_mod-docs-content-type: ASSEMBLY [id="forwarding-to-third-party-systems"] -= Forwarding to third-party systems += Forwarding to third party systems include::_attributes/common-attributes.adoc[] :context: forwarding-to-third-party-systems @@ -13,22 +13,16 @@ include::modules/logging-external-destination-compatibility.adoc[leveloffset=+1] include::modules/cluster-logging-collector-log-forwarding-about.adoc[leveloffset=+1] -include::modules/configuring-otlp-output.adoc[leveloffset=+1] - -include::modules/logging-http-forward.adoc[leveloffset=+1] +include::modules/creating-log-forwarding-secret-cli.adoc[leveloffset=+1] -include::modules/http-output-format.adoc[leveloffset=+1] +include::modules/configuring-otlp-output.adoc[leveloffset=+1] include::modules/cluster-logging-collector-log-forward-loki.adoc[leveloffset=+1] -include::modules/cluster-logging-collector-log-forward-logs-from-application-pods.adoc[leveloffset=+1] - -include::modules/cluster-logging-collector-log-forward-syslog.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-syslog-rfc5424-defaults.adoc[leveloffset=+2] - -include::modules/syslog-output-format.adoc[leveloffset=+2] - -include::modules/cluster-logging-collector-log-forward-syslog-log-source.adoc[leveloffset=+2] +[role="_additional-resources"] +.Additional resources -include::modules/cluster-logging-collector-log-forward-project.adoc[leveloffset=+1] +* xref:../configuring/forwarding-logs-over-http.adoc#forwarding-logs-over-http[Forwarding logs over HTTP] +* xref:../configuring/forwarding-from-pods-and-projects.adoc#forwarding-from-pods-and-projects[Forwarding from pods and projects] +* xref:../configuring/configuring-outputs.adoc#configuring-outputs[Configuring outputs] +* xref:../troubleshooting/troubleshoot-log-forwarding.adoc#troubleshoot-log-forwarding[Troubleshooting log forwarding] diff --git a/configuring/monitor-collector-metrics.adoc b/configuring/monitor-collector-metrics.adoc new file mode 100644 index 000000000000..f1db47f5ecdd --- /dev/null +++ b/configuring/monitor-collector-metrics.adoc @@ -0,0 +1,30 @@ +:_mod-docs-content-type: ASSEMBLY +[id="monitor-collector-metrics"] += Monitor collector metrics +include::_attributes/common-attributes.adoc[] +:context: monitor-collector-metrics + +toc::[] + +[role="_abstract"] +You can monitor and configure collector metrics to understand monitoring impact and reduce cardinality in multitenant environments. + +include::modules/collector-metrics-cardinality-impact.adoc[leveloffset=+1] + +include::modules/configuring-collector-metrics-profiles.adoc[leveloffset=+1] + +include::modules/collector-minimal-metrics-reference.adoc[leveloffset=+1] + +include::modules/best-practices-multitenant-logging.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/configure-the-collector.adoc#configure-the-collector[Configure the collector] +* xref:../troubleshooting/troubleshoot-the-collector.adoc#troubleshoot-the-collector[Troubleshoot the collector] +* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] +* xref:../configuring/configuring-filters.adoc#configuring-filters[Configuring filters] +* link:https://docs.openshift.com/container-platform/latest/observability/monitoring/configuring-the-monitoring-stack.html#creating-cluster-monitoring-configmap_configuring-the-monitoring-stack[Configuring the monitoring stack] +* link:https://github.com/openshift/enhancements/blob/master/enhancements/monitoring/metrics-collection-profiles.md[Metrics collection profiles enhancement] +* link:https://prometheus.io/docs/practices/naming/#labels[Prometheus metric and label naming best practices] +* link:https://www.robustperception.io/cardinality-is-key[Understanding cardinality in Prometheus] diff --git a/configuring/selecting-loki-storage-provider.adoc b/configuring/selecting-loki-storage-provider.adoc new file mode 100644 index 000000000000..047609d16cad --- /dev/null +++ b/configuring/selecting-loki-storage-provider.adoc @@ -0,0 +1,33 @@ +:_mod-docs-content-type: ASSEMBLY +include::_attributes/common-attributes.adoc[] +[id="selecting-loki-storage-provider"] += Selecting an object storage provider for Loki + +:context: selecting-loki-storage-provider + +toc::[] + +[role="_abstract"] +You must configure object storage for Loki before creating a LokiStack custom resource. Loki supports multiple storage providers including AWS S3, Azure Blob Storage, Google Cloud Storage, and S3-compatible solutions. + +include::modules/logging-loki-storage.adoc[leveloffset=+1] + +include::modules/logging-loki-storage-aws.adoc[leveloffset=+1] + +include::modules/logging-loki-storage-azure.adoc[leveloffset=+1] + +include::modules/logging-loki-storage-azure-entra.adoc[leveloffset=+1] + +include::modules/logging-loki-storage-gcp.adoc[leveloffset=+1] + +include::modules/logging-loki-storage-s3-compatible.adoc[leveloffset=+1] + +include::modules/logging-loki-storage-odf.adoc[leveloffset=+1] + +include::modules/logging-loki-storage-swift.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/about-the-lokistack-log-store.adoc#about-the-lokistack-log-store[About the LokiStack log store] +* xref:../configuring/configuring-the-log-store.adoc#configuring-the-log-store[Configure the log store] diff --git a/configuring/tuning-loki-for-performance-and-scalability.adoc b/configuring/tuning-loki-for-performance-and-scalability.adoc new file mode 100644 index 000000000000..fb0129d43dcb --- /dev/null +++ b/configuring/tuning-loki-for-performance-and-scalability.adoc @@ -0,0 +1,35 @@ +:_mod-docs-content-type: ASSEMBLY +include::_attributes/common-attributes.adoc[] +[id="tuning-loki-for-performance-and-scalability"] += Tuning Loki for performance and scalability + +:context: tuning-loki-for-performance-and-scalability + +toc::[] + +[role="_abstract"] +You can configure reliability, performance, high availability, and scalability settings to ensure Loki operates efficiently in production environments and handles increased log volume. + +include::modules/about-loki-reliability-performance.adoc[leveloffset=+1] + +include::modules/loki-pod-placement.adoc[leveloffset=+2] + +include::modules/logging-loki-reliability-hardening.adoc[leveloffset=+2] + +include::modules/logging-loki-retention.adoc[leveloffset=+2] + +include::modules/loki-memberlist-ip.adoc[leveloffset=+2] + +include::modules/loki-restart-hardening.adoc[leveloffset=+2] + +include::modules/about-loki-advanced-deployment.adoc[leveloffset=+1] + +include::modules/loki-sizing-vs-component-scaling.adoc[leveloffset=+2] + +include::modules/loki-zone-aware-replication.adoc[leveloffset=+2] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/about-log-store-tuning.adoc#about-log-store-tuning[About log store tuning] +* xref:../configuring/configuring-loki-network-policies.adoc#configuring-loki-network-policies[Configure Loki network policies] diff --git a/configuring/tuning-the-log-store.adoc b/configuring/tuning-the-log-store.adoc deleted file mode 100644 index ace063386918..000000000000 --- a/configuring/tuning-the-log-store.adoc +++ /dev/null @@ -1,64 +0,0 @@ -:_mod-docs-content-type: ASSEMBLY -include::_attributes/common-attributes.adoc[] -[id="tuning-the-log-store"] -= Tuning the log store - -:context: tuning-the-log-store - -toc::[] - -[role="_abstract"] -Configure advanced settings for Loki to optimize reliability, performance, scalability, security, and alerting in production environments. - -== Prerequisites - -* You have created a `LokiStack` custom resource. - -include::modules/understanding-lokistack-dashboards.adoc[leveloffset=+1] - -//Enhanced reliability and performance -include::modules/about-loki-reliability-performance.adoc[leveloffset=+1] - -include::modules/loki-pod-placement.adoc[leveloffset=+2] - -include::modules/logging-loki-reliability-hardening.adoc[leveloffset=+2] - -include::modules/logging-loki-retention.adoc[leveloffset=+2] - -include::modules/loki-memberlist-ip.adoc[leveloffset=+2] - -include::modules/loki-restart-hardening.adoc[leveloffset=+2] - -//Advanced deployment and scalability -include::modules/about-loki-advanced-deployment.adoc[leveloffset=+1] - -include::modules/loki-sizing-vs-component-scaling.adoc[leveloffset=+2] - -include::modules/loki-zone-aware-replication.adoc[leveloffset=+2] - -include::modules/loki-zone-fail-recovery.adoc[leveloffset=+2] - -include::modules/troubleshoot-loki-rate-limit-errors.adoc[leveloffset=+2] - -//Loki network policies for added security -include::modules/about-loki-network-security.adoc[leveloffset=+1] - -include::modules/loki-network-policies.adoc[leveloffset=+2] - -include::modules/configuring-a-network-policy-for-loki.adoc[leveloffset=+2] - -include::modules/loki-networkpolicy-resources.adoc[leveloffset=+2] - -include::modules/integrating-loki-network-policy-with-external-systems.adoc[leveloffset=+2] - -//Log based alerts -include::modules/about-loki-log-based-alerts.adoc[leveloffset=+1] - -include::modules/loki-rbac-rules-permissions.adoc[leveloffset=+2] - -include::modules/enabling-loki-alerts.adoc[leveloffset=+2] - -[role="_additional-resources"] -== Additional resources - -* xref:../installing/creating-the-lokistack.adoc#creating-the-lokistack[Creating the LokiStack] diff --git a/installing/creating-the-lokistack.adoc b/installing/creating-the-lokistack.adoc index 4452209f6709..91bc6c7aefc6 100644 --- a/installing/creating-the-lokistack.adoc +++ b/installing/creating-the-lokistack.adoc @@ -30,5 +30,5 @@ include::modules/logging-creating-new-group-cluster-admin-user-role.adoc[levelof * xref:../installing/loki-deployment-sizing.adoc#loki-deployment-sizing[Loki deployment sizing] * xref:../installing/configuring-storage-for-lokistack.adoc#configuring-storage-for-lokistack[Configuring storage for LokiStack] * link:https://docs.redhat.com/en/documentation/openshift_container_platform/4.18/html/authentication_and_authorization/managing-cloud-provider-credentials#cco-short-term-creds[Manual mode with short-term credentials for components] -* xref:../configuring/tuning-the-log-store.adoc#tuning-the-log-store[Tuning the log store] +* xref:../configuring/about-log-store-tuning.adoc#about-log-store-tuning[About log store tuning] * xref:../configuring/configuring-log-forwarding.adoc#configuring-log-forwarding[Configuring log forwarding] diff --git a/installing/loki-deployment-sizing.adoc b/installing/loki-deployment-sizing.adoc index 6d7c15242708..0ee51d6abf77 100644 --- a/installing/loki-deployment-sizing.adoc +++ b/installing/loki-deployment-sizing.adoc @@ -14,7 +14,7 @@ This topic provides guidance for both pre-installation and post-installation siz Starting with an appropriate initial size ensures a smooth deployment and minimizes the need for resizing. -If your deployment exceeds the capacity of the chosen size, you can either increase the LokiStack size or scale individual components. For guidance on when to resize versus scale components, see xref:../configuring/tuning-the-log-store.adoc#loki-sizing-vs-component-scaling_tuning-the-log-store[LokiStack sizing versus component scaling]. +If your deployment exceeds the capacity of the chosen size, you can either increase the LokiStack size or scale individual components. For guidance on when to resize versus scale components, see xref:../configuring/tuning-loki-for-performance-and-scalability.adoc#loki-sizing-vs-component-scaling_tuning-loki-for-performance-and-scalability[LokiStack sizing versus component scaling]. include::modules/understanding-loki-sizing.adoc[leveloffset=+1] diff --git a/installing/overview-of-openshift-logging-installation.adoc b/installing/overview-of-openshift-logging-installation.adoc index b66ab1e09e7b..2110b1911766 100644 --- a/installing/overview-of-openshift-logging-installation.adoc +++ b/installing/overview-of-openshift-logging-installation.adoc @@ -32,5 +32,5 @@ include::modules/logging-storage-prerequisites.adoc[leveloffset=+1] * xref:../installing/installing-the-loki-operator.adoc#installing-the-loki-operator[Installing the Loki Operator] * xref:../installing/installing-the-red-hat-openshift-logging-operator.adoc#installing-the-red-hat-openshift-logging-operator[Installing the Red Hat OpenShift Logging Operator] * xref:../installing/creating-the-lokistack.adoc#creating-the-lokistack[Creating the LokiStack] -* xref:../configuring/tuning-the-log-store.adoc#tuning-the-log-store[Tuning the log store] +* xref:../configuring/about-log-store-tuning.adoc#about-log-store-tuning[About log store tuning] * xref:../configuring/configuring-log-forwarding.adoc#configuring-log-forwarding[Configuring log forwarding] diff --git a/logging_alerts/configuring-loki-log-based-alerts.adoc b/logging_alerts/configuring-loki-log-based-alerts.adoc new file mode 100644 index 000000000000..f12fa8a98d24 --- /dev/null +++ b/logging_alerts/configuring-loki-log-based-alerts.adoc @@ -0,0 +1,24 @@ +:_mod-docs-content-type: ASSEMBLY +include::_attributes/common-attributes.adoc[] +[id="configuring-loki-log-based-alerts"] += Configuring Loki log-based alerts + +:context: configuring-loki-log-based-alerts + +toc::[] + +[role="_abstract"] +You can configure log-based alerts for Loki by creating an `AlertingRule` custom resource (CR). + +include::modules/about-loki-log-based-alerts.adoc[leveloffset=+1] + +include::modules/loki-rbac-rules-permissions.adoc[leveloffset=+2] + +include::modules/enabling-loki-alerts.adoc[leveloffset=+2] + +[role="_additional-resources"] +.Additional resources + +* xref:../logging_alerts/default-logging-alerts.adoc#default-logging-alerts[Default logging alerts] +* xref:../logging_alerts/custom-logging-alerts.adoc#custom-logging-alerts[Custom logging alerts] +* xref:../configuring/about-log-store-tuning.adoc#about-log-store-tuning[About log store tuning] diff --git a/modules/about-loki-advanced-deployment.adoc b/modules/about-loki-advanced-deployment.adoc index d0c8c70d3855..9c27f3bfa39a 100644 --- a/modules/about-loki-advanced-deployment.adoc +++ b/modules/about-loki-advanced-deployment.adoc @@ -1,6 +1,8 @@ // Module included in the following assemblies: // -// * configuring/tuning-the-log-store.adoc +// * configuring/about-log-store-tuning.adoc +// * configuring/tuning-loki-for-performance-and-scalability.adoc +// * configuring/configuring-loki-network-policies.adoc :_mod-docs-content-type: CONCEPT [id="about-loki-advanced-deployment_{context}"] diff --git a/modules/about-loki-log-based-alerts.adoc b/modules/about-loki-log-based-alerts.adoc index 0437222bc46c..0eb1ff5c6710 100644 --- a/modules/about-loki-log-based-alerts.adoc +++ b/modules/about-loki-log-based-alerts.adoc @@ -1,6 +1,8 @@ // Module included in the following assemblies: // -// * configuring/tuning-the-log-store.adoc +// * configuring/about-log-store-tuning.adoc +// * configuring/tuning-loki-for-performance-and-scalability.adoc +// * configuring/configuring-loki-network-policies.adoc :_mod-docs-content-type: CONCEPT [id="about-loki-log-based-alerts_{context}"] diff --git a/modules/about-loki-network-security.adoc b/modules/about-loki-network-security.adoc index c38425e32381..2e296443f58a 100644 --- a/modules/about-loki-network-security.adoc +++ b/modules/about-loki-network-security.adoc @@ -1,6 +1,8 @@ // Module included in the following assemblies: // -// * configuring/tuning-the-log-store.adoc +// * configuring/about-log-store-tuning.adoc +// * configuring/tuning-loki-for-performance-and-scalability.adoc +// * configuring/configuring-loki-network-policies.adoc :_mod-docs-content-type: CONCEPT [id="about-loki-network-security_{context}"] diff --git a/modules/about-loki-reliability-performance.adoc b/modules/about-loki-reliability-performance.adoc index 8b00f31bdb1d..699c8049e0a5 100644 --- a/modules/about-loki-reliability-performance.adoc +++ b/modules/about-loki-reliability-performance.adoc @@ -1,6 +1,8 @@ // Module included in the following assemblies: // -// * configuring/tuning-the-log-store.adoc +// * configuring/about-log-store-tuning.adoc +// * configuring/tuning-loki-for-performance-and-scalability.adoc +// * configuring/configuring-loki-network-policies.adoc :_mod-docs-content-type: CONCEPT [id="about-loki-reliability-performance_{context}"] diff --git a/modules/adding-a-drop-filter.adoc b/modules/adding-a-drop-filter.adoc index 5acf0973f18b..99e54ce286a5 100644 --- a/modules/adding-a-drop-filter.adoc +++ b/modules/adding-a-drop-filter.adoc @@ -60,9 +60,8 @@ spec: You can set either `matches` or `notMatches` for a single field, but not both. If a field referenced in a test does not exist in the log record, that condition evaluates to `False`. ==== -+ *Example: Keep only high-priority log records* -+ + [source,yaml] ---- spec: @@ -76,11 +75,11 @@ spec: - field: .level matches: "info|warning" ---- -+ + This filter drops records where the message does not contain "critical" or "error" and the level is "info" or "warning". -+ + *Example: Multiple test conditions (OR logic)* -+ + [source,yaml] ---- spec: @@ -97,7 +96,7 @@ spec: - field: .kubernetes.pod_name notMatches: "my-pod" ---- -+ + This filter drops logs that either come from a namespace starting with `openshift` (first test), or are application logs that do not have `my-pod` in the pod name (second test). . Apply the updated `ClusterLogForwarder` CR by running the following command: diff --git a/modules/best-practices-multitenant-logging.adoc b/modules/best-practices-multitenant-logging.adoc index a442be7c9346..7378a4d6427c 100644 --- a/modules/best-practices-multitenant-logging.adoc +++ b/modules/best-practices-multitenant-logging.adoc @@ -165,11 +165,13 @@ A single output reduces the component count. Multiple pipelines can share the sa Each pipeline creates additional components for routing and processing. Where possible, combine pipelines that share inputs and outputs. Separate pipelines are necessary when: + * Logs require different transformations before reaching different outputs * Security or compliance requires strict separation of processing paths * Different tenants require different delivery guarantees (for example, `AtLeastOnce` versus `AtMostOnce`) Pipelines can be combined when: + * Logs go through the same filters to reach the same output * The only difference is the source namespace or labels * Tenant isolation is handled at the destination @@ -177,11 +179,13 @@ Pipelines can be combined when: Creating multiple `ClusterLogForwarder` custom resources increases the overall component count. Each `ClusterLogForwarder` instance deploys a separate collector pod with its own set of components. Use multiple `ClusterLogForwarder` instances when: + * Different service accounts are required for different log collection purposes * Different security or network policies apply * Logs from different sources require completely different processing pipelines A single `ClusterLogForwarder` is sufficient when: + * All logs can use the same service account * Tenant isolation is achieved through filtering and routing * Network policies allow a single collector to reach all destinations @@ -254,17 +258,15 @@ spec: outputRefs: - compliance-s3 ---- -+ + This configuration uses: -+ --- + * A single input for all application logs, controlled by pod label * A single filter that applies to all tenants * A single LokiStack output for all tenant application and infrastructure logs * A separate output for compliance because it goes to a different destination type * Three pipelines for different log types, sharing inputs and outputs where possible --- -+ + This configuration creates approximately 40-60 `component_id` values, compared to 400-500 values for a per-tenant input/output design. In `LokiStack`, tenant isolation is achieved through the namespace labels in the logs. Use `LogQL` queries to filter logs by tenant: @@ -295,12 +297,4 @@ Each histogram metric creates 10 time series per component_id: * `vector_component_received_events_count_bucket`: 70 × 10 = 700 series * `vector_buffer_send_duration_seconds_bucket`: 70 × 10 = 700 series -These estimates are approximate. Use the diagnostic procedures in "Troubleshooting high collector metrics cardinality" to measure actual cardinality. - -[role="_additional-resources"] -.Additional resources - -* xref:../configuring/cluster-logging-collector.adoc#collector-metrics-cardinality-impact_cluster-logging-collector[Understanding collector metrics cardinality and monitoring impact] -* xref:../configuring/cluster-logging-collector.adoc#troubleshooting-collector-metrics-cardinality_cluster-logging-collector[Troubleshooting high collector metrics cardinality] -* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] -* xref:../configuring/configuring-filters.adoc#configuring-filters[Configuring filters] +These estimates are approximate. Use the diagnostic procedures in "Troubleshoot the collector" to measure actual cardinality. diff --git a/modules/clf-outputs-reference.adoc b/modules/clf-outputs-reference.adoc index 6b50e858f40c..de8b8ed1c5da 100644 --- a/modules/clf-outputs-reference.adoc +++ b/modules/clf-outputs-reference.adoc @@ -65,8 +65,3 @@ The `otlp` output uses the OpenTelemetry data model, which differs from the ViaQ **Use case**: Developers and site reliability engineers often prefer Splunk because it is a familiar platform with powerful search and visualization capabilities. Organizations can route application and infrastructure logs to Splunk for operational analysis. `syslog`:: Forwards logs to an external syslog receiver by using RFC3164 or RFC5424 format. This output supports TCP, TLS, and UDP transports. You can customize facility, severity, application name, process ID, and message ID fields by using template syntax. - -[role="_additional-resources"] -.Additional resources - -* xref:../configuring/forwarding-to-third-party-systems.adoc#forwarding-to-third-party-systems[Forwarding to third-party systems] diff --git a/modules/clf-pipelines-reference.adoc b/modules/clf-pipelines-reference.adoc index 98b8b24a97fc..1fbe4ac02f38 100644 --- a/modules/clf-pipelines-reference.adoc +++ b/modules/clf-pipelines-reference.adoc @@ -201,10 +201,3 @@ spec: outputRefs: - lokistack ---- - -[role="_additional-resources"] -.Additional resources - -* xref:../configuring/configuring-inputs.adoc#configuring-inputs[Configuring inputs] -* xref:../configuring/configuring-filters.adoc#configuring-filters[Configuring filters] -* xref:../configuring/configuring-outputs.adoc#configuring-outputs[Configuring outputs] diff --git a/modules/cloudwatch-output-format.adoc b/modules/cloudwatch-output-format.adoc index 72103a1e07b5..08bcef1e3b93 100644 --- a/modules/cloudwatch-output-format.adoc +++ b/modules/cloudwatch-output-format.adoc @@ -10,7 +10,7 @@ [role="_abstract"] When you forward logs to Amazon CloudWatch, the log collector transforms OpenShift log records into the CloudWatch Logs API format. Understanding this transformation helps you configure CloudWatch queries and dashboards effectively. -The log collector sends logs to CloudWatch by using the `Put`LogEvent`s` API. Each log record becomes a CloudWatch ``LogEvent`` with two main components: +The log collector sends logs to CloudWatch by using the PutLogEvents API. Each log record becomes a CloudWatch `LogEvent` with two main components: `message`:: The entire OpenShift log record serialized as a JSON string. This field contains all log data including Kubernetes metadata, application messages, and timestamps. @@ -78,5 +78,5 @@ Both fields can use dynamic values from the log record. The collector evaluates [role="_additional-resources"] .Additional resources -* link:https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_`Put`LogEvent`s`.html[`Put`LogEvent`s` API reference] +* link:https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html[PutLogEvents API reference] * link:https://vector.dev/docs/reference/configuration/sinks/aws_cloudwatch_logs/[Vector CloudWatch sink documentation] diff --git a/modules/cluster-logging-collector-log-forward-cloudwatch.adoc b/modules/cluster-logging-collector-log-forward-cloudwatch.adoc index 4422d73fbce6..285e5be6598b 100644 --- a/modules/cluster-logging-collector-log-forward-cloudwatch.adoc +++ b/modules/cluster-logging-collector-log-forward-cloudwatch.adoc @@ -13,7 +13,9 @@ To configure log forwarding to CloudWatch, create a `ClusterLogForwarder` custom .Procedure -include::snippets/creating-an-aws-secret-access-key.adoc[] +// vale AsciiDocDITA.TaskInclude = NO +include::snippets/creating-an-aws-secret-access-key.adoc[] +// vale AsciiDocDITA.TaskInclude = YES . Create or edit a YAML file that defines the `ClusterLogForwarder` CR object. In the file, specify the name of the secret. For example: + diff --git a/modules/cluster-logging-collector-log-forwarding-about.adoc b/modules/cluster-logging-collector-log-forwarding-about.adoc index 01e3fcb67b7c..7568808dac1f 100644 --- a/modules/cluster-logging-collector-log-forwarding-about.adoc +++ b/modules/cluster-logging-collector-log-forwarding-about.adoc @@ -83,20 +83,3 @@ spec: `index`:: The `index` field reads the `.log_type` field value and falls back to "unknown" if not found. `secretName`:: Use username and password to authenticate to the server `ca`:: Enable Mutual Transport Layer Security (mTLS) between collector and Elasticsearch. The spec identifies the keys and secret for the certificates that they represent. - -== Creating a Secret - -You can create a secret in the directory that has your certificate and key files by using the following command: - -[source,terminal] ----- -$ oc create secret generic -n \ - --from-file=ca-bundle.crt= \ - --from-literal=username= \ - --from-literal=password= ----- - -[NOTE] -==== -Red Hat recommends generic or opaque secrets for best results. -==== diff --git a/modules/collector-metrics-cardinality-impact.adoc b/modules/collector-metrics-cardinality-impact.adoc index 984c94031aba..f8e7980c21b5 100644 --- a/modules/collector-metrics-cardinality-impact.adoc +++ b/modules/collector-metrics-cardinality-impact.adoc @@ -99,11 +99,3 @@ Consider reviewing your `ClusterLogForwarder` configuration if: * You observe high cardinality warnings in Prometheus logs You can check the current cardinality of Vector metrics by using the Prometheus API. See "Troubleshooting high collector metrics cardinality" for diagnostic procedures. - -[role="_additional-resources"] -.Additional resources - -* xref:../configuring/cluster-logging-collector.adoc#best-practices-multitenant-logging_cluster-logging-collector[Best practices for multitenant logging configurations] -* xref:../configuring/cluster-logging-collector.adoc#troubleshooting-collector-metrics-cardinality_cluster-logging-collector[Troubleshooting high collector metrics cardinality] -* link:https://prometheus.io/docs/practices/naming/#labels[Prometheus metric and label naming best practices] -* link:https://www.robustperception.io/cardinality-is-key[Understanding cardinality in Prometheus] diff --git a/modules/collector-network-policy.adoc b/modules/collector-network-policy.adoc index 8da67872721c..6130d1a01602 100644 --- a/modules/collector-network-policy.adoc +++ b/modules/collector-network-policy.adoc @@ -25,6 +25,3 @@ Restricts traffic to specific ports based on the configured inputs, outputs, and If you do not define a `spec.collector.networkPolicy` field, the Operator will not create a `NetworkPolicy` resource for the collector. Without a `NetworkPolicy` resource, logging components might not function if a cluster-wide `AdminNetworkPolicy` restricts traffic or other restrictive network policies are in place. -[role="_additional-resources"] -.Additional resources -* link:https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html-single/network_security/index#network-policy-apis[Understanding network policy APIs] diff --git a/modules/configuring-collector-metrics-profiles.adoc b/modules/configuring-collector-metrics-profiles.adoc index 1a70d5a94855..4002015225cf 100644 --- a/modules/configuring-collector-metrics-profiles.adoc +++ b/modules/configuring-collector-metrics-profiles.adoc @@ -95,9 +95,3 @@ $ oc adm top pod -n openshift-monitoring prometheus-k8s-0 ---- + You should observe reduced CPU and memory consumption compared to the full profile. - -[role="_additional-resources"] -.Additional resources - -* link:https://docs.openshift.com/container-platform/latest/observability/monitoring/configuring-the-monitoring-stack.html#creating-cluster-monitoring-configmap_configuring-the-monitoring-stack[Configuring the monitoring stack] -* link:https://github.com/openshift/enhancements/blob/master/enhancements/monitoring/metrics-collection-profiles.md[Metrics collection profiles enhancement] diff --git a/modules/creating-an-adminnetworkpolicy-rule-for-collector-network-policy.adoc b/modules/creating-an-adminnetworkpolicy-rule-for-collector-network-policy.adoc index 8fddbf0db1a9..7944a118015e 100644 --- a/modules/creating-an-adminnetworkpolicy-rule-for-collector-network-policy.adoc +++ b/modules/creating-an-adminnetworkpolicy-rule-for-collector-network-policy.adoc @@ -81,6 +81,3 @@ To apply this network policy to all the resources in a namespace, remove `app.ku $ oc apply -f .yaml ---- -[role="_additional-resources"] -.Additional resources -* link:https://docs.redhat.com/en/documentation/openshift_container_platform/4.19/html-single/network_security/index#ovn-k-anp[OVN-Kubernetes AdminNetworkPolicy] diff --git a/modules/loki-create-object-storage-secret-cli.adoc b/modules/creating-log-forwarding-secret-cli.adoc similarity index 77% rename from modules/loki-create-object-storage-secret-cli.adoc rename to modules/creating-log-forwarding-secret-cli.adoc index 5de3264d3370..bd43f3b6006a 100644 --- a/modules/loki-create-object-storage-secret-cli.adoc +++ b/modules/creating-log-forwarding-secret-cli.adoc @@ -1,15 +1,16 @@ // Module is included in the following assemblies: // // * configuring/configuring-the-log-store.adoc +// * configuring/forwarding-to-third-party-systems.adoc :_mod-docs-content-type: PROCEDURE -[id="loki-create-object-storage-secret-cli_{context}"] -= Creating a secret for Loki object storage by using the CLI +[id="creating-log-forwarding-secret-cli_{context}"] += Creating a secret by using the CLI [role="_abstract"] -Create a secret for Loki object storage by using the CLI. +Create a secret by using the CLI. -To configure Loki object storage, you must create a secret. You can do this by using the {oc-first}. +You can create a secret by using the {oc-first}. .Prerequisites diff --git a/modules/default-splunk-metadata-key-values.adoc b/modules/default-splunk-metadata-key-values.adoc index 6a1634acbceb..2571466bedd9 100644 --- a/modules/default-splunk-metadata-key-values.adoc +++ b/modules/default-splunk-metadata-key-values.adoc @@ -11,26 +11,9 @@ The following table describes the default value that the collector uses for Splunk metadata, depending on `log_type` and `log_source` attributes. .Default metadata key values -[cols="1,1,1,1,1"] +[cols="2,2,2,2,3"] |==== -|Key -a|Infrastructure Journal - -* `log_type`: infrastructure -* `log_source`: node - -a|Infrastructure or application container - -* `log_type`: infrastructure, application - -* `log_source`: container - -a|Audit - -* `log_type`: audit -* `log_source`: `auditd`, `ovn`, `openshiftAPI`, `kubeAPI` - -|Note +|Metadata field |Infrastructure (journal) |Infrastructure/Application (container) |Audit logs |Notes diff --git a/modules/forwarding-logs-to-amazon-s3-endpoint.adoc b/modules/forwarding-logs-to-amazon-s3-endpoint.adoc index 625b0b479797..6d1d5b865802 100644 --- a/modules/forwarding-logs-to-amazon-s3-endpoint.adoc +++ b/modules/forwarding-logs-to-amazon-s3-endpoint.adoc @@ -15,7 +15,9 @@ To configure log forwarding to an Amazon S3 endpoint, create a `ClusterLogForwar .Procedure -include::snippets/creating-an-aws-secret-access-key.adoc[] +// vale AsciiDocDITA.TaskInclude = NO +include::snippets/creating-an-aws-secret-access-key.adoc[] +// vale AsciiDocDITA.TaskInclude = YES . Create or edit a YAML file that defines the `ClusterLogForwarder` CR object and specify the secret name. For example: + diff --git a/modules/integrating-loki-network-policy-with-external-systems.adoc b/modules/integrating-loki-network-policy-with-external-systems.adoc index 03a7bcf458ac..87b0b52b1d2f 100644 --- a/modules/integrating-loki-network-policy-with-external-systems.adoc +++ b/modules/integrating-loki-network-policy-with-external-systems.adoc @@ -3,7 +3,7 @@ :_mod-docs-content-type: PROCEDURE [id="integrating-loki-network-policy-with-external-systems_{context}"] -= Integrating Loki network policy with external systems += Integrating a Loki network policy with external systems [role="_abstract"] Create additional network policies to integrate Loki with external systems. diff --git a/modules/logfilemetricexporter-network-policy.adoc b/modules/logfilemetricexporter-network-policy.adoc index affba322aff9..a5f16b4a30c1 100644 --- a/modules/logfilemetricexporter-network-policy.adoc +++ b/modules/logfilemetricexporter-network-policy.adoc @@ -22,7 +22,3 @@ Allows all ingress and egress traffic. If you do not define a `spec.networkPolicy` field, the Operator will not create a `NetworkPolicy` resource for the `LogFileMetricExporter`. Without a `NetworkPolicy` resource, logging components might not function if a cluster-wide `AdminNetworkPolicy` restricts traffic or other restrictive network policies are in place. -[role="_additional-resources"] -.Additional resources -* link:https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html-single/network_security/index#network-policy-apis[Understanding network policy APIs] - diff --git a/modules/logging-audit-log-filtering.adoc b/modules/logging-audit-log-filtering.adoc index 4c6b579c4a7a..de54f3883a9c 100644 --- a/modules/logging-audit-log-filtering.adoc +++ b/modules/logging-audit-log-filtering.adoc @@ -133,12 +133,7 @@ spec: # A catch-all rule to log all other requests at the Metadata level. - level: Metadata ---- -+ -`inputRefs`:: The collected log types. The value for this field can be `audit` for audit logs, `application` for application logs, `infrastructure` for infrastructure logs, or a named input that is defined for your application. -+ -`filterRefs`:: The name of your audit policy. -[role="_additional-resources"] -.Additional resources +`inputRefs`:: The collected log types. The value for this field can be `audit` for audit logs, `application` for application logs, `infrastructure` for infrastructure logs, or a named input that is defined for your application. -* link:https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/#audit-policy[Kubernetes audit policy] +`filterRefs`:: The name of your audit policy. diff --git a/modules/logging-forwarding-azure-logs-ingestion.adoc b/modules/logging-forwarding-azure-logs-ingestion.adoc index 79cd04e6f2bc..d88f6641d13f 100644 --- a/modules/logging-forwarding-azure-logs-ingestion.adoc +++ b/modules/logging-forwarding-azure-logs-ingestion.adoc @@ -143,7 +143,7 @@ spec: outputRefs: - azure-logs-ingestion ---- -+ + Workload identity authentication requires additional {entra-id} configuration, including a federated identity credential that trusts your {product-title} cluster's OIDC issuer. *Configuring custom timestamp field* diff --git a/modules/logging-loki-reliability-hardening.adoc b/modules/logging-loki-reliability-hardening.adoc index 174100117adf..d8b9fea1a99c 100644 --- a/modules/logging-loki-reliability-hardening.adoc +++ b/modules/logging-loki-reliability-hardening.adoc @@ -37,8 +37,11 @@ spec: # ... ---- -spec.template.ingester.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution:: +requiredDuringSchedulingIgnoredDuringExecution:: The stanza to define a required rule. -spec.template.ingester.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution.labelSelector.matchLabels:: +labelSelector.matchLabels:: The key-value pair (label) that must be matched to apply the rule. + +topologyKey:: + The topology domain for the node. Use `kubernetes.io/hostname` to schedule pods on different nodes. diff --git a/modules/loki-output-format.adoc b/modules/loki-output-format.adoc index cb15584913a5..39b83f5b2275 100644 --- a/modules/loki-output-format.adoc +++ b/modules/loki-output-format.adoc @@ -204,4 +204,4 @@ After selecting streams by labels, you can filter log lines by content: * link:https://grafana.com/docs/loki/latest/fundamentals/labels/[Loki labels and streams] * link:https://grafana.com/docs/loki/latest/logql/[LogQL query language] * link:https://grafana.com/docs/loki/latest/best-practices/[Loki best practices] -* xref:../configuring/configuring-the-log-store.adoc#configuring-lokistack-storage[Configuring the log store] +* xref:../configuring/configuring-the-log-store.adoc#configuring-the-log-store[Configure the log store] diff --git a/modules/loki-sizing-vs-component-scaling.adoc b/modules/loki-sizing-vs-component-scaling.adoc index 778276af4b6c..0af5e1ab6b1e 100644 --- a/modules/loki-sizing-vs-component-scaling.adoc +++ b/modules/loki-sizing-vs-component-scaling.adoc @@ -1,6 +1,8 @@ // Module included in the following assemblies: // -// * configuring/tuning-the-log-store.adoc +// * configuring/about-log-store-tuning.adoc +// * configuring/tuning-loki-for-performance-and-scalability.adoc +// * configuring/configuring-loki-network-policies.adoc :_mod-docs-content-type: CONCEPT [id="loki-sizing-vs-component-scaling_{context}"] @@ -119,4 +121,4 @@ Use the following decision guide to choose between changing the size and scaling * xref:../installing/loki-deployment-sizing.adoc#loki-deployment-sizing[Loki deployment sizing] * xref:../installing/loki-deployment-sizing.adoc#calculating-sizing-after-installation_loki-deployment-sizing[Calculating sizing after installation] * xref:../configuring/loki-query-performance-troubleshooting.adoc#loki-query-performance-troubleshooting[Loki query performance troubleshooting] -* xref:../configuring/tuning-the-log-store.adoc#tuning-the-log-store[Tuning the log store] +* xref:../configuring/about-log-store-tuning.adoc#about-log-store-tuning[About log store tuning] diff --git a/modules/troubleshoot-loki-rate-limit-errors.adoc b/modules/troubleshoot-loki-rate-limit-errors.adoc index dffbfb49ad90..3c720a73a6b9 100644 --- a/modules/troubleshoot-loki-rate-limit-errors.adoc +++ b/modules/troubleshoot-loki-rate-limit-errors.adoc @@ -2,7 +2,9 @@ // // * troubleshooting/troubleshoot-log-forwarding.adoc // * configuring/configuring-the-log-store.adoc -// * configuring/tuning-the-log-store.adoc +// * configuring/about-log-store-tuning.adoc +// * configuring/tuning-loki-for-performance-and-scalability.adoc +// * configuring/configuring-loki-network-policies.adoc :_mod-docs-content-type: PROCEDURE [id="troubleshoot-loki-rate-limit-errors_{context}"] diff --git a/modules/troubleshooting-collector-metrics-cardinality.adoc b/modules/troubleshooting-collector-metrics-cardinality.adoc index 82bebdc54502..c6cce86241d1 100644 --- a/modules/troubleshooting-collector-metrics-cardinality.adoc +++ b/modules/troubleshooting-collector-metrics-cardinality.adoc @@ -1,6 +1,6 @@ // Module included in the following assemblies: // -// * observability/logging/log_collection_forwarding/cluster-logging-collector.adoc +// * troubleshooting/troubleshoot-the-collector.adoc :_mod-docs-content-type: PROCEDURE [id="troubleshooting-collector-metrics-cardinality_{context}"] @@ -236,7 +236,7 @@ prometheus_tsdb_storage_blocks_bytes [role="_additional-resources"] .Additional resources -* xref:../configuring/cluster-logging-collector.adoc#collector-metrics-cardinality-impact_cluster-logging-collector[Understanding collector metrics cardinality and monitoring impact] -* xref:../configuring/cluster-logging-collector.adoc#best-practices-multitenant-logging_cluster-logging-collector[Best practices for multitenant logging configurations] +* xref:../configuring/monitor-collector-metrics.adoc#collector-metrics-cardinality-impact_monitor-collector-metrics[Understanding collector metrics cardinality and monitoring impact] +* xref:../configuring/monitor-collector-metrics.adoc#best-practices-multitenant-logging_monitor-collector-metrics[Best practices for multitenant logging configurations] * link:https://prometheus.io/docs/prometheus/latest/storage/[Prometheus storage documentation] * link:https://access.redhat.com/solutions/7137995[Red Hat Knowledgebase: Prometheus storage size impacted by high cardinality of Vector metrics] diff --git a/modules/understanding-loki-sizing.adoc b/modules/understanding-loki-sizing.adoc index 59e074205630..2644c63cf79b 100644 --- a/modules/understanding-loki-sizing.adoc +++ b/modules/understanding-loki-sizing.adoc @@ -21,7 +21,7 @@ QPS (Queries Per Second) measures the rate of log queries against your LokiStack A single user running multiple dashboards can generate 10-20 QPS. The xref:../installing/loki-deployment-sizing.adoc#loki-sizing_loki-deployment-sizing[LokiStack sizing table] shows average QPS at 200ms query latency. -After you install the `LokiStack`, you can view actual QPS metrics on the *LokiStack Reads* and *LokiStack Writes* dashboards. In the {ocp-product-title} web console, navigate to *Observe* → *Dashboards*. The *LokiStack Writes* dashboard is particularly important for scaling decisions, as it shows the write load on LokiStack components receiving logs from the collector. For more information about the available dashboards, see xref:../configuring/tuning-the-log-store.adoc#understanding-lokistack-dashboards_tuning-the-log-store[Understanding LokiStack dashboards]. +After you install the `LokiStack`, you can view actual QPS metrics on the *LokiStack Reads* and *LokiStack Writes* dashboards. In the {ocp-product-title} web console, navigate to *Observe* → *Dashboards*. The *LokiStack Writes* dashboard is particularly important for scaling decisions, as it shows the write load on LokiStack components receiving logs from the collector. For more information about the available dashboards, see xref:../configuring/about-log-store-tuning.adoc#understanding-lokistack-dashboards_about-log-store-tuning[Understanding LokiStack dashboards]. [id="log-volume-affects-sizing_{context}"] == How log volume affects sizing diff --git a/modules/understanding-lokistack-dashboards.adoc b/modules/understanding-lokistack-dashboards.adoc index 84f4da0a13e8..09b875d1127d 100644 --- a/modules/understanding-lokistack-dashboards.adoc +++ b/modules/understanding-lokistack-dashboards.adoc @@ -1,6 +1,8 @@ // Module included in the following assemblies: // -// * configuring/tuning-the-log-store.adoc +// * configuring/about-log-store-tuning.adoc +// * configuring/tuning-loki-for-performance-and-scalability.adoc +// * configuring/configuring-loki-network-policies.adoc :_mod-docs-content-type: CONCEPT [id="understanding-lokistack-dashboards_{context}"] diff --git a/troubleshooting/troubleshoot-the-collector.adoc b/troubleshooting/troubleshoot-the-collector.adoc new file mode 100644 index 000000000000..6bb8c6e59a26 --- /dev/null +++ b/troubleshooting/troubleshoot-the-collector.adoc @@ -0,0 +1,18 @@ +:_mod-docs-content-type: ASSEMBLY +[id="troubleshoot-the-collector"] += Troubleshoot the collector +include::_attributes/common-attributes.adoc[] +:context: troubleshoot-the-collector + +toc::[] + +[role="_abstract"] +Diagnose and resolve issues with the log collector, including high metrics cardinality. + +include::modules/troubleshooting-collector-metrics-cardinality.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources + +* xref:../configuring/monitor-collector-metrics.adoc#monitor-collector-metrics[Monitor collector metrics] +* xref:../configuring/configure-the-collector.adoc#configure-the-collector[Configure the collector]