Merge branch 'master' into docs-improve-sanitizing-instructions

Author: pavangudiwada

docs/configuration/holmesgpt/builtin_toolsets.rst

@@ -9,6 +9,7 @@ Builtin Toolsets
    toolsets/argocd
    toolsets/aws
    toolsets/confluence
+   toolsets/coralogix_logs
    toolsets/datetime
    toolsets/docker
    toolsets/grafanaloki
@@ -52,6 +53,11 @@ by the user by providing credentials or API keys to external systems.
         :link: toolsets/confluence
         :link-type: doc
 
+    .. grid-item-card:: :octicon:`cpu;1em;` Coralogix logs
+        :class-card: sd-bg-light sd-bg-text-light
+        :link: toolsets/coralogix_logs
+        :link-type: doc
+
     .. grid-item-card:: :octicon:`cpu;1em;` Datetime
         :class-card: sd-bg-light sd-bg-text-light
         :link: toolsets/datetime

docs/configuration/holmesgpt/custom_toolsets.rst

@@ -115,9 +115,9 @@ This toolset lets Holmes view Grafana dashboards and suggest relevant dashboards
             command: "curl \"${GRAFANA_URL}/api/search\" -H \"Authorization: Bearer ${GRAFANA_API_KEY}\""
 
           - name: "grafana_get_url"
-            description: "Get the URL of a Grafana dashboard by UID, including the real url of grafana"
+            description: "Get the URL of a Grafana dashboard by UID, including the real URL of Grafana"
             # in this command we use a variable called `{{ dashboard_uid }}`
-            # unlike enviroment variables that were provided by the user, variables like `{{ dashboard_uid }}` are provided by the AI model
+            # unlike environment variables that were provided by the user, variables like `{{ dashboard_uid }}` are provided by the AI model
             # the AI model sees the tool description, decides to use this tool, and then provides a value for all {{ template_variables }} to invoke the tool
             command: "echo \"${GRAFANA_URL}/d/{{ dashboard_uid }}\""
 
@@ -211,23 +211,23 @@ First `create a GitHub Personal Access Token with fine-grained permissions <http
                   description: "Fetches the most recent commits for a repository"
                   command: "curl -H 'Authorization: token ${GITHUB_TOKEN}' https://api.github.com/repos/{{ owner }}/{{ repo }}/commits?per_page={{ limit }} "
 
-              - name: "get_repo_details"
-                description: "Fetches details of a specific repository"
-                command: "curl -H 'Authorization: token ${GITHUB_TOKEN}' https://api.github.com/repos/{{ owner }}/{{ repo }}"
-
-                # In the above examples, LLM-provided parameters like {{ owner }} are inferred automatically from the command
-                # you can also define them explicitly - this is useful if:
-                # - You want to enforce parameter requirements (e.g., `owner` and `repo` are required).
-                # - You want to define provide a default value for optional parameters.
-                parameters:
-                  owner:
-                    type: "string"
-                    description: "Owner of the repository."
-                    required: true
-                  repo:
-                    type: "string"
-                    description: "Name of the repository."
-                    required: true
+                - name: "get_repo_details"
+                  description: "Fetches details of a specific repository"
+                  command: "curl -H 'Authorization: token ${GITHUB_TOKEN}' https://api.github.com/repos/{{ owner }}/{{ repo }}"
+
+                  # In the above examples, LLM-provided parameters like {{ owner }} are inferred automatically from the command
+                  # you can also define them explicitly - this is useful if:
+                  # - You want to enforce parameter requirements (e.g., `owner` and `repo` are required).
+                  # - You want to define provide a default value for optional parameters.
+                  parameters:
+                    owner:
+                      type: "string"
+                      description: "Owner of the repository."
+                      required: true
+                    repo:
+                      type: "string"
+                      description: "Name of the repository."
+                      required: true
 
     Update your Helm values with the provided YAML configuration, then apply the changes with Helm upgrade:
 
@@ -244,7 +244,7 @@ First `create a GitHub Personal Access Token with fine-grained permissions <http
 
   .. md-tab-item:: Holmes CLI
 
-    First, add the following environment vairables:
+    First, add the following environment variables:
 
     .. code-block:: bash
       
@@ -271,7 +271,7 @@ First `create a GitHub Personal Access Token with fine-grained permissions <http
                 # In the above examples, LLM-provided parameters like {{ owner }} are inferred automatically from the command
                 # you can also define them explicitly - this is useful if:
                 # - You want to enforce parameter requirements (e.g., `owner` and `repo` are required).
-                # - You want to define provide a default value for optional parameters.
+                # - You want to provide a default value for optional parameters.
                 parameters:
                   owner:
                     type: "string"
@@ -283,7 +283,7 @@ First `create a GitHub Personal Access Token with fine-grained permissions <http
                     required: true
     To test, run: 
 
-      .. code-block:: yaml
+      .. code-block:: bash
         
         holmes ask -t github_toolset.yaml "who made the last commit to the robusta-dev/holmesgpt repo on github?"
 

docs/configuration/holmesgpt/toolsets/_toolsets_that_provide_logging.inc.rst

@@ -0,0 +1,5 @@
+HolmesGPT provides several out-of-the-box alternatives for log access. You can select from these options:
+
+* :ref:`kubernetes/logs <toolset_kubernetes_logs>`: Access logs with ``kubectl logs`` commands. **This is the default toolset.**
+* :ref:`coralogix/logs <toolset_coralogix_logs>`: Access logs through Coralogix.
+* :ref:`grafana/loki <toolset_grafana_loki>`: Access Loki logs by proxying through a Grafana instance.

docs/configuration/holmesgpt/toolsets/argocd.rst

@@ -12,7 +12,35 @@ This toolset requires an ``ARGOCD_AUTH_TOKEN`` environment variable. Generate su
 You can consult the `available environment variables <https://argo-cd.readthedocs.io/en/latest/user-guide/environment-variables/>`_
 on argocd's official documentation for the CLI.
 
-In addition to the auth token, you will need to tell argocd how to connect to the server. This can be done two ways:
+The permissions required are below (``kubectl edit configmap argocd-rbac-cm -n argocd``). You can consult
+ArgoCD's documentation on `user creation <https://argo-cd.readthedocs.io/en/stable/operator-manual/user-management/>`_
+and `permissions <https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/>`_.
+
+.. code-block:: yaml
+
+  # Ensure this data block is present in your argocd-rbac-cm configmap.
+  # It enables the permissions for holmes to fetch the data it needs to
+  # investifate argocd issues.
+  # -
+  # These permissions depend on a new user `holmesgpt` being created,
+  # for example using the `argocd-cm` configmap
+  data:
+    policy.default: role:readonly
+    policy.csv: |
+      p, role:admin, *, *, *, allow
+      p, role:admin, accounts, apiKey, *, allow
+      p, holmesgpt, accounts, apiKey, holmesgpt, allow
+      p, holmesgpt, projects, get, *, allow
+      p, holmesgpt, applications, get, *, allow
+      p, holmesgpt, repositories, get, *, allow
+      p, holmesgpt, clusters, get, *, allow
+      p, holmesgpt, applications, manifests, */*, allow
+      p, holmesgpt, applications, resources, */*, allow
+      g, admin, role:admin
+
+
+In addition to setting permissiong and generating an auth token, you will need to tell argocd how to connect to the server.
+This can be done two ways:
 
 1. **Using port forwarding**. This is the recommended approach if your argocd is inside your Kubernetes cluster.
 2. **Setting the env var** ``SERVER_URL``. This is the recommended approach if your argocd is reachable through a public DNS
@@ -39,7 +67,7 @@ HolmesGPT needs permission to establish a port-forward to ArgoCD. The configurat
         toolsets:
             argocd/core:
                 enabled: true
-                
+
 .. note::
 
     Change the namespace ``--port-forward-namespace <your_argocd_namespace>`` to the namespace in which your argocd service
@@ -92,10 +120,10 @@ This is the recommended approach if your argocd is reachable through a public DN
               argocd/core:
                   enabled: true
 
-    To test, run: 
+    To test, run:
 
     .. code-block:: yaml
-      
+
         holmes ask "Which argocd applications are failing and why?"
 
 Capabilities
@@ -109,12 +137,16 @@ Capabilities
 
    * - Tool Name
      - Description
+   * - argocd_app_list
+     - List the applications in Argocd
    * - argocd_app_get
      - Retrieve information about an existing application, such as its status and configuration
+   * - argocd_app_manifests
+     - Retrieve manifests for an application
+   * - argocd_app_resources
+     - List resources of an application
    * - argocd_app_diff
      - Display the differences between the current state of an application and the desired state specified in its Git repository
-   * - argocd_app_list
-     - List the applications in Argocd
    * - argocd_app_history
      - List the deployment history of an application in ArgoCD
    * - argocd_repo_list

docs/configuration/holmesgpt/toolsets/coralogix_logs.rst

@@ -0,0 +1,115 @@
+.. _toolset_coralogix_logs:
+
+Coralogix logs
+==============
+
+By enabling this toolset, HolmesGPT will fetch node and pods logs from `Coralogix <https://coralogix.com/>`_.
+
+You **should** enable this toolset to replace the default :ref:`kubernetes/logs <toolset_kubernetes_logs>`
+toolset if all your kubernetes/pod logs are consolidated inside Coralogix. It will make it easier for HolmesGPT
+to fetch incident logs, including the ability to precisely consult past logs.
+
+
+.. include:: ./_toolsets_that_provide_logging.inc.rst
+
+Prerequisites
+^^^^^^^^^^^^^
+
+1. A `Coralogix API key <https://coralogix.com/docs/developer-portal/apis/data-query/direct-archive-query-http-api/#api-key>`_ which is assigned the ``DataQuerying`` permission preset
+2. A `Coralogix domain <https://coralogix.com/docs/user-guides/account-management/account-settings/coralogix-domain/>`_. For example ``eu2.coralogix.com``
+3. Your team's `name or hostname <https://coralogix.com/docs/user-guides/account-management/organization-management/create-an-organization/#teams-in-coralogix>`_. For example ``your-company-name``
+
+
+You can deduct the ``domain`` and ``team_hostname`` configuration fields by looking at the URL you use to access the Coralogix UI.
+
+For example if you access coralogix at ``https://my-team.app.eu2.coralogix.com/``` then the ``team_hostname`` is ``my-team``and the coralogix ``domain`` is ``eu2.coralogix.com``.
+
+Configuration
+^^^^^^^^^^^^^
+
+.. md-tab-set::
+
+  .. md-tab-item:: Robusta Helm Chat
+
+    .. code-block:: yaml
+
+      holmes:
+        toolsets:
+          coralogix/logs:
+            enabled: true
+            config:
+              api_key: <your coralogix API key>
+              domain: eu2.coralogix.com # Your Coralogix domain
+              team_hostname: my-team # Your team's hostname in coralogix, without the domain part
+              labels:
+                pod: "kubernetes.pod_name"
+                namespace: "kubernetes.namespace_name"
+                app: "kubernetes.labels.app"
+
+          kubernetes/logs:
+            enabled: false # Disable HolmesGPT's default logging mechanism
+
+
+    .. include:: ./_toolset_configuration.inc.rst
+
+  .. md-tab-item:: Holmes CLI
+
+    Add the following to **~/.holmes/config.yaml**, creating the file if it doesn't exist:
+
+    .. code-block:: yaml
+
+      toolsets:
+        coralogix/logs:
+          enabled: true
+          config:
+            api_key: <your coralogix API key>
+            domain: eu2.coralogix.com # Your Coralogix domain
+            team_hostname: my-team # Your team's hostname in coralogix
+            labels:
+              pod: "kubernetes.pod_name"
+              namespace: "kubernetes.namespace_name"
+              app: "kubernetes.labels.app"
+
+        kubernetes/logs:
+          enabled: false # Disable HolmesGPT's default logging mechanism
+
+
+**Search labels**
+
+You can tweak the labels used by the toolset to identify kubernetes resources. This is only needed if your
+logs settings for ``pod``, ``namespace``, and ``app`` differ from the defaults in the example above.
+
+You can verify what labels to use by attempting to run a query in the coralogix ui:
+
+.. image:: /images/coralogix-logs-for-holmes-labels.png
+  :width: 600
+  :align: center
+
+
+**Disabling the default toolset**
+
+If Coralogix is your primary datasource for logs, it is **advised** to disable the default HolmesGPT logging
+tool by disabling the ``kubernetes/logs`` toolset. Without this. HolmesGPT may still use kubectl to
+fetch logs instead of Coralogix.
+
+.. code-block:: yaml
+
+    holmes:
+        toolsets:
+            kubernetes/logs:
+                enabled: false
+
+
+Capabilities
+^^^^^^^^^^^^
+
+.. include:: ./_toolset_capabilities.inc.rst
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - Tool Name
+     - Description
+   * - coralogix_fetch_logs
+     - Retrieve logs from Coralogix

docs/configuration/holmesgpt/toolsets/datetime.rst

@@ -10,6 +10,7 @@ The following built-in toolsets depend on ``datetime``:
 
 * :ref:`grafana/loki <toolset_grafana_loki>`
 * :ref:`prometheus/metrics <toolset_prometheus>`
+* :ref:`coralogix/logs <toolset_coralogix_logs>`
 
 Configuration
 -------------

docs/configuration/holmesgpt/toolsets/grafanaloki.rst

@@ -10,6 +10,9 @@ You **should** enable this toolset to replace the default :ref:`kubernetes/logs
 toolset if all your kubernetes/pod logs are consolidated inside Loki. It will make it easier for HolmesGPT
 to fetch incident logs, including the ability to precisely consult past logs.
 
+
+.. include:: ./_toolsets_that_provide_logging.inc.rst
+
 Prerequisites
 ^^^^^^^^^^^^^
 
@@ -136,6 +139,7 @@ Use the following commands to list Loki's labels and determine which ones to use
     curl http://localhost:3100/loki/api/v1/labels
 
 
+**Disabling the default toolset**
 
 If Loki is your primary datasource for logs, it is **advised** to disable the default HolmesGPT logging
 tool by disabling the ``kubernetes/logs`` toolset. Without this. HolmesGPT may still use kubectl to

docs/configuration/holmesgpt/toolsets/kubernetes.rst

@@ -63,11 +63,10 @@ it.
 By enabling this toolset, HolmesGPT will be able to read kubernetes pod logs.
 
 You may disable this toolset if you provide Holmes with an alternative toolset to
-fetch logs from your kubernetes cluster and applications. The available built-in
-toolsets that provide logging are:
+fetch logs from your kubernetes cluster and applications.
 
-* :ref:`grafana/loki <toolset_grafana_loki>`: Access Loki logs by proxying through a Grafana instance
 
+.. include:: ./_toolsets_that_provide_logging.inc.rst
 
 Configuration
 ^^^^^^^^^^^^^