[New integration] Hugging Face TGI (DataDog#20905)

* Add skeleton

* Add payload fixture

* Add tests

* Update models

* Fill out metadata.csv

* Fix tests

* Implement python logic

* Fix missing import

* Fix E2E test

* Fix ddev validations

* Run linter

* Rename changelog

* Fix metadata.csv metric types and unit names

* Fix manifest.json

* Fix metric types in metadata.csv again

* Sync CI

* Update README

* Add metrics check to manifest

* Sync configuration files

* Sync configuration files

* Fix metadata.csv for validation

* Sync labeler

* Attempt a real E2E env

* Revert "Attempt a real E2E env"

This reverts commit 11c00ea.

* Remove images

* Bump minimum base package version

* Remove `le` label rename

---------

Co-authored-by: Steven Yuen <steven.yuen@datadoghq.com>

Author: dkirov-dd

.codecov.yml

@@ -286,6 +286,10 @@ coverage:
         target: 75
         flags:
         - hazelcast
+      Hugging_Face_TGI:
+        target: 75
+        flags:
+        - hugging_face_tgi
       IBM_ACE:
         target: 75
         flags:
@@ -1183,6 +1187,11 @@ flags:
     paths:
     - http_check/datadog_checks/http_check
     - http_check/tests
+  hugging_face_tgi:
+    carryforward: true
+    paths:
+    - hugging_face_tgi/datadog_checks/hugging_face_tgi
+    - hugging_face_tgi/tests
   ibm_ace:
     carryforward: true
     paths:

.github/workflows/config/labeler.yml

@@ -299,6 +299,8 @@ integration/hubspot_content_hub:
 - hubspot_content_hub/**/*
 integration/hudi:
 - hudi/**/*
+integration/hugging_face_tgi:
+- hugging_face_tgi/**/*
 integration/hyperv:
 - hyperv/**/*
 integration/iam_access_analyzer:

.github/workflows/test-all.yml

@@ -1522,6 +1522,25 @@ jobs:
       minimum-base-package: ${{ inputs.minimum-base-package }}
       pytest-args: ${{ inputs.pytest-args }}
     secrets: inherit
+  jc3781e1:
+    uses: ./.github/workflows/test-target.yml
+    with:
+      job-name: Hugging Face TGI
+      target: hugging_face_tgi
+      platform: linux
+      runner: '["ubuntu-22.04"]'
+      repo: "${{ inputs.repo }}"
+      python-version: "${{ inputs.python-version }}"
+      latest: ${{ inputs.latest }}
+      agent-image: "${{ inputs.agent-image }}"
+      agent-image-py2: "${{ inputs.agent-image-py2 }}"
+      agent-image-windows: "${{ inputs.agent-image-windows }}"
+      agent-image-windows-py2: "${{ inputs.agent-image-windows-py2 }}"
+      test-py2: ${{ inputs.test-py2 }}
+      test-py3: ${{ inputs.test-py3 }}
+      minimum-base-package: ${{ inputs.minimum-base-package }}
+      pytest-args: ${{ inputs.pytest-args }}
+    secrets: inherit
   j5a9585a:
     uses: ./.github/workflows/test-target.yml
     with:

hugging_face_tgi/CHANGELOG.md

@@ -0,0 +1,4 @@
+# CHANGELOG - Hugging Face TGI
+
+<!-- towncrier release notes start -->
+

hugging_face_tgi/README.md

@@ -0,0 +1,82 @@
+# Agent Check: Hugging Face TGI
+
+## Overview
+
+This check monitors [Hugging Face Text Generation Inference (TGI)][1] through the Datadog Agent. TGI is a toolkit for deploying and serving Large Language Models (LLMs) optimized for text generation with features like continuous batching, tensor parallelism, token streaming, and production-ready optimizations.
+
+The integration provides comprehensive monitoring of your TGI servers by collecting:
+- Request performance metrics including latency, throughput, and token generation rates
+- Batch processing metrics for inference optimization insights
+- Queue depth and request flow monitoring
+- Model serving health and operational metrics
+
+This enables teams to optimize LLM inference performance, track resource utilization, troubleshoot bottlenecks, and ensure reliable model serving at scale.
+
+## Setup
+
+Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the [Autodiscovery Integration Templates][3] for guidance on applying these instructions.
+
+### Installation
+
+The Hugging Face TGI check is included in the [Datadog Agent][2] package.
+No additional installation is needed on your server.
+
+### Configuration
+
+#### Metrics
+
+1. Ensure that your TGI server is exposing Prometheus metrics on the default metrics endpoint. TGI automatically exposes metrics at `/metrics` endpoint when running. For more information about TGI monitoring, see the [official documentation][10].
+
+2. Edit the `hugging_face_tgi.d/conf.yaml` file, which is located in the `conf.d/` folder at the root of your [Agent's configuration directory][11], to start collecting your Hugging Face TGI performance data. See the [sample hugging_face_tgi.d/conf.yaml][4] for all available configuration options.
+
+   ```yaml
+   instances:
+     - openmetrics_endpoint: http://localhost:80/metrics
+   ```
+
+3. [Restart the Agent][5].
+
+### Validation
+
+[Run the Agent's status subcommand][6] and look for `hugging_face_tgi` under the Checks section.
+
+## Data Collected
+
+### Metrics
+
+See [metadata.csv][7] for a list of metrics provided by this integration.
+
+Key metrics include:
+
+- **Request metrics**: Total requests, successful requests, failed requests, and request duration
+- **Queue metrics**: Queue size and queue duration for monitoring throughput bottlenecks
+- **Token metrics**: Generated tokens, input length, and mean time per token for performance analysis
+- **Batch metrics**: Batch size, batch concatenation, and batch processing durations for optimization insights
+- **Inference metrics**: Forward pass duration, decode duration, and filter duration for model performance monitoring
+
+### Events
+
+The Hugging Face TGI integration does not include any events.
+
+### Service Checks
+
+See [service_checks.json][8] for a list of service checks provided by this integration.
+
+## Troubleshooting
+
+In containerized environments, ensure that the Agent has network access to the TGI metrics endpoint specified in the `hugging_face_tgi.d/conf.yaml` file.
+
+Need help? Contact [Datadog support][9].
+
+
+[1]: https://huggingface.co/docs/text-generation-inference/index
+[2]: /account/settings/agent/latest
+[3]: https://docs.datadoghq.com/agent/kubernetes/integrations/
+[4]: https://github.com/DataDog/integrations-core/blob/master/hugging_face_tgi/datadog_checks/hugging_face_tgi/data/conf.yaml.example
+[5]: https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent
+[6]: https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information
+[7]: https://github.com/DataDog/integrations-core/blob/master/hugging_face_tgi/metadata.csv
+[8]: https://github.com/DataDog/integrations-core/blob/master/hugging_face_tgi/assets/service_checks.json
+[9]: https://docs.datadoghq.com/help/
+[10]: https://huggingface.co/docs/text-generation-inference/en/basic_tutorials/monitoring
+[11]: https://docs.datadoghq.com/agent/configuration/agent-configuration-files/#agent-configuration-directory

hugging_face_tgi/assets/configuration/spec.yaml

@@ -0,0 +1,15 @@
+name: Hugging Face TGI
+files:
+- name: hugging_face_tgi.yaml
+  options:
+    - template: init_config
+      options:
+        - template: init_config/openmetrics
+    - template: instances
+      options:
+        - template: instances/openmetrics
+          overrides:
+            openmetrics_endpoint.value.example: http://localhost:80/metrics
+            openmetrics_endpoint.description: |
+              Endpoint exposing Hugging Face TGI's Prometheus metrics. For more information, refer to
+              https://huggingface.co/docs/text-generation-inference/en/basic_tutorials/monitoring 

hugging_face_tgi/changelog.d/20905.added

@@ -0,0 +1 @@
+Initial Release

hugging_face_tgi/datadog_checks/hugging_face_tgi/__about__.py

@@ -0,0 +1,4 @@
+# (C) Datadog, Inc. 2025-present
+# All rights reserved
+# Licensed under a 3-clause BSD style license (see LICENSE)
+__version__ = '0.0.1'

hugging_face_tgi/datadog_checks/hugging_face_tgi/__init__.py

@@ -0,0 +1,7 @@
+# (C) Datadog, Inc. 2025-present
+# All rights reserved
+# Licensed under a 3-clause BSD style license (see LICENSE)
+from .__about__ import __version__
+from .check import HuggingFaceTgiCheck
+
+__all__ = ['__version__', 'HuggingFaceTgiCheck']

hugging_face_tgi/datadog_checks/hugging_face_tgi/check.py

@@ -0,0 +1,24 @@
+# (C) Datadog, Inc. 2025-present
+# All rights reserved
+# Licensed under a 3-clause BSD style license (see LICENSE)
+from datadog_checks.base import OpenMetricsBaseCheckV2
+from datadog_checks.hugging_face_tgi.metrics import METRIC_MAP, RENAME_LABELS_MAP
+
+
+class HuggingFaceTgiCheck(OpenMetricsBaseCheckV2):
+    __NAMESPACE__ = 'hugging_face_tgi'
+
+    DEFAULT_METRIC_LIMIT = 0
+
+    def __init__(self, name, init_config, instances=None):
+        super(HuggingFaceTgiCheck, self).__init__(
+            name,
+            init_config,
+            instances,
+        )
+
+    def get_default_config(self):
+        return {
+            'metrics': [METRIC_MAP],
+            'rename_labels': RENAME_LABELS_MAP,
+        }