Skip to content

README.md

Latest commit

 

History

History
112 lines (90 loc) · 16.7 KB

README.md

File metadata and controls

112 lines (90 loc) · 16.7 KB
Raw

Terraform AzAPI Sentinel Codeless Connector

Microsoft Sentinel Codeless Connector Framework (CCF) connectors as code, via the Azure/azapi provider. A CCF connector is Sentinel's managed poller-as-a-service: it polls any REST API on a schedule (no Azure Function to run or maintain) and lands the results in a Log Analytics table through a data collection rule. This module builds both halves, the connector's page in Sentinel and its poller connections, from a small typed surface.

CI Release Terraform Registry License

The two halves, one small surface

  • The connector page (dataConnectorDefinition, kind Customizable): only title, publisher, description and graph_table are required. The rest of the page (the ingestion graph, sample query, data-type freshness query, the HasDataConnectors connectivity check, and the workspace prerequisite permissions) is auto-derived from the table name, so a minimal call gets a complete, correct page for free. Every derived section is overridable for an ISV-grade page, and the Instructions tab widgets (Textbox, OAuthForm, CopyableLabel, ...) pass through raw.
  • The poller connections (RestApiPoller dataConnectors), a map keyed by connection name, all linked to the definition. Each has a typed auth (Basic, APIKey, OAuth2, or JwtToken), request surface (endpoint, poll window, the paired time-window attributes, rate limits, retries), response parsing (JSON, CSV, XML), and typed paging covering all seven CCF paging types. The DCR coordinates (dcr block) compose straight from the Libre DevOps data-collection module by id, per the estate's pass-ids principle.

What the module enforces

At plan: the auth type enum, the Custom- DCR stream prefix, the paired start/end time-window attributes, the retry (1 to 6) and timeout (1 to 180) ranges, the response format enum, and the _CL table suffix. Unset optionals are dropped from every body rather than sent as null. An advisory check warns when the UI offers an OAuthForm but no connection actually uses OAuth2 (a common copy-paste mismatch). Both API versions are variables, so consumers are never pinned to this module's release cadence.

Notes worth knowing

  • The azapi embedded schema lags the CCF surface, so schema_validation_enabled = false is set on both resources: the service validates the whole body on the PUT, which is the real authority.
  • Escaping: the Microsoft docs show auth parameters written as [[parameters('x')]. That is an ARM deployment template artefact; through azapi you feed plain Terraform values (a variable, or a Key Vault data source), never that form.
  • Secrets: pass client_secret, api_key and passwords from a variable or a azurerm_key_vault_secret data source. They land in the request body, so treat them as sensitive.

Requirements

Name Version
terraform >= 1.9.0, < 2.0.0
azapi >= 2.0.0, < 3.0.0

Providers

Name Version
azapi >= 2.0.0, < 3.0.0

Modules

No modules.

Resources

Name Type
azapi_resource.connections resource
azapi_resource.definition resource

Inputs

Name Description Type Default Required
connections The poller connections (RestApiPoller dataConnectors) keyed by connection name, all linked to this
module's definition. One definition can carry several connections that poll different endpoints. Per
connection:

- api_endpoint (required): the source REST URL to poll.
- dcr (required): where polled events land, an object of:
- data_collection_endpoint (required): the DCE ingestion URL.
- data_collection_rule_immutable_id (required): the DCR immutable id.
- stream_name (required): the DCR stream, must begin Custom-.
Compose these from the Libre DevOps data-collection module by id (pass-ids).
- auth (required): the authentication object, exactly one type populated:
- type (required): Basic | APIKey | OAuth2 | JwtToken.
- the fields for that type (user_name/password; api_key/api_key_name/api_key_identifier/
is_api_key_in_post_payload; client_id/client_secret/grant_type/token_endpoint/scope/
authorization_endpoint/redirect_uri; jwt user_name/password/token_endpoint/
jwt_token_json_path/...). Prefer feeding secrets from a Key Vault data source or a variable,
never a literal.
- data_type (optional): the connector's dataType label.
- http_method (optional): GET (default) or POST.
- query_window_in_min (optional): poll window minutes (min 1, default 5).
- query_time_format (optional): the endpoint's time format (default ISO 8601 UTC).
- start_time_attribute_name / end_time_attribute_name (optional, paired): the query parameter
names for the poll window bounds.
- headers / query_parameters (optional): request header and query maps.
- rate_limit_qps / retry_count (1..6) / timeout_in_seconds (1..180) (optional): throttling and
resilience.
- response (optional): { events_json_paths (list, default ["$"]), format (json|csv|xml, default
json), success_status_json_path, success_status_value, is_gzip_compressed }.
- paging (optional): raw paging object (pagingType plus its fields), passed through.
- request_extra / properties_extra (optional): raw attributes merged over the generated request /
properties last, for fields this module does not model.		 
map(object({
    api_endpoint = string

    dcr = object({
      data_collection_endpoint          = string
      data_collection_rule_immutable_id = string
      stream_name                       = string
    })

    auth = object({
      type = string

      # Basic
      user_name = optional(string)
      password  = optional(string)

      # APIKey
      api_key                    = optional(string)
      api_key_name               = optional(string)
      api_key_identifier         = optional(string)
      is_api_key_in_post_payload = optional(bool)

      # OAuth2
      client_id              = optional(string)
      client_secret          = optional(string)
      grant_type             = optional(string)
      token_endpoint         = optional(string)
      scope                  = optional(string)
      authorization_endpoint = optional(string)
      authorization_code     = optional(string)
      redirect_uri           = optional(string)

      # JwtToken (user_name/password reused above; token_endpoint reused)
      jwt_token_json_path       = optional(string)
      is_credentials_in_headers = optional(bool)
      is_json_request           = optional(bool)

      # Raw escape hatch merged over the generated auth object last.
      extra = optional(any, {})
    })

    data_type           = optional(string)
    http_method         = optional(string, "Get")
    query_window_in_min = optional(number)
    query_time_format   = optional(string)

    start_time_attribute_name = optional(string)
    end_time_attribute_name   = optional(string)

    headers          = optional(map(string))
    query_parameters = optional(map(string))

    rate_limit_qps     = optional(number)
    retry_count        = optional(number)
    timeout_in_seconds = optional(number)

    response = optional(object({
      events_json_paths        = optional(list(string), ["$"])
      format                   = optional(string, "json")
      success_status_json_path = optional(string)
      success_status_value     = optional(string)
      is_gzip_compressed       = optional(bool)
    }), {})

    # The union of the paging surface across all pagingType values, every field optional. A typed
    # object (rather than any) is required so connections with and without paging unify into the
    # map; set pagingType plus the fields that type needs (see the README paging table).
    paging = optional(object({
      pagingType = string

      pageSize              = optional(number)
      pageSizeParameterName = optional(string)
      pagingInfoPlacement   = optional(string)
      pagingQueryParamOnly  = optional(bool)

      # Link/URL based
      linkHeaderTokenJsonPath            = optional(string)
      nextPageUrl                        = optional(string)
      nextPageParaName                   = optional(string)
      nextPageRequestHeader              = optional(string)
      nextPageUrlQueryParameters         = optional(map(string))
      nextPageUrlQueryParametersTemplate = optional(string)

      # Token based
      nextPageTokenJsonPath       = optional(string)
      nextPageTokenResponseHeader = optional(string)
      hasNextFlagJsonPath         = optional(string)

      # Offset / count based
      offsetParaName       = optional(string)
      pageNumberParaName   = optional(string)
      zeroBasedIndexing    = optional(bool)
      totalResultsJsonPath = optional(string)
      pageNumberJsonPath   = optional(string)
      pageCountJsonPath    = optional(string)
    }))

    request_extra    = optional(map(string), {})
    properties_extra = optional(map(string), {})
  }))
 {} no
connector_api_version API version for Microsoft.SecurityInsights/dataConnectors. 2025-09-01 is the current stable version; variablised so consumers are never pinned to this module's release cadence. string "2025-09-01" no
connector_ui The connector's UI page (connectorUiConfig on the Customizable dataConnectorDefinition). Only the
human-facing text is required: title, publisher, and description; the rest of the page (the
ingestion graph, sample queries, data types, connectivity check, and prerequisite permissions) is
auto-derived from graph_table so a minimal call gets a complete, correct page for free. Every
derived section can be overridden for an ISV-grade page.

- title / publisher / description (required): the page heading, provider, and markdown blurb.
- graph_table (required): the destination custom table name (the poller's stream target, ending
_CL). Drives the default graph query, sample query, data-type freshness query, and the
{{graphQueriesTableName}} placeholder.
- id (optional): internal connector id; defaults to definition_name.
- logo (optional): path to an SVG logo; the platform default is used when null.
- is_preview (optional): mark the connector preview in the gallery.
- graph_queries / sample_queries / data_types (optional): override the auto-derived query sets
(each a list of the documented shapes); null keeps the derived defaults.
- connectivity_criteria (optional): override the connectivity check; defaults to HasDataConnectors
(the recommended check for API pollers, connected once a connection is active).
- resource_provider_permissions (optional): override the prerequisite permission rows; defaults to
read+write on the workspace.
- custom_permissions (optional): extra prerequisite notes (name/description), for example the
source API credential the analyst must supply.
- instruction_steps (optional): the Instructions tab widgets (raw, passed through); defaults to a
single explanatory step. Build credential inputs here (Textbox, OAuthForm, ...).
- extra_ui (optional): raw connectorUiConfig attributes merged over the generated config last, for
fields this module does not model.		 
object({
    title       = string
    publisher   = string
    description = string
    graph_table = string

    id         = optional(string)
    logo       = optional(string)
    is_preview = optional(bool, false)

    graph_queries         = optional(list(any))
    sample_queries        = optional(list(any))
    data_types            = optional(list(any))
    connectivity_criteria = optional(list(any))

    resource_provider_permissions = optional(list(any))
    custom_permissions            = optional(list(object({ name = string, description = string })))
    instruction_steps             = optional(list(any))
    extra_ui                      = optional(any, {})
  })
 n/a yes
definition_api_version API version for Microsoft.SecurityInsights/dataConnectorDefinitions. 2025-09-01 is the current stable version; variablised so consumers are never pinned to this module's release cadence. string "2025-09-01" no
definition_name Name of the dataConnectorDefinition resource (the connector's page in Sentinel). Also the connectorDefinitionName every poller connection links to. string n/a yes
retry_error_message_regex Regular expressions azapi retries on when a connector call fails. Defaults to the freshly created workspace propagation race and transient service noise; null disables retries. list(string) 
[
  "(?i)workspace could not be found",
  "(?i)not found",
  "(?i)too many requests",
  "(?i)service unavailable",
  "(?i)internal server error"
]
 no
workspace_id Resource id of the Sentinel-onboarded Log Analytics workspace the connector definition and its poller connections are created under (the azapi parent), per the pass-ids principle. string n/a yes

Outputs

Name Description
connection_ids Map of connection name to its RestApiPoller data connector resource id.
connection_ids_zipmap Map of connection name to {name, id} for easy composition.
definition_id Resource id of the data connector definition (the connector's page in Sentinel).
definition_name Name of the data connector definition, the connectorDefinitionName every connection links to.
graph_table The destination custom table the connector's UI and its connections target.