feat: last copy pipe executions monitoring endpoint#3520
Conversation
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
This PR is being reviewed by Cursor BugbotDetailsYour team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team. To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial. |
Bug: MultiIf Logic Fails on Consecutive ErrorsThe |
Creates a new tinybird pipe for copy pipe executions
Monitors Tinybird copy job executions and exposes their status as Prometheus gauge metrics. Consumed by datadog scraper (
dd-tinybird-copy-pipe-executions-scraper)Tracks the latest execution status for each copy pipe (starting today) and generates one-hot encoded metrics where each pipe gets a value of
1for its current status ('ok','error','cancelled','queued','working') and0for all other statuses.Metric:
copy_pipes_latest_execution_status(gauge)Labels:
pipe_name,statusNote: The
'queued'status is virtual—it's derived by detecting the error message"You have reached the maximum number of copy jobs"rather than being a native Tinybird status. This is because tinybird retries these again when possible but returns an error status. Similarly,'ok'is mapped from the native'done'status for datadog color-scheme convention.