[pull] master from DataDog:master#588
Merged
Merged
Conversation
…23529) * postgres data-observability - use cron to precisely schedule queries Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * review Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * fix: apply ddev formatter Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * Skip invalid queries at construction time Signed-off-by: Maciej Obuchowski <maciej.obuchowski@datadoghq.com> * remove lookback as configurable param Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * Replace croniter with datadog_checks_base CronScheduler utility Drop the third-party croniter==6.2.2 dependency and rewrite the cron scheduling in PostgresDataObservability on top of the CronExpression / CronScheduler utilities added to datadog_checks_base in #23741. Changes: - _filter_valid_queries: build a CronScheduler per monitor_id inside a try/except(ValueError, TypeError) instead of calling croniter.is_valid. Scheduler captures startup_lookback at construction time. - _get_due_queries: collapse the entire cron branch to a single due_ticks(now) call; state registration, lookback recovery, tick detection, and advancement are all handled by CronScheduler. - run_job: remove the post-fire croniter re-advance; due_ticks() already advanced the scheduler at poll time. - Tests: replace _next_run[mid] accesses with _schedulers[mid].next_tick; update boundary test to reflect CronScheduler's inclusive (<=) lookback semantics vs the old strict (<) comparison. - Remove croniter from agent_requirements.in, postgres/pyproject.toml, and LICENSE-3rdparty.csv; bump datadog-checks-base pin to >=37.39.0 (first release shipping cron.py). Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * Address code review: type hints, explicit state returns, lateness comment, error guard - _filter_valid_queries: add Iterable[Query] parameter type; return (tuple[Query,...], dict[int,CronScheduler]) so callers own the assignment — removes the hidden self._schedulers side-effect. __init__ now assigns both fields explicitly. - _get_due_queries: remove setdefault seed for _last_execution; use .get() and treat None as first-sight. Scheduling-state mutations (seed and advance) are now consolidated in run_job. - run_job: add comment distinguishing lateness (scheduling delay) from execution time; wrap emit_failures count in a nested try/except so a concurrent _shutdown() cannot turn the error handler into a job crash. - Tests: merge _make_cron_lookback_check into _make_cron_check via optional window_seconds/monkeypatch params; collapse three identical lookback-window tests into one parametrized test_cron_startup_lookback_window_behavior. Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * fmt Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * code review fixes Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * postgres DO: fix cron boundary miss and improve schedule error message - due_ticks now probes now+0.001 so a first poll landing exactly on a cron tick boundary fires instead of skipping the cycle - invalid cron schedule warning now includes the exception message and the monitor_id to help customers identify and fix the bad config - regression test added for the exact-boundary case Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * revert cron.py boundary fix (moved to postgres data_observability) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * fix cron exact-boundary detection without inflating lookback window The previous +0.001 epsilon was passed directly to due_ticks(), which caused CronScheduler to compare (now+0.001) - prev <= lookback, breaking the inclusive-boundary test where now - prev == lookback exactly. Instead, call due_ticks(now) normally and only on the first poll (before next_tick is cached) probe previous_tick(before=now+0.001) to detect whether now itself is a tick. This isolates the epsilon to the boundary detection and leaves the lookback window comparison untouched. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> * simplify cron boundary fix; update over-precise test Use due_ticks(now + 0.001) — the simpler approach. The previous complex wrapper was added to avoid breaking test_cron_startup_lookback_boundary_inclusive, which was asserting now - prev == window exactly. That exact-equality check belongs in test_cron.py (CronScheduler unit tests), not here. Updated the test to use a clock 55s after the tick (clearly inside the 60s window), keeping the meaningful assertion that recovery fires inside the window and does not fire outside it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> --------- Signed-off-by: mobuchowski <maciej.obuchowski@datadoghq.com> Signed-off-by: Maciej Obuchowski <maciej.obuchowski@datadoghq.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Add initial scaffolding * Remove manifest * Add implementation * Remove unnecessary files * Fix claude's suggestions * Address Claude's comments * Optimiza calls * Fix mock * Fix validations and skip lab on E2E * Fix typo * Map traffic type to overlay * Address Claude's comments * Address Claude's comments * Fix tags * Uncompress fixtures * Removed unused appliance values * Sync conf example * Address Claude's comments * Sync conf example * Address Claude's comments * Fix README format * Sync conf example * Fix Claude's suggestion * Fix typing * Change metric format * Add concurrency to minute stats requests * Sync models * Remove None type option from namespace * Add parsing of tunnel color for edge cases * Add send_ndm_metadata * Remove unnecessary dd_save_state * Add _parse_config, config tags, and dedicated orch credentials - Add _parse_config method to resolve config once during initialization - Include user-configured tags in all emitted metrics - Replace reused HTTP basic auth username/password with dedicated orch_username/orch_password fields - Remove unnecessary auth=None override since basic auth fields are no longer populated Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Sync config descriptions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Address comments * Validate ci * Refactor tests * docs(hpe_aruba_edgeconnect): add fleet_configurable flags and clarify permissions - Mark integration and all instance options as fleet_configurable in spec.yaml - Clarify README that Monitor permissions are sufficient for most metrics Rationale: Enable Fleet Automation support and correct misleading permission requirements This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * Add alarm events * Address comments * Address comments * Document persist connections and remove header * Update hpe_aruba_edgeconnect/README.md Co-authored-by: Janine Chan <64388808+janine-c@users.noreply.github.com> * Update hpe_aruba_edgeconnect/assets/configuration/spec.yaml Co-authored-by: Janine Chan <64388808+janine-c@users.noreply.github.com> * Update hpe_aruba_edgeconnect/assets/configuration/spec.yaml Co-authored-by: Janine Chan <64388808+janine-c@users.noreply.github.com> * Sync config --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Janine Chan <64388808+janine-c@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )