Skip to content

docs(open_feature): make scoped agent guidance discoverable#6029

Merged
leoromanovsky merged 4 commits into
masterfrom
leo.romanovsky/open-feature-agents-customer-review
Jul 14, 2026
Merged

docs(open_feature): make scoped agent guidance discoverable#6029
leoromanovsky merged 4 commits into
masterfrom
leo.romanovsky/open-feature-agents-customer-review

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

AI-generated: This PR includes a small AI-generated, repository-local Codex eval runner and fixtures.

Motivation

The scoped OpenFeature guide shipped in #5968 and was refined in #5989. However, after #5910 had merged master containing that guide, human review still had to explicitly redirect the contributor to AGENTS.md. A contributor should discover applicable guidance before beginning OpenFeature work, including during contribution-rotation handoffs.

We turned that suspicion into a red eval before relying on a documentation change. Three fresh-session awareness cases cover OpenFeature work under lib/, spec/, and sig/; running them against the exact pre-PR commit, 7df73817ee, produced:

PASS lib_awareness
FAIL sig_awareness
  missing instruction files: lib/datadog/open_feature/AGENTS.md
  result: {"instruction_files":["AGENTS.md"]}
FAIL spec_awareness
  missing instruction files: lib/datadog/open_feature/AGENTS.md
  result: {"instruction_files":["AGENTS.md"]}

The nested guide was discovered for work under lib/, but not for the sibling spec and signature trees it claims to govern. That failing evidence is the motivation for this change.

Changes

The root AGENTS.md now explicitly routes OpenFeature code, spec, and signature work to lib/datadog/open_feature/AGENTS.md. A lightweight runner starts fresh, read-only Codex sessions and asserts that they identify both the root and scoped instruction files. The scoped guide's content is unchanged.

Decisions

This first phase proves discovery only; it does not rewrite the scoped guide or test individual lint conventions. The evals remain dependency-free, manual, ephemeral, read-only, and outside CI and LLMObs for now.

Next steps

The stacked follow-up #6034 adds a writable content-adherence eval proving that agents find and run the configured RuboCop rules, then independently verifies that the resulting code is lint-clean. It depends on #6032 making the root RuboCop workflow operational and keeps individual rules in executable configuration rather than duplicating them in AGENTS.md.

Validation

Running the same cases on this PR produces:

PASS lib_awareness
PASS sig_awareness
PASS spec_awareness
3 eval(s) passed

The runner also passes Ruby syntax, JSON parsing, and git diff --check.

Impact

Future OpenFeature contributors and their tools should have the scoped guide in context before changing code, specs, or signatures. This should reduce guidance-related review churn while leaving human reviewers focused on architecture and APM customer needs.

Change log entry

None.

@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

👋 Hey @DataDog/ruby-guild, please fill "Change log entry" section in the pull request description.

If changes need to be present in CHANGELOG.md you can state it this way

**Change log entry**

Yes. A brief summary to be placed into the CHANGELOG.md

(possible answers Yes/Yep/Yeah)

Or you can opt out like that

**Change log entry**

None.

(possible answers No/Nope/None)

Visited at: 2026-07-13 15:42:26 UTC

@dd-octo-sts dd-octo-sts Bot added docs Involves documentation openfeature A new component that provider an ability to configure feature flags labels Jul 13, 2026
@leoromanovsky leoromanovsky added the AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos label Jul 13, 2026
@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Jul 13, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 90.04%

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b811b5b | Docs | Datadog PR Page | Give us feedback!

Comment thread tools/agent-evals/open_feature_guidance/cases/lib_awareness.json Outdated
Comment thread lib/datadog/open_feature/AGENTS.md Outdated
@leoromanovsky leoromanovsky changed the title docs(open_feature): focus agent guidance on customer correctness docs(open_feature): make scoped agent guidance discoverable Jul 13, 2026
@leoromanovsky
leoromanovsky marked this pull request as ready for review July 13, 2026 17:14
@leoromanovsky
leoromanovsky requested a review from a team as a code owner July 13, 2026 17:14
@leoromanovsky
leoromanovsky merged commit 4e01cc7 into master Jul 14, 2026
311 checks passed
@leoromanovsky
leoromanovsky deleted the leo.romanovsky/open-feature-agents-customer-review branch July 14, 2026 15:06
@dd-octo-sts dd-octo-sts Bot added this to the 2.38.0 milestone Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos docs Involves documentation openfeature A new component that provider an ability to configure feature flags

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants