docs: add viability test to contribution guidelines#641
Open
JensGrote wants to merge 1 commit into
Open
Conversation
Recognition ≠ activation. The existing activation test checks whether a model knows a term, but not whether naming it changes the output. Add a Viability Test section (EN + DE) that checks: - Before/After structural change - Silent substitution risk - Confabulation risk - Cross-model portability Also: rename 'Not in training data' rejection category to 'Insufficient training-data density' with expanded definition covering the three failure modes. Refs: training-data-vs-practice article, LLM-Coding#585
Contributor
WalkthroughDie PR ergänzt das Issue-Template und die Beitragsdokumentation um einen Viability-Test für semantische Anker, aktualisiert die deutsche und öffentliche Doku dazu und benennt in den abgelehnten Vorschlägen eine Kategorie um. ChangesViability-Test und Dokumentation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/rejected-proposals.adoc`:
- Around line 12-13: The rejected-proposals category rename is only partially
applied, leaving mixed terminology between “Insufficient training-data density”
and the older label. Update the relevant rejected-proposals documentation and
any linked references in the same proposal stack so the category is named
consistently everywhere, using the existing anchor and section names in this
document to locate the affected entries.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: 8fbd932c-68e8-4cac-bf96-838204d909f5
📒 Files selected for processing (5)
.github/ISSUE_TEMPLATE/propose-anchor.ymlCONTRIBUTING.adocdocs/CONTRIBUTING.de.adocdocs/rejected-proposals.adocwebsite/public/CONTRIBUTING.de.adoc
Comment on lines
+12
to
+13
| Insufficient training-data density:: | ||
| The term is recognised (models can talk _about_ it) but does not reliably _activate_ the concept in outputs. Naming it does not structurally change the output compared to a generic prompt, or triggers silent substitution / confabulation on weaker models. Terms in this category may become viable anchors as training data grows — consider a link:#/contracts[contract] for now. See link:#/training-data-vs-practice[An Anchor Delivers Only as Far as the Prior Reaches] for the empirical methodology. Fails the viability test. |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Kategorie-Umbenennung nicht isoliert ausrollen.
Hier wird „Insufficient training-data density“ bereits eingeführt, aber der bereitgestellte Kontext zeigt, dass docs/plans/2026-03-09-rejected-proposals-design.md weiterhin „Not in training data“ verwendet. Bitte die Umbenennung und alle Referenzen im selben Stack aktualisieren, sonst entsteht eine doppelte Bezeichnung für dieselbe Kategorie.
Also applies to: 45-45
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/rejected-proposals.adoc` around lines 12 - 13, The rejected-proposals
category rename is only partially applied, leaving mixed terminology between
“Insufficient training-data density” and the older label. Update the relevant
rejected-proposals documentation and any linked references in the same proposal
stack so the category is named consistently everywhere, using the existing
anchor and section names in this document to locate the affected entries.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
(German version below)
docs: add viability test to contribution guidelines
Recognition ≠ activation. The current contribution workflow tests whether a model recognizes a proposed anchor term — but not whether naming it actually changes the output. The training-data-vs-practice article demonstrated empirically that these are different things: "Use-Case 3.0" is recognized by every model, yet naming it produces silent substitution or confabulation rather than reliable activation.
Changes
1. New section in CONTRIBUTING.adoc (EN + DE): "Viability Test — Does the Anchor Deliver?"
Positioned between "Testing Your Semantic Anchor" and "Developer Setup". Describes:
2. New optional field in issue template (propose-anchor.yml)
"Viability Test" textarea — encourages proposers to include a before/after comparison. Starts as optional; can be made mandatory once adopted.
3. Renamed rejection category (rejected-proposals.adoc)
"Not in training data" → "Insufficient training-data density" — expanded definition covering the three failure modes and pointing to the article for methodology. Updated the MIRRR UX Framework entry to use the new category name.
Rationale
The existing four quality criteria (Precise, Rich, Consistent, Attributable) are necessary but not sufficient. A term can pass all four yet still fail in practice because:
The viability test makes the implicit ★★★/★★/★ tier distinction explicit and testable — providing clear criteria for reviewers and a methodology proposers can follow.
Not included (deliberately)
Deutsche Version
docs: Viability-Test zu den Contribution-Guidelines hinzufügen
Erkennung ≠ Aktivierung. Der aktuelle Contribution-Workflow testet, ob ein Modell einen vorgeschlagenen Anchor-Term erkennt — aber nicht, ob das Benennen die Ausgabe tatsächlich verändert. Der training-data-vs-practice-Artikel hat empirisch gezeigt, dass das verschiedene Dinge sind: "Use-Case 3.0" wird von jedem Modell erkannt, produziert aber Silent Substitution oder Confabulation statt zuverlässiger Aktivierung.
Änderungen
1. Neuer Abschnitt in CONTRIBUTING.adoc (EN + DE): "Viability-Test — Liefert der Anker?"
Positioniert zwischen "Testen Ihres semantischen Ankers" und "Entwickler-Setup". Beschreibt:
2. Neues optionales Feld im Issue-Template (propose-anchor.yml)
"Viability Test" Textarea — ermutigt Proposer, einen Before/After-Vergleich einzufügen. Startet als optional; kann obligatorisch werden sobald adoptiert.
3. Umbenannte Rejection-Kategorie (rejected-proposals.adoc)
"Not in training data" → "Insufficient training-data density" — erweiterte Definition die die drei Failure-Modes abdeckt und auf den Artikel für Methodik verweist. MIRRR UX Framework-Eintrag auf neuen Kategorienamen aktualisiert.
Begründung
Die bestehenden vier Qualitätskriterien (Precise, Rich, Consistent, Attributable) sind notwendig aber nicht hinreichend. Ein Term kann alle vier bestehen und dennoch in der Praxis scheitern, weil:
Der Viability-Test macht die implizite ★★★/★★/★ Tier-Unterscheidung explizit und testbar — mit klaren Kriterien für Reviewer und einer Methodik der Proposer folgen können.
Summary by CodeRabbit
Neue Funktionen
Dokumentation