Skip to content

Commit 7c46f52

Browse files
committed
Add LLM policy
1 parent 7291298 commit 7c46f52

1 file changed

Lines changed: 68 additions & 0 deletions

File tree

content/wiki/llm_policy.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# LLM contribution policy for Linebender projects
2+
3+
Open-source projects are facing an increasing amount of submissions generated in whole or in part by LLMs.
4+
5+
Maintainers tend to strongly dislike them: they require very little effort to create (since a machine is doing all the work) but a lot of effort to review (since LLMs make mistakes that are hard to track down). Because LLMs are good at mimmicking high-effort contributors, maintainers often feel pressured to give the benefit of the doubt to the submitter and give a level of feedback that isn't warranted by the effort the submitter made.
6+
7+
Still, LLM agents can produce high-quality code, so we don't want to band them altogether.
8+
9+
This document outlines Linebender's official policy for LLM contributions, inspired by [this zulip discussion](https://xi.zulipchat.com/#narrow/channel/419691-linebender/topic/AI.20slop.20policy/near/575407715).
10+
11+
12+
## Disclosure
13+
14+
Contributors should disclose their LLM usage ahead of time. Reviewers will *not* be happy if a contributor only admits to using LLMs after being prodded, and will tend to assume the reviewer is downplaying their level of AI use.
15+
16+
This above applies even if the reviewer double-checked everything the LLM wrote. In our experience, people overestimate their level of understanding of agent-produced code when the agent had a high level of initiative. "Code you wrote yourself" and "Code you read after the LLM wrote it" do not warrant the same level of scrutiny.
17+
18+
Contributors should also disclose content that wasn't written by an agent, but was a direct result of LLM outputs, notably:
19+
20+
- Code or documentation that was created following an outline, plan or architecture proposed by an LLM.
21+
- Code that was copy-pasted by an AI chat tool (e.g. ChatGPT), even if the code was double-checked or adjusted.
22+
23+
This disclosure should be included in the PR description, so that it appears in the final commit message.
24+
25+
### Copilot-like tools
26+
27+
Tools like Github Copilot or Cursor Tab that do small tab-completion of code you're writing fall in a grey area.
28+
29+
Completing single lines of code or small blocks of boilerplate is fine and doesn't require disclosure.
30+
31+
Completing large blocks of code or entire functions, however, is similar to asking an agent to generate it for you. Writing a comment that says "The following does X" and then hitting tab a few times is almost identical to using an agent.
32+
33+
As a rule of thumb, in Rust, by the time Copilot produces completions with multiple `;` characters, you should treat it as LLM code that requires disclosure.
34+
35+
### Non-generation use of LLM
36+
37+
Using LLMs for debugging, exploration, testing (*not* generating test files) is accepted without disclosure.
38+
39+
### Anti-disclosure
40+
41+
Given the above, declaring that a PR *doesn't* include AI-generated content is redundant and unverifiable. Please don't include such messages unless asked.
42+
43+
44+
## Level of effort
45+
46+
The rule of thumb for all LLM content is "You should not ask someone to read text if reading it would take longer than it took you to create it".
47+
48+
As such, we do not allow LLM bots, PRs that were generated end-to-end by LLMs, or AI-generated PR descriptions (translations are fine with disclosure).
49+
50+
If a PR includes AI-generated content, we fully expect the submitter to review their own PR before asking anyone else to look at it. They should spend as much effort on this self-review as they would on a human-authored PR.
51+
52+
In discussion spaces like Github comments and the Zulip server, please avoid posting AI-generated analyses, even if you vetted them.
53+
54+
55+
## Documentation
56+
57+
In general, avoid generating *any* documentation longer than one line with AI. *Never* generate documentation longer than two paragraphs with AI.
58+
59+
AI-generated documentation tends to be very verbose and redundant. Because generating it doesn't cost the contributor anything, they lean towards "more is better", which translates to a worse reading experience.
60+
61+
If you're not confident in your documentation skills, writing stub documentation with TODO comments is better than verbose-but-meaningless LLM documentation.
62+
63+
64+
## Agent files
65+
66+
We will not merge agentic markdown files.
67+
68+
Some projects may include common agent files in their `.gitignore`.

0 commit comments

Comments
 (0)