Project-wide LLM policy#3959
Conversation
clarfonthey
added
the
T-leadership-council
| * Neurodivergent authors tend to replicate the "terseness" of many LLMs, and often show up as false positives in LLM detection | ||
| * Kenyan authors, many of whom helped filter the data for LLMs, often show up as false positives in LLM detection |
There was a problem hiding this comment.
FWIW, while I've tried to cite everything across the RFC, this section was added after the first draft and I knew both of these had citations, but had a lot of trouble finding explicit articles about it. So, if you happen to find sources for these, I'd be happy to update the RFC to include them.
There was a problem hiding this comment.
When reading that section, I thought back to this article: https://marcusolang.substack.com/p/im-kenyan-i-dont-write-like-chatgpt.
|
|
||
| 1. If the LLM usage is *trivial*, it is completely ignored by the policy and always allowed. Generally, this means that changes made by LLMs are indistinguishable from those made by humans, where the LLM didn't have any creative input into the change. | ||
| 2. If the LLM usage is *slop*, it is considered spam and moderated accordingly. Generally, this means submitting changes made by LLMs with minimal human intervention. | ||
| 3. *Nontrivial* LLM usage must be *disclosed* in ideally as detailed as a manner as possible. This may necessitate additional tooling to notify new contributors about the policy and explain how disclosure works. |
There was a problem hiding this comment.
In PRs I've submitted to rust and cargo, I include a single sentence at the end usually of the form:
No AI tooling of any kind was used during the creation of this PR.
I think including an AI disclosure prompt/template in the GitHub PR template is a simple way to cover off disclosure while allowing the author the freedom to go into as much or as little detail as they feel is appropriate.
There was a problem hiding this comment.
Yeah, having a section on a PR template just to fill in LLM disclosure is a good idea, and that's one of the multiple options that could be used for tooling.
This is actually the one difference between the summary for the RFC and the one proposed for inclusion in the actual policy: for the RFC, the tooling is an important point since it doesn't exist yet, but for the final policy, the goal is to already have that.
I mostly leave the details out of the RFC since they're almost certainly going to be done via experimentation, but this is a pretty easy one to start with.
|
|
||
| While there are arguments to be made about some models having less power consumption, it ultimately doesn't matter if they fundamentally require operation based upon brute force. As hopefully any programmer even vaguely educated on complexity knows, brute force is *the worst* way to solve any problem, and should always be used as a last resort. LLMs put brute force front and center as the best option. | ||
|
|
||
| With code, we have methods to bound operation: for example, a famous case is sorting algorithms based upon quicksort, a quadratically-bounded algorithm, which fall back to some explicitly-optimal method like heapsort or merge-sort. Rust uses these algorithms in its standard library. |
There was a problem hiding this comment.
The argument in this section could be clarified a bit.
You seem to be saying that the idea of an unbounded loop where the model picks some action (writing code, running a tool), receives the result, and uses it to inform future actions, is inherently "brute force" and a "feedback loop" and bad.
But that can't be true, because that also describes how humans work.
Of course, LLMs are considerably worse at it than humans. And I can think of ways in which those terms apply uniquely to LLMs. "Brute force" applies uniquely to LLMs in that one strategy LLMs use is to make up for a lack of intelligence with sheer persistence. "Feedback loop" applies uniquely to LLMs in that LLMs are much more vulnerable than humans to getting stuck with a bad assumption and spending a long time barking up the wrong tree. Or perhaps more generally in that LLM performance tends to fall off drastically past certain complexity levels, which could be analogized to an audio feedback loop amplifying noise.
However, you don't really explain that. As written, the argument could apply just as well to humans. And I could respond: sure, it would be nice if there were a way to solve arbitrary programming problems with a bounded amount of work, but that just doesn't exist.
There was a problem hiding this comment.
Hmm, I feel like I did technically cover this point, but I do think you have a point here. I'm mostly replying to point out that I do want to consider what you've written, but am going to think about it a bit to figure out how to properly articulate that.
There was a problem hiding this comment.
Also for some additional context, this piece is probably the closest to being directly inspired by this massive essay I've been writing for years going more in-depth on this. Like, there, I spend a lot more time explaining the specifics of complexity theory, incompleteness proofs, and other things to make a point, but like, I both haven't written that fully and am not going to write that fully here. But I feel like there's some nugget of information that could be gleaned without going past the surface level.
| 2. If the LLM usage is *slop*, it is considered spam and moderated accordingly. Generally, this means submitting changes made by LLMs with minimal human intervention. | ||
| 3. *Nontrivial* LLM usage must be *disclosed* in ideally as detailed as a manner as possible. This may necessitate additional tooling to notify new contributors about the policy and explain how disclosure works. | ||
| 4. If a contributor does not fully understand the code they submit, their contribution may be rejected for that reason alone. Note that such usage is not always considered *slop*, and is considered separately. (For example, they may understand a large portion, but not all of it, which shows that they still put in a lot of effort.) | ||
| 5. If a user is found to be repeatedly lying about LLM usage (using LLMs without disclosing that usage), this is a COC violation that will be moderated accordingly. |
There was a problem hiding this comment.
This all looks great! "Honesty over purity" is a more friendly way of saying that non-disclosed nontrivial LLM uses is bad faith.
I'd suggest rewording 5 so that a single offence can trigger moderation. You could just remove the word "repeatedly", but then the sentence maybe stronger than you'd like, since some lack of clarity exists on the first offence. "repeatedly or dangerously" maybe? Or some rephrasing?
Anyway I do think you want to leave yourself room to act strongly on a first offence. You maybe confident 6 and 7 cover those cases, but I think you're not doing the vibers any favours by giving the impression of a free pass here.
You should probably change "lying" to "hiding" too, because someone might think that saying nothing is not lying. There can be explicit questions in PR templates etc that make "hiding" more clearly be "lying", but this policy seems more general, so "hiding" seems clearer here.
There was a problem hiding this comment.
So, not 100% sure if you've read the whole thing, since I do go a bit more in depth in the start of the motivation to point out what "honesty over purity" means; nontrivial use is allowed (less purity) with disclosure (honesty). The point is that this is a compromise over a more strict policy where honesty is seen more important than preventing all (nontrivial) LLM usage.
With regard to enforcement, 5 already leaves enforcement largely up to the moderation team, and I would rather that they have broad authority to determine whether something is intentional or not. The main method of determining usage where things might be ambiguous is by asking people directly, and then you're kind of forced to say something about it; at that point, if you lie, that's just lying, not hiding.
Right now there are already a lot of moderation details I kept out of the policy for simplicity: for example, the moderation team tends to prefer bans for slop PRs because bans can be rescinded; the only change is they have to talk to the mods about it before future contributions. For stuff like what to do when lying, I'd rather the mods who are already pretty lenient (intentionally) make the determination whether someone was being malicious or not.
Plus, COC enforcement also is inherently asymmetrical because it's more heavily weighted against team members (who should know better), so we kind of get the desired enforcement built in: team members who lie are treated much more seriously and might not get a second chance, whereas for ordinary contributors who do it, we're more likely to just reject PRs unless it becomes a problem.
Co-authored-by: Arhan Chaudhary <arhan.ch@gmail.com>
8 participants
Preface
A lot of discussion has occurred in private about the topic of LLM policy, and while some of that context has been included in the prior art, most of it is intentionally omitted here.
To keep things focused on policy, there are two broad categories of comments we'd like to request you avoid:
In general, defer to the code of conduct.
This RFC is long, and most of it is the sections surrounding and justifying the actual policy, rather than the policy itself. You are both welcome to and encouraged to skip around using the outline feature on GitHub. (In the rendered view, this is the bulleted list button on the top-right of the file view.)
While I have received a lot of feedback that the RFC is perhaps too long, a lot of the sections that might be considered for removal constitute important arguments that have been mentioned. In terms of discussion on length alone, I would appreciate that these arguments be directed on potentially simplifying the text of the policy itself, rather than just removing sections of motivation. Similarly, if you feel a particular argument could be expanded or added, feel free to mention that as well.
In general, I know that this RFC is going to be exhausting, and the discussion before it has been too. Removing details is not really going to help, although I'd be happy to accept any feedback on revising the text of these sections to be shorter without losing any meaning.
Important
Since RFCs involve many conversations at once that can be difficult to follow, please use review comment threads on the text changes instead of direct comments on the RFC.
If you don't have a particular section of the RFC to comment on, you can click on the "Comment on this file" button on the top-right corner of the diff, to the right of the "Viewed" checkbox. This will create a separate thread even if others have commented on the file too.
Existing policies
Right now, there are a few concurrent policy decisions:
rust-lang/rust: Add an LLM policy forrust-lang/rustrust-forge#1040This RFC intends to supersede the two existing policy RFCs, but it intentionally does not supersede the current policy scoped to
rust-lang/rust; that policy is free to be merged before this one, as its original goal was to put a policy in place before an RFC was accepted. In fact, even if after an RFC is accepted, that one can still be merged, since updating the policies everywhere takes time, and getting a policy out immediately is still a net benefit.Once an RFC is accepted, things can be adjusted for consistency.
Summary
This RFC proposes a strict policy regarding generative Artificial Intelligence (AI) models, specifically Large Language Models (LLMs), and their use within the rust-lang organization.
It proposes an "honesty over purity" policy where maintainers are given broad authority to decide what amount of LLM-generated code is acceptable, while avoiding repercussions for those who do use LLMs and are honest about it. This can be summarized in the following checklist with terms that will be defined throughout the RFC:
In terms of additional tooling for disclosure, this RFC encourages the creation of a bot that automatically replies to contributions from new users informing them of the LLM policy and what constitutes sufficient disclosure. As mentioned, in general, going into as much detail as possible (e.g. prompts used, etc.) is preferred, but not always required. The RFC leaves the exact details of such implementation unspecified and up for revision later.
Rendered