Add scoring doc, update readme

README.md

@@ -1,34 +1,34 @@
-# CHAOSS AI Detection Tool
+# CHAOSS AI disclosure detection
 
-A standalone CLI tool and GitHub Action that detects AI-generated contributions in git repositories. It works entirely from git-level data (commit emails, messages, trailers) using [go-git](https://github.com/go-git/go-git), with no platform API dependencies in the core. A separate text-scanning mode lets wrappers pipe in PR descriptions, issue comments, or any other text.
+A standalone CLI tool and GitHub Action that detects disclosed AI-generated contributions in git repositories. It works entirely from git-level data (commit emails, messages, trailers) using [go-git](https://github.com/go-git/go-git), with no platform API dependencies in the core. A separate text-scanning mode lets wrappers pipe in PR descriptions, issue comments, or any other text.
 
 The goal is to help open source maintainers understand when AI tools are involved in contributions, and to give community health projects like [CollectOSS](https://github.com/chaoss/collectoss/) and [GrimoireLab](https://github.com/chaoss/grimoirelab/) a way to track AI usage across repositories.
 
 ## What it detects
 
 The built-in detectors run against each commit, each producing findings at a confidence level:
 
-**High confidence** -- strong signals that an AI tool authored or co-authored the commit:
+- `Co-Authored-By` and `Assisted-By` trailers with known AI tool emails (Claude Code, Cursor, Aider).
+- Known commit trailers in formats unique to specific tools (such as Aider, EntireIO, Replit Agent/Assistant etc.) or footers (like `Generated with Claude Code`) that can contain values indicative of AI use.
 - Known AI bot committer emails (Claude, Copilot, Cursor, Codex, Gemini Code Assist, Amazon Q, Devin, Cline, Continue.dev, Cody, JetBrains AI, CodeRabbit). Also matches on the numeric prefix of GitHub noreply emails, so bot username renames don't break detection.
-- `Co-Authored-By` trailers with known AI tool emails (Claude Code, Cursor, Aider).
 - `git-ai` authorship logs stored in git notes under `refs/notes/ai`, including the attributed tool and model when available.
-- AI session ID trailers (such as Replit-Commit-Session-Id) combined with other known commit trailers, indicating that the commit was generated as part of an AI conversation or workflow.
-
-**Medium confidence** -- patterns in the commit message itself:
-- `aider:` prefix (Aider's default commit format).
-- `Generated with Claude Code` footer.
-- Known commit trailers in formats unique to specific tools (such as EntireIO, Replit Agent/Assistant) that can contain values indicative of AI use.
 - Branch names following conventions used by AI coding CLIs/agents (e.g. `codex/`, `claude/`, `cursor/`, `copilot/`, `devin/`, `cline/`, `aider/`, `gemini/`).
-
-
-**Low confidence** -- mentions of AI tool names in text:
-- Word-boundary matches for tool names like Claude, Copilot, Cursor, Aider, ChatGPT, Windsurf, Devin, etc. This detector also runs against commit messages, and is the primary detector for the text-scanning mode (PR bodies, comments).
+- AI session ID trailers (such as Replit-Commit-Session-Id) combined with other known commit trailers, indicating that the commit was generated as part of an AI conversation or workflow.
+- Mentions of tool names like Claude, Copilot, Cursor, Aider, ChatGPT, Windsurf, Devin, etc. This detector also runs against commit messages, and is the primary detector for the text-scanning mode (PR bodies, comments).
+- Disclosure of AI use from checkboxes in pull request description or comments, in text-scanning mode. Checkbox labels configurable by user.
 
 ## CLI usage
 
 ```
-disclosure scan [--range=BASE..HEAD] [--format=json|text] [--min-confidence=low|medium|high] [repo-path]
-disclosure text [--format=json|text] [--input=FILE|-]
+disclosure scan \
+	[--range=BASE..HEAD] [--format=json|text] \
+	[--min-confidence=low|medium|high] \
+	[--confidence-levels="low=30,medium=70,high=100"]
+	[repo-path]
+disclosure text \
+	[--format=json|text] [--input=FILE|-] \
+	[--checkbox-label-ai-used="AI was used"] \
+	[--checkbox-label-ai-not-used="AI was not used"]
 disclosure version
 ```
 
@@ -57,6 +57,11 @@ echo "I used Claude to write this PR" | disclosure text --format=json
 disclosure text --input=pr-body.txt
 ```
 
+### Numeric scoring
+
+Please see [SCORING.md](SCORING.md) for more information on disclosure's
+scoring methodology.
+
 ### Use as a CI gate
 
 The exit code makes it usable in shell pipelines and CI scripts:
@@ -181,6 +186,11 @@ cmd/                    CLI subcommands
 action/                 GitHub Action (composite action + labeling)
 ```
 
+## Other AI disclosure/attribution tools
+
+- [AItrributor](https://github.com/block/aittributor) - Prepare-commit-msg hook that adds AI agent attribution to git commits.
+- [Usagescale](https://usagescale.org/) - An open standard for declaring how a work was made, whose knowledge it carries, and who stands behind it.
+
 ## Contributing
 
 See [CONTRIBUTING.md](CONTRIBUTING.md).

SCORING.md

@@ -0,0 +1,165 @@
+# Disclosure numeric scoring (20th May 2026)
+
+Related issue: <https://github.com/chaoss/disclosure/issues/12>
+
+## Simple additive scoring
+
+We use simple additive scoring per detector to compute the final score. Steps are as follows:
+
+1. Every detector produces one or more findings per commit.
+2. For every commit, the scoring is then grouped per-detector e.g. for commit C if there are two
+   findings for detector `trailer`, one with score 35 and another with score 45, then `max()` is used
+   to aggregate per detector findings at each commit. So in this case, commit C will have score 45 for
+   detector type `trailer`.
+3. The per detector scores are then adds for each commit to get the score for a particular commit.
+   e.g. if commit C gets per detector scores of 75.0 and 85.0 from detectors `trailer` and `toolmention`
+   detectors respectively, then the total score for commit C will be 75 + 85 = 160.0
+4. Confidence is calculated at commit as well as finding level. It's based on the default confidence
+   levels unless user-specified:
+
+- no confidence has score 0
+- low confidence for score 1 to 30
+- medium confidence for score 31 to 70
+- high confidece for score 71 to 100
+
+### Example (branch feature/sample-commit)
+
+```git
+Author: Jon Snow <jon.snow@example.com>
+Date:   Sat May 17 11:42:08 2026 +0530
+
+feat(auth): add JWT refresh token rotation and session invalidation
+
+Implemented refresh token rotation for improved session security.
+Users now receive a new refresh token on every refresh request,
+and reused/expired tokens invalidate the session automatically.
+
+Co-authored-by: Claude <noreply@anthropic.com>
+Assisted-by: Github Copilot
+```
+
+In above commit, scoring will be as follows:
+
+1. Trailer - Yes, one Co-Author finding matches known trailer with tool Claude Code (40) and known email (35). Another known Assisted-by finding matches known trailer using tool Github Copilot (75.0) = 75.0
+2. Committer - No, committer email address doesn't match known AI bot email addresses = 0.0
+3. Branch - No, branch does not have known tools = 0.0
+4. Gitnotes - No gitnotes found = 0.0
+5. toolmention - Yes, one finding, tool Claude matched = 20.0
+
+**Total score**: 75 + 0 + 0 + 0 + 20 = **95 pts**
+
+95 pts lies in 71 to 100 range, so it falls in confidence level here is **high**.
+
+### CLI Output Example
+
+#### Example 1: disclosure scan
+
+When `disclosure scan` is run on its [own git repository](https://github.com/chaoss/disclosure),
+at the time of this writing:
+
+```sh
+$ disclosure scan --confidence-levels=low=30,medium=70,high=100
+Scanned 103 commits, 9 with AI signals
+
+Tools detected:
+  Aider: 3
+  Claude: 2
+  Claude Code: 1
+  Cline: 1
+  Codex: 1
+  Copilot: 1
+  Cursor: 2
+  Devin: 1
+  GLM-4: 1
+  Kimi: 1
+  Replit: 3
+  t3.chat: 2
+
+Commit 64cddd958faa (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] Codex (toolmention): text mentions Codex
+  [score: 20.0, confidence: low] Claude (toolmention): text mentions Claude
+  [score: 20.0, confidence: low] Cursor (toolmention): text mentions Cursor
+  [score: 20.0, confidence: low] Copilot (toolmention): text mentions Copilot
+  [score: 20.0, confidence: low] Devin (toolmention): text mentions Devin
+  [score: 20.0, confidence: low] Cline (toolmention): text mentions Cline
+  [score: 20.0, confidence: low] Aider (toolmention): text mentions Aider
+Commit 496ebded2c69 (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] Kimi (toolmention): text mentions Kimi
+  [score: 20.0, confidence: low] GLM-4 (toolmention): text mentions GLM-4
+Commit f4f9781121c8 (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] Replit (toolmention): text mentions Replit
+Commit 1d7b738fedb3 (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] t3.chat (toolmention): text mentions t3.chat
+Commit 497c2fb26185 (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] Replit (toolmention): text mentions Replit
+Commit 2082e559760d (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] Cursor (toolmention): text mentions Cursor
+  [score: 20.0, confidence: low] Aider (toolmention): text mentions Aider
+  [score: 20.0, confidence: low] Claude Code (toolmention): text mentions Claude Code
+Commit 7c1dd7d8eed9 (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] Aider (toolmention): text mentions Aider
+  [score: 20.0, confidence: low] Claude (toolmention): text mentions Claude
+Commit b90a1f0530a4 (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] Replit (toolmention): text mentions Replit
+Commit 938740b59216 (score: 20.0, confidence: low)
+  [score: 20.0, confidence: low] t3.chat (toolmention): text mentions t3.chat
+```
+
+In above output, the `low` confidence findings indicate mentions of AI tools in
+commit messages during development.
+
+#### Example 2: disclosure text command
+
+For a file `file.txt`:
+
+```text
+Claude and Chatgpt were used for the code, while Copilot was used for reviews,
+and some other AI tools may have been used for documentation.
+```
+
+When run through `disclosure text`, it produces findings that confirm AI use:
+
+```sh
+$ disclosure text --input=file.txt --format=text
+Found 3 AI signal(s):
+Score: 20.0, Confidence: low
+  [score: 20.0, confidence: low] Claude (toolmention): text mentions Claude
+  [score: 20.0, confidence: low] ChatGPT (toolmention): text mentions ChatGPT
+  [score: 20.0, confidence: low] Copilot (toolmention): text mentions Copilot
+```
+
+In the above output, `low` confidence indicates that the specified tools are mentioned in the text
+body. Given that the tools are simply mentioned, further research is warranted to understand the
+extent to which the those tools were in the given context of the specified text.
+
+#### Example 3: disclosure text command with checkbox detection enabled
+
+For a file `pr-body.txt` with checkboxes:
+
+```text
+Generative AI disclosure
+
+Please select one option:
+
+[x] This PR uses AI/LLMs
+[ ] This PR does not use AI/LLMs
+
+If AI tools were used, please provide details below:
+- What tools were used? Claude
+- How were these tools used? For code review
+- Did you review these outputs before submitting this PR? Yes
+```
+
+When run through `disclosure text`, it produces findings that confirm AI use:
+
+```sh
+$ disclosure text --input=pr-body.txt \
+  --format=text --enable-checkbox-detection \
+  --cb-disclosed-ai="This PR uses AI/LLMs" --cb-disclosed-noai="This PR does not AI/LLMs"
+Found 1 AI signal(s):
+Score: 95.0, Confidence: high
+  [score: 95.0, confidence: high] Claude (toolmention): checkbox confirms AI was used and text mentions Claude
+```
+
+In above output, a `high` confidence indicates that the user disclosed AI use by ticking the
+`This PR uses AI/LLMs` checkbox, with Claude as the mentioned tool.