Skip to content

[Bug]: Glob/Grep permission rules never cover the directory they are suggested for #2826

Description

@lihongyuan99

Prerequisites

  • I have searched the existing issues and discussions, and this is not a duplicate.
  • This is a bug, not a usage question. (For questions, please use Discussions instead.)

Background / Description

Glob and Grep offer a permission rule for the directory a search runs in, and PermissionEngine decides allow/deny/ask through each tool's match_rule. For these two directory-rooted tools the offered rule never matches the search it came from, so approving it does not prevent the next identical prompt.

Two separate defects combine here:

  1. generate_suggestions emits only <root>/** (_glob.py:213, _grep.py:247). match_rule compares that pattern against the search root itself, and fnmatch("<root>", "<root>/**") is False — the pattern requires at least one character after <root>/. So the rule shown in the approval prompt (tui/_chat.py:232) does not cover the very invocation that produced it, at any depth.
  2. Glob.match_rule reads the raw argument only: path = tool_input.get("path", "") (_glob.py:178). When path is omitted, Glob.call searches the working directory — its documented default — but the matcher sees an empty string, so no directory rule can match. Grep.match_rule already falls back to await self._backend.getcwd() (_grep.py:219-221), and tests/builtin_grep_test.py::GrepToolTest::test_match_rule_defaults_to_cwd pins that fallback as the intended contract. Glob is the outlier.

The file tools do not have this problem: Read, Write and Edit suggest <parent dir>/**, and their file_path is a file inside that directory, so their suggestion matches the invocation it came from. Glob/Grep input is the directory, so the file-tool shape misses.

The same mismatch weakens a hand-written DENY rule: <dir>/** blocks a search rooted at a subdirectory of <dir>, yet allows a search rooted at <dir> itself, which recursively discloses exactly the entries the rule was meant to protect (last two lines below).

I plan to fix both halves above in a follow-up PR. One design point I would like guidance on, and which I would keep out of that PR: should a rule for <dir>/** also cover a search rooted above <dir> (e.g. path=<parent> with a deny rule on <parent>/secret/**)? That is a containment question rather than a string-matching one, so I would rather not decide it unilaterally.

Error Messages

No exception is raised; the bug is a wrong permission decision. Each line below feeds one invocation to generate_suggestions and then matches the result against that same invocation:

BROKEN Glob  [('pattern', '*.py')]
          suggestion=['<cwd>/**'] matches=[False]
BROKEN Glob  [('path', '<tmp>/pkg'), ('pattern', '*.py')]
          suggestion=['<tmp>/pkg/**'] matches=[False]
BROKEN Grep  [('pattern', 'x')]
          suggestion=['<cwd>/**'] matches=[False]
BROKEN Grep  [('path', '<tmp>/pkg'), ('pattern', 'x')]
          suggestion=['<tmp>/pkg/**'] matches=[False]
OK     Read  [('file_path', '<tmp>/pkg/main.py')]
          suggestion=['<tmp>/pkg/**'] matches=[True]
OK     Write [('file_path', '<tmp>/pkg/main.py')]
          suggestion=['<tmp>/pkg/**'] matches=[True]
OK     Edit  [('file_path', '<tmp>/pkg/main.py')]
          suggestion=['<tmp>/pkg/**'] matches=[True]

and this is the DENY half, with a <tmp>/secret/token.txt behind a rule that names its directory:

Glob  deny='<tmp>/secret/**' path='<tmp>/secret' -> allow, returned=['<tmp>/secret/token.txt']
Glob  deny='<tmp>/secret/**' path='<tmp>/secret/nested' -> deny
Grep  deny='<tmp>/secret/**' path='<tmp>/secret' -> allow, returned=['<tmp>/secret/token.txt']
Grep  deny='<tmp>/secret/**' path='<tmp>/secret/nested' -> deny

Steps to Reproduce

No network access, no API keys and no container are required.

  1. Code:
# -*- coding: utf-8 -*-
"""Reproduce the Glob/Grep permission-rule mismatch."""
import asyncio
import os
import tempfile

from agentscope.permission import (
    PermissionBehavior,
    PermissionEngine,
    PermissionMode,
    PermissionRule,
)
from agentscope.permission._context import PermissionContext
from agentscope.tool import Glob, Grep

ROOT = tempfile.mkdtemp()
SECRET = os.path.join(ROOT, "secret")
os.makedirs(os.path.join(SECRET, "nested"))
with open(os.path.join(SECRET, "token.txt"), "w", encoding="utf-8") as f:
    f.write("hunter2\n")


def short(value):
    """Redact the temp root so the output is stable across runs."""
    text = str(value)
    return text.replace(ROOT, "<tmp>").replace(os.getcwd(), "<cwd>")


async def round_trip(tool, tool_input):
    """Do the suggested rules cover the call they were derived from?"""
    rules = await tool.generate_suggestions(tool_input)
    contents = [short(_.rule_content) for _ in rules]
    hits = [await tool.match_rule(_.rule_content, tool_input) for _ in rules]
    print(f"{tool.name:<5} {short(sorted(tool_input.items()))}")
    print(f"          suggestion={contents} matches={hits}")


async def denied(tool, rule_content, tool_input):
    """What does a DENY rule decide, and what still gets returned?"""
    engine = PermissionEngine(PermissionContext(mode=PermissionMode.DEFAULT))
    engine.add_rule(
        PermissionRule(
            tool_name=tool.name,
            rule_content=rule_content,
            behavior=PermissionBehavior.DENY,
            source="userSettings",
        ),
    )
    decision = await engine.check_permission(tool, dict(tool_input))
    head = (
        f"{tool.name:<5} deny={short(rule_content)} "
        f"path={short(tool_input.get('path'))}"
    )
    if decision.behavior is PermissionBehavior.DENY:
        print(f"{head} -> deny")
        return
    chunk = await tool.call(**tool_input)
    print(f"{head} -> allow, returned={[short(_.text) for _ in chunk.content]}")


async def main():
    nested = os.path.join(SECRET, "nested")
    for tool, base in (
        (Glob(), {"pattern": "*.txt"}),
        (Grep(), {"pattern": "hunter"}),
    ):
        # 1: no path, so the search runs in the working directory
        await round_trip(tool, dict(base))
        # 2: the suggestion covers children only, never its own search root
        await round_trip(tool, {**base, "path": SECRET})
        await round_trip(tool, {**base, "path": nested})
        # 3: so a DENY rule for a directory does not block that directory,
        # while it does block a search rooted below it
        await denied(tool, f"{SECRET}/**", {**base, "path": SECRET})
        await denied(tool, f"{SECRET}/**", {**base, "path": nested})


asyncio.run(main())
  1. Run: python repro.py on main at 2c885b5
  2. See: every Glob/Grep round trip reports matches=[False], and a search rooted at the denied directory itself is allowed while the same search rooted one level lower is denied.

Environment

  • AgentScope Version: 2.0.8 (main @ 2c885b5)
  • Python Version: 3.12.12
  • OS: macOS 15.6 (arm64) — the matching is string-based, so this reproduces identically on Linux and Windows

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

triage/confirmedVerified: the reported defect exists

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions