Skip to content

Commit 96cbbc7

Browse files
llama-atmosphere-agent: general-purpose default prompt, run_command for any command
The default system prompt called the agent a "careful coding agent" and described run_command as a way "to build, test or inspect the project"; the tool description said "build, test, grep or list files". Qwen3-4B read that as a prohibition and refused "list the docker images" ("my tools are only for files") although the tool was registered and docker was on PATH -- in a fresh single-turn run too, so it was the prompt, not the chat history. - LocalAgent.systemPrompt: a general-purpose assistant on the user's machine. With --allow-shell it states that run_command executes any command line through the named shell, starting in the workspace but not limited to it, and that the model should run a command instead of explaining one; without it, the prompt says commands are unavailable and names the flag. Answers in the user's language. - ShellTool: description widened to "any command line on this machine through <shell>"; isWindows()/shellName() shared with the prompt. - READMEs: general-purpose wording, an "everything at once" example with --allow-shell --system, a "The system prompt" section, generic paths only. - CLAUDE.md: why the prompt must not be narrowed back; ShellToolTest is POSIX-only. Verified with Qwen3-4B-Instruct-2507 on Vulkan: the same question now calls run_command {command=docker images} with the default prompt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E2h8gXyyE5UeimkL9vQv9G
1 parent 49ca3bf commit 96cbbc7

7 files changed

Lines changed: 139 additions & 28 deletions

File tree

‎CLAUDE.md‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2140,7 +2140,7 @@ releases as a signed Central Portal bundle upload (staging repo → zip → Publ
21402140

21412141
## Local coding agent with Atmosphere (`llama-atmosphere-agent/`, standalone)
21422142

2143-
A **copy-and-run terminal coding agent** (Claude Code / OpenCode reduced to the essentials, offline)
2143+
A **copy-and-run general-purpose terminal agent** (Claude Code / OpenCode reduced to the essentials, offline)
21442144
that pairs [Atmosphere](https://github.com/Atmosphere/atmosphere)'s built-in OpenAI-compatible
21452145
agent runtime with this project's `OpenAiCompatServer`. Like `android-llmservice/` it is a
21462146
**standalone Maven project, NOT a reactor module and NOT published** — it is an application, and it
@@ -2192,7 +2192,7 @@ the moment anything runs on the module path.
21922192
lines: `AiConfig.configure` → `BuiltInAgentRuntime` → `AgentExecutionContext` + `ToolLoopPolicies`),
21932193
`ConsoleSession` (streams to stdout, prints `⚙ tool {args}` / `↳ result`, supplies the
21942194
`WorkspaceAgentFileSystem` via `injectables()`), `ShellTool` (opt-in `run_command`, `sh -c` /
2195-
`cmd /c` in the workspace, timeout kills the process tree, output tail-truncated), `LocalAgent`
2195+
`cmd /c` starting in the workspace, timeout kills the process tree, output tail-truncated), `LocalAgent`
21962196
(`--base-url` = external server, `--model` = in-process `LlamaModel` + loopback `OpenAiCompatServer`
21972197
with `enableJinja()` and `setLogVerbosity(2)` by default — llama.cpp logs to **stderr**, the console the
21982198
streamed answer shares, so the per-request `slot …` INFO lines would interleave with it; `--log-verbosity <n>`
@@ -2205,6 +2205,19 @@ a `jvm.config` takes no comments, so REUSE can only read its metadata from that
22052205
`REUSE Compliance Check` job fails on `main` — which is how it was found, the PR run having been cancelled.
22062206
Spotless (palantir) is configured in its own pom; the model-free CI job runs `spotless:check`.
22072207

2208+
**The default system prompt is general-purpose on purpose — do not narrow it back.** `LocalAgent.systemPrompt`
2209+
and the `ShellTool` description describe `run_command` as running *any* command line through the named
2210+
shell (`ShellTool.shellName()`), not limited to the workspace, and tell the model to run a command rather
2211+
than explain one. The earlier wording ("careful *coding agent*", `run_command` "to build, test or inspect
2212+
the project" / "build, test, grep or list files") made Qwen3-4B refuse "list the docker images" — "my
2213+
tools are only for files" — with the tool registered and `docker` on `PATH`; a fresh single-turn run
2214+
refused too, so it was the prompt, not the chat history. Without `--allow-shell` the prompt says commands
2215+
are unavailable and names the flag, so the model does not invent its own limitation. Pinned by
2216+
`AgentOptionsTest.defaultSystemPromptIsGeneralPurposeAndAllowsAnyCommandWithTheShell` and
2217+
`shellToolDescriptionDoesNotNarrowItToTheProject`. `ShellToolTest` uses POSIX commands (`ls`, `sleep`,
2218+
`exit 3`), so it passes only on Linux/macOS — CI runs it on Linux; on Windows run the suite with
2219+
`-Dtest=!ShellToolTest`.
2220+
22082221
**Version bump note.** The pom's `llama.version` property is the **release** version, not the
22092222
reactor's `-SNAPSHOT` (CI always overrides it, so a not-yet-published default never breaks CI).
22102223
`versions:set` does not touch this standalone pom, so at release time bump it by hand together with

‎README.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1021,8 +1021,9 @@ not yet forwarded).
10211021

10221022
### Local coding agent with Atmosphere (`llama-atmosphere-agent/`)
10231023

1024-
A copy-and-run **terminal coding agent on the JVM** — Claude Code / OpenCode reduced to the
1025-
essentials, fully offline — built from [Atmosphere](https://github.com/Atmosphere/atmosphere)'s
1024+
A copy-and-run **general-purpose terminal agent on the JVM** — Claude Code / OpenCode reduced to the
1025+
essentials, fully offline; it edits files and, with `--allow-shell`, runs any command on your machine
1026+
(`docker`, `git`, build tools) — built from [Atmosphere](https://github.com/Atmosphere/atmosphere)'s
10261027
built-in OpenAI-compatible agent runtime (streaming, tool loop, workspace file tools) driven
10271028
**headless** against this project's OpenAI-compatible server. It is a standalone Maven project (not a
10281029
reactor module, not published); you copy the folder and run it. With java-llama.cpp already running
@@ -1043,6 +1044,10 @@ mvn -q compile exec:java \
10431044
# or without a separate server: load the GGUF in-process
10441045
mvn -q compile exec:java \
10451046
-Dexec.args="--model /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --ngl 99 --workspace /path/to/project"
1047+
1048+
# everything at once: shell access plus your own system prompt (replaces the built-in one)
1049+
mvn -q compile exec:java \
1050+
-Dexec.args="--model /models/Qwen3-4B-Instruct-2507-Q4_K_M.gguf --ngl 99 --ctx-size 16384 --workspace /path/to/project --allow-shell --system 'You are a local assistant on this machine with full shell access. run_command executes any command line, including docker, git and build tools. When asked about the system, run a command instead of explaining it. Answer in the language of the user.'"
10461051
```
10471052

10481053
The full streaming tool-calling loop (tools → `delta.tool_calls` → Java tool → `role:"tool"` result →

‎llama-atmosphere-agent/README.md‎

Lines changed: 38 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,12 @@ SPDX-FileCopyrightText: 2026 Bernard Ladenthin <bernard.ladenthin@gmail.com>
44
SPDX-License-Identifier: MIT
55
-->
66

7-
# llama-atmosphere-agent — a local JVM coding agent on java-llama.cpp
7+
# llama-atmosphere-agent — a local, general-purpose JVM agent on java-llama.cpp
88

9-
A minimal, copy-and-run **terminal coding agent** (think Claude Code / OpenCode, reduced to the
10-
essentials) that runs entirely on the JVM and entirely offline:
9+
A minimal, copy-and-run **general-purpose terminal agent** (think Claude Code / OpenCode, reduced to
10+
the essentials): it reads and edits files, and with `--allow-shell` it runs any command on your
11+
machine — `docker`, `git`, build tools, system information. It runs entirely on the JVM and entirely
12+
offline:
1113

1214
- **Model:** any GGUF served by java-llama.cpp's OpenAI-compatible HTTP surface — either a server
1315
you start yourself, or the GGUF loaded **in this process**.
@@ -16,7 +18,8 @@ essentials) that runs entirely on the JVM and entirely offline:
1618
workspace-confined file tools (`ls`, `read_file`, `write_file`, `edit_file`, `glob`, `grep`,
1719
`delete`, `rename`). Driven **headless** — no Spring Boot, no servlet container, no `@Agent`
1820
scanning — through `BuiltInAgentRuntime`.
19-
- **Shell:** an opt-in `run_command` tool (`--allow-shell`) so the model can build and test.
21+
- **Shell:** an opt-in `run_command` tool (`--allow-shell`) that runs any command line through the
22+
system shell (`cmd.exe` on Windows, `sh` elsewhere).
2023

2124
This folder is a **standalone Maven project**, deliberately *not* a reactor module and *not*
2225
published: CI builds and tests it against the core of the same checkout; you copy the folder and
@@ -66,6 +69,18 @@ mvn -q compile exec:java \
6669
-Dexec.args="--model /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --ngl 99 --workspace /path/to/project"
6770
```
6871

72+
**Everything at once — shell access and your own system prompt:**
73+
74+
```bash
75+
mvn -q compile exec:java \
76+
-Dexec.args="--model /models/Qwen3-4B-Instruct-2507-Q4_K_M.gguf --ngl 99 --ctx-size 16384 --workspace /path/to/project --allow-shell --system 'You are a local assistant on this machine with full shell access. run_command executes any command line, including docker, git and build tools. When asked about the system, run a command instead of explaining it. Read a file before you edit it. Answer in the language of the user.'"
77+
```
78+
79+
Then ask, for example, *"which docker images are available?"* or *"build the project and fix the
80+
first compiler error"*. On Windows PowerShell, quote the whole argument instead:
81+
`"-Dexec.args=--model C:\models\… --allow-shell --system '…'"`. Inside `--system '…'` avoid the
82+
apostrophe (write *the user* rather than *user's*): the value is already single-quoted.
83+
6984
GPU natives: pick the core classifier, e.g. `-Dllama.classifier=cuda13-linux-x86-64` or
7085
`vulkan-windows-x86-64` (the vendor runtime must be installed — see the root README's classifier
7186
table). Without it the default CPU jar (incl. macOS Metal) is used. In mode A the classifier is
@@ -79,8 +94,8 @@ irrelevant: inference stays in the running server, the agent's JVM loads no mode
7994
| `--model <file.gguf>` | load this GGUF in-process instead | — |
8095
| `--ngl <n>` / `--ctx-size <n>` | GPU layers / context size for `--model` | `0` / `8192` |
8196
| `--log-verbosity <n>` / `--verbose` | llama.cpp log threshold for `--model` (1 errors, 2 warnings, 3 info, 4 trace, 5 debug) / log everything | `2` / off |
82-
| `--workspace <dir>` | directory the file tools (and `run_command`) are confined to | cwd |
83-
| `--allow-shell` | register `run_command` | off |
97+
| `--workspace <dir>` | directory the file tools are confined to, and where `run_command` starts | cwd |
98+
| `--allow-shell` | register `run_command`: any command line, starting in the workspace | off |
8499
| `--system <text>` | replace the default system prompt | built-in |
85100
| `--prompt <text>`, `-p` | one turn, then exit | interactive |
86101
| `--temperature <t>` / `--max-tokens <n>` | sampling / per-call budget | `0.2` / `2048` |
@@ -103,9 +118,25 @@ code page it saw at startup, so umlauts and emoji in the answer would turn into
103118
project's `.mvn/jvm.config` pins `-Dstdout.encoding=UTF-8 -Dstderr.encoding=UTF-8` for the `mvn`
104119
JVM so both sides agree.
105120

121+
### The system prompt
122+
123+
Without `--system` the agent uses a built-in **general-purpose** prompt: it names the file tools and
124+
the workspace they work on, and — only with `--allow-shell` — states that `run_command` runs *any*
125+
command line on this machine (the shell is named, so the model writes the right syntax) and that the
126+
model should run a command rather than explain one. Without `--allow-shell` it tells the model it
127+
cannot run commands and to suggest the flag, so the model does not invent a limitation of its own.
128+
129+
This wording matters more than it looks: an earlier default called the agent a *coding agent* and
130+
described `run_command` as a way to *"build, test or inspect the project"*, and Qwen3-4B then refused
131+
*"list the docker images"* ("my tools are only for files") although the tool was registered and the
132+
command worked. `--system <text>` replaces the default **completely** — include whatever the model
133+
still needs to know (the workspace, the shell, your language) in your own text.
134+
106135
Pick a **tool-capable instruct model** (Qwen2.5/Qwen3-Instruct, Llama-3.x-Instruct, Mistral,
107136
Hermes, …). Quality of the loop is the model's: a 1.5B model calls one tool and reads its result, a
108-
7B–32B model does multi-step edit/build/test work.
137+
7B–32B model does multi-step edit/build/test work. Qwen3-4B-Instruct-2507 is a good fast default (fits
138+
an 8 GB GPU with a 16k context); Qwen2.5-Coder-7B, in contrast, wrote the call as a JSON code block
139+
into its answer instead of calling the tool.
109140

110141
## What is verified, and where
111142

‎llama-atmosphere-agent/src/main/java/net/ladenthin/llama/atmosphere/AgentOptions.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@ public static String usage() {
167167
"",
168168
"Agent:",
169169
" --workspace <dir> directory the file tools are confined to (default: cwd)",
170-
" --allow-shell add the run_command tool (runs shell commands in the workspace)",
170+
" --allow-shell add the run_command tool (runs any command line, starting in the workspace)",
171171
" --system <text> replace the default system prompt",
172172
" --prompt <text>, -p run one turn and exit (default: interactive; /exit to quit)",
173173
" --temperature <t> sampling temperature (default " + DEFAULT_TEMPERATURE + ")",

‎llama-atmosphere-agent/src/main/java/net/ladenthin/llama/atmosphere/LocalAgent.java‎

Lines changed: 21 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
import org.jspecify.annotations.Nullable;
2424

2525
/**
26-
* A local, terminal coding agent in the spirit of Claude Code / OpenCode, built from two parts that
26+
* A local, general-purpose terminal agent in the spirit of Claude Code / OpenCode, built from two parts that
2727
* already exist: <b>Atmosphere</b>'s built-in OpenAI-compatible agent runtime (streaming, tool loop,
2828
* workspace file tools) and <b>java-llama.cpp</b>'s OpenAI-compatible server.
2929
*
@@ -212,20 +212,33 @@ static ModelParameters modelParameters(AgentOptions options) {
212212
/**
213213
* The default system prompt, or the {@code --system} override.
214214
*
215+
* <p>The default describes a general-purpose agent on this machine, not a coding agent confined to a
216+
* project: a small model reads a narrow role or tool description as a prohibition and then refuses
217+
* requests such as "list the docker images" even though {@code run_command} could do it. With
218+
* {@code --allow-shell} the prompt therefore states that any command line is allowed and that the
219+
* model should run a command rather than explain one; without it, the prompt says so honestly
220+
* instead of letting the model invent a limitation.
221+
*
215222
* @param options the options
216223
* @return the system prompt
217224
*/
218225
static String systemPrompt(AgentOptions options) {
219226
if (options.getSystemPrompt() != null) {
220227
return options.getSystemPrompt();
221228
}
229+
String files = " The file tools ls, read_file, write_file, edit_file, glob, grep, delete and rename work"
230+
+ " on the directory " + options.getWorkspace() + "; their paths are relative to it.";
222231
String shell = options.isAllowShell()
223-
? " Use run_command to build, test or inspect the project with shell commands."
224-
: "";
225-
return "You are a careful coding agent working in the directory " + options.getWorkspace() + "."
226-
+ " Use the tools to inspect and change files: ls, read_file, write_file, edit_file, glob,"
227-
+ " grep, delete, rename. Paths are relative to that directory." + shell
228-
+ " Work step by step: read a file before you edit it, verify the result after a change,"
229-
+ " and finish with a short summary of what you did.";
232+
? " You have full shell access: run_command executes any command line through "
233+
+ ShellTool.shellName() + " on this machine, starting in that directory but not"
234+
+ " limited to it, e.g. docker, git, package managers, build tools or system"
235+
+ " information. When the user asks about this machine or wants something done,"
236+
+ " run the command instead of explaining how to do it."
237+
: " You cannot run shell commands in this session; if a request needs one, say so and"
238+
+ " suggest restarting the agent with --allow-shell.";
239+
return "You are a helpful general-purpose assistant running locally on the user's computer, with"
240+
+ " tools to act on it." + files + shell
241+
+ " Work step by step: read a file before you edit it, check the result after a change,"
242+
+ " and finish with a short summary. Answer in the user's language.";
230243
}
231244
}

‎llama-atmosphere-agent/src/main/java/net/ladenthin/llama/atmosphere/ShellTool.java‎

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
import org.atmosphere.ai.tool.ToolDefinition;
1616

1717
/**
18-
* The {@code run_command} tool: runs a shell command inside the workspace and returns its exit code and
18+
* The {@code run_command} tool: runs any shell command (starting in the workspace) and returns its exit code and
1919
* (merged, truncated) output. Opt-in via {@code --allow-shell} — a model-driven shell is exactly as
2020
* powerful as the user account it runs under.
2121
*/
@@ -40,9 +40,11 @@ private ShellTool() {}
4040
public static ToolDefinition definition(Path workspace, Duration defaultTimeout, int maxOutputChars) {
4141
return ToolDefinition.builder(
4242
TOOL_NAME,
43-
"Run a shell command in the workspace directory and return its exit code and output"
44-
+ " (stdout and stderr merged). Use it to build, test, grep or list files.")
45-
.parameter(PARAM_COMMAND, "The command line to run through the system shell", "string", true)
43+
"Run any command line on this machine through " + shellName() + " and return its exit"
44+
+ " code and output (stdout and stderr merged). It starts in the workspace"
45+
+ " directory but is not limited to it: use it for every task a terminal can do,"
46+
+ " e.g. docker, git, package managers, build tools or system information.")
47+
.parameter(PARAM_COMMAND, "The command line to run through " + shellName(), "string", true)
4648
.parameter(PARAM_TIMEOUT, "Seconds to wait before the command is killed", "integer", false)
4749
.executor(args -> {
4850
Object command = args.get(PARAM_COMMAND);
@@ -55,6 +57,26 @@ public static ToolDefinition definition(Path workspace, Duration defaultTimeout,
5557
.build();
5658
}
5759

60+
/**
61+
* Whether commands run through {@code cmd.exe} rather than {@code sh}.
62+
*
63+
* @return {@code true} on Windows
64+
*/
65+
static boolean isWindows() {
66+
return System.getProperty("os.name", "")
67+
.toLowerCase(java.util.Locale.ROOT)
68+
.contains("win");
69+
}
70+
71+
/**
72+
* The shell {@link #run} uses, named for the model so it writes commands in the right syntax.
73+
*
74+
* @return {@code "cmd.exe on Windows"} or {@code "sh"}
75+
*/
76+
static String shellName() {
77+
return isWindows() ? "cmd.exe on Windows" : "sh";
78+
}
79+
5880
private static Duration timeoutOf(Object raw, Duration fallback) {
5981
if (raw instanceof Number n && n.longValue() > 0) {
6082
return Duration.ofSeconds(n.longValue());
@@ -85,11 +107,8 @@ private static Duration timeoutOf(Object raw, Duration fallback) {
85107
*/
86108
static String run(Path workspace, String command, Duration timeout, int maxOutputChars)
87109
throws IOException, InterruptedException {
88-
boolean windows = System.getProperty("os.name", "")
89-
.toLowerCase(java.util.Locale.ROOT)
90-
.contains("win");
91110
ProcessBuilder builder =
92-
windows ? new ProcessBuilder("cmd.exe", "/c", command) : new ProcessBuilder("sh", "-c", command);
111+
isWindows() ? new ProcessBuilder("cmd.exe", "/c", command) : new ProcessBuilder("sh", "-c", command);
93112
builder.directory(workspace.toFile());
94113
builder.redirectErrorStream(true);
95114
Process process = builder.start();

0 commit comments

Comments
 (0)