jerboa-mcp executes raw scheme binary (eval/verify exit 127, AGENTS.md violation) + stale jmcp hides writer tools #53

Closed
opened 2026-09-10 21:32:19 -04:00 by ober · 0 comments
Owner

jerboa-mcp shells out to a raw scheme binary — eval/verify broken on machines without it, violates the repo's own AGENTS.md ban

Summary

The MCP server (mcp/server.ss) launches a bare Chez scheme binary as its code-execution runtime for jerboa_eval, jerboa_write_file (verify=true), and related verify/compile paths. This:

  1. Violates this repository's own AGENTS.md, which says under "Chez Scheme Is Off-Limits — Jerboa Only": "NEVER invoke the scheme binary … The ONLY legal ways to run code are jerboa run file.ss, jerboa eval '<expr>', jerboa test, jerboa repl, jerboa-mcp tools, project make targets." The MCP server is the one component that must never do this, and it does.
  2. Fails outright on machines with no scheme on PATH and none of the hardcoded fallback paths present: every eval/verify subprocess dies with exit 127 (/bin/sh: scheme: command not found).
  3. Blocked knowledge-store work: jerboa_write_file with verify: true refuses writes when verification can't run, which silently degrades the documented "Save What You Learn" workflow (recipes/error-fixes can't be verified before saving; agents fall back to verify: false).

Environment

  • macOS tarm64osx, jerboa/jmcp at ~/.local/bin/jerboa (jmcp → jerboa symlink)
  • MCP env: JERBOA_MCP_REPO=/Users/user/mine/jerboa, JERBOA_MCP_DATA_DIR=/Users/user/.jmap/data
  • No scheme binary on PATH; no ~/mine/ChezScheme/tarm64osx/bin/tarm64osx/scheme; no ~/.local/bin/scheme
  • Observed across sessions 2026-09-09 → 2026-09-10 (still reproducing after today's jmcp rebuild/restart)

Defect 1 — jerboa_eval exit 127

Reproduction (any session, or stdio):

tools/call jerboa_eval {"expression": "(+ 1 2)"}

Result:

/bin/sh: scheme: command not found

JERBOA-MCP-EXIT-STATUS: 127

Command detail:
  command: scheme --libdirs /Users/user/.cache/jerbuild/<hash>/lib --script /tmp/jmcp-<n>.ss
  temp_script: /tmp/jmcp-<n>.ss

Same failure today with the freshly rebuilt jmcp.

Defect 2 — jerboa_write_file verify=true blocks writes, misattributes the cause

15 consecutive jerboa_write_file {verify: true} calls during a verified knowledge import were blocked by the same exit-127 (logged in ~/mcp-issues.md, entries 2026-09-10T23:30:15.991Z … .809Z). Two secondary problems in the same output:

  • Wrong repair advice: the failure advisor matched the known fix missing-libdirs ("Run with: jerboa your-file.ss") — a JERBOA_HOME/libdirs problem is not a missing-binary problem, so the agent gets sent down the wrong repair path.
  • 30s watchdog noise: the runner's slow-process killer appends Terminated: 15 … event=process.slow elapsed_ms=30000 blocks for what is an instant 127.

Also observed: eval failures have previously been returned with isError: false, so clients can record success when nothing ran (see feature suggestion eval-runtime-preflight-and-status, catalog t:xwei4k, filed 2026-09-09).

Defect 3 — hardcoded sibling-checkout fallbacks in scheme-path

mcp/server.ss:

(def (scheme-path)
  (def explicit (getenv "JERBOA_MCP_SCHEME_PATH"))
  (def repo-scheme (repo-local-scheme-path))
  (def home-scheme (path-join (jerboa-home #f) ".chez" "bin" "scheme"))
  (cond
    [(and explicit (file-exists? explicit)) explicit]
    [repo-scheme repo-scheme]
    [(file-exists? home-scheme) home-scheme]
    [(file-exists? (path-join (getenv "HOME") "mine" "ChezScheme" "tarm64osx" "bin" "tarm64osx" "scheme")) ...]
    [(file-exists? (path-join (getenv "HOME") ".local" "bin" "scheme")) ...]
    [else "scheme"]))
  • ~/mine/ChezScheme/tarm64osx/... is exactly the machine-specific sibling-checkout pattern this repo's own contribution rules forbid in build files.
  • tool-preflight reports scheme: … MISSING as a first-class concept — the server's health model requires a Chez install that AGENTS.md says must never be used directly.

Defect 4 — stale jmcp binaries silently hide documented writer tools

Today's investigation of "writers missing from the catalog":

  • Current mcp/server.ss critical-tools includes jerboa_howto_add and jerboa_error_fix_add (so hybrid mode should list them).
  • The installed ~/.local/bin/jerboa (built Sep 10 13:13) predates that source (16:38) and serves 45 tools without either writer, while jerboa_eval, jerboa_anti_pattern_add, etc. are present.
  • Both opencode (~/.config/opencode/opencode.json) and codex (~/.codex/config.toml) run jmcp with only JERBOA_MCP_REPO/JERBOA_MCP_DATA_DIR; JERBOA_MCP_MODE is unset → default hybrid. Hybrid is fine — the binary was just stale.

Consequences (logged ~/mcp-issues.md 2026-09-10 ~15:30): AGENTS.md-mandated knowledge saves (verified incremental SHA-256 recipe, no entry for "fcntl" error-fix) could not be written; sessions lose hours discovering the catalog shrank. The dispatcher still routes to registered-but-hidden tools (jerboa tool + tool: "jerboa_howto_add" works), but nothing tells the client its binary is stale, and nothing in tools/list indicates server build/version.

Requested changes

  1. Route ALL code execution through the Jerboa toolchain — jerboa eval '<expr>', jerboa run, jerboa test (or the internal jerbuild runtime the CLI itself uses). Never spawn scheme from the MCP server. Delete the sibling-path fallbacks in scheme-path and the scheme-centric tool-preflight.
  2. Preflight the runtime once at startup (and on demand): if no legal execution entry point exists, fail fast with a clear isError: true message naming the missing piece and the env var to set — never return success-shaped results for subprocess failures (exit 126/127, killed, etc.).
  3. Fix error classification for missing-binary exits so the advisor stops suggesting missing-libdirs for exit 127, and suppress the 30s slow-watchdog block when the process already exited.
  4. Expose server version/build in initialize + tools/list (e.g. serverInfo.version = jerboa VERSION + build timestamp) so clients can detect stale jmcp binaries before losing writer tools.
  5. Consider documenting JERBOA_MCP_MODE (full/hybrid/mini) and the dispatcher-can-route-to-hidden-tools behavior in the repo docs, since AGENTS.md tells agents to use writer tools that hybrid+stale-binary combinations can hide.
  • Feature suggestion eval-runtime-preflight-and-status (catalog t:xwei4k, 2026-09-09) — covers the preflight/isError part; this issue adds the AGENTS.md violation, write-blocking, misattribution, and staleness angles.
  • ~/mcp-issues.md entries: 2026-09-09 (writer internal error), 2026-09-10T23:30 (6 blocked verify=true writes + fallback verification manifest /Users/user/mcp-import-examples/verification.json), 2026-09-10 ~15:30 (writers not exposed).

Workarounds in current use

  • jerboa_eval: none possible — tool unusable on this machine (verified again today).
  • jerboa_write_file: always pass verify: false, verify separately via jerboa_check_balance + a jerbuild-through-jerboa exec, or project make targets.
  • Writer tools: call through the compact dispatcher by name (jerboa tool with tool: "jerboa_howto_add") — works even when hidden from tools/list.
# jerboa-mcp shells out to a raw `scheme` binary — eval/verify broken on machines without it, violates the repo's own AGENTS.md ban ## Summary The MCP server (`mcp/server.ss`) launches a **bare Chez `scheme` binary** as its code-execution runtime for `jerboa_eval`, `jerboa_write_file` (verify=true), and related verify/compile paths. This: 1. **Violates this repository's own AGENTS.md**, which says under "Chez Scheme Is Off-Limits — Jerboa Only": *"NEVER invoke the `scheme` binary … The ONLY legal ways to run code are `jerboa run file.ss`, `jerboa eval '<expr>'`, `jerboa test`, `jerboa repl`, jerboa-mcp tools, project `make` targets."* The MCP server is the one component that must never do this, and it does. 2. **Fails outright** on machines with no `scheme` on PATH and none of the hardcoded fallback paths present: every eval/verify subprocess dies with exit 127 (`/bin/sh: scheme: command not found`). 3. **Blocked knowledge-store work**: `jerboa_write_file` with `verify: true` refuses writes when verification can't run, which silently degrades the documented "Save What You Learn" workflow (recipes/error-fixes can't be verified before saving; agents fall back to `verify: false`). ## Environment - macOS tarm64osx, `jerboa`/`jmcp` at `~/.local/bin/jerboa` (`jmcp` → `jerboa` symlink) - MCP env: `JERBOA_MCP_REPO=/Users/user/mine/jerboa`, `JERBOA_MCP_DATA_DIR=/Users/user/.jmap/data` - No `scheme` binary on PATH; no `~/mine/ChezScheme/tarm64osx/bin/tarm64osx/scheme`; no `~/.local/bin/scheme` - Observed across sessions 2026-09-09 → 2026-09-10 (still reproducing after today's jmcp rebuild/restart) ## Defect 1 — `jerboa_eval` exit 127 Reproduction (any session, or stdio): ``` tools/call jerboa_eval {"expression": "(+ 1 2)"} ``` Result: ``` /bin/sh: scheme: command not found JERBOA-MCP-EXIT-STATUS: 127 Command detail: command: scheme --libdirs /Users/user/.cache/jerbuild/<hash>/lib --script /tmp/jmcp-<n>.ss temp_script: /tmp/jmcp-<n>.ss ``` Same failure today with the freshly rebuilt jmcp. ## Defect 2 — `jerboa_write_file` verify=true blocks writes, misattributes the cause 15 consecutive `jerboa_write_file {verify: true}` calls during a verified knowledge import were **blocked** by the same exit-127 (logged in `~/mcp-issues.md`, entries `2026-09-10T23:30:15.991Z` … `.809Z`). Two secondary problems in the same output: - **Wrong repair advice**: the failure advisor matched the known fix `missing-libdirs` ("Run with: jerboa your-file.ss") — a JERBOA_HOME/libdirs problem is not a missing-binary problem, so the agent gets sent down the wrong repair path. - **30s watchdog noise**: the runner's slow-process killer appends `Terminated: 15 … event=process.slow elapsed_ms=30000` blocks for what is an instant 127. Also observed: eval failures have previously been returned with `isError: false`, so clients can record success when nothing ran (see feature suggestion `eval-runtime-preflight-and-status`, catalog `t:xwei4k`, filed 2026-09-09). ## Defect 3 — hardcoded sibling-checkout fallbacks in `scheme-path` `mcp/server.ss`: ```scheme (def (scheme-path) (def explicit (getenv "JERBOA_MCP_SCHEME_PATH")) (def repo-scheme (repo-local-scheme-path)) (def home-scheme (path-join (jerboa-home #f) ".chez" "bin" "scheme")) (cond [(and explicit (file-exists? explicit)) explicit] [repo-scheme repo-scheme] [(file-exists? home-scheme) home-scheme] [(file-exists? (path-join (getenv "HOME") "mine" "ChezScheme" "tarm64osx" "bin" "tarm64osx" "scheme")) ...] [(file-exists? (path-join (getenv "HOME") ".local" "bin" "scheme")) ...] [else "scheme"])) ``` - `~/mine/ChezScheme/tarm64osx/...` is exactly the machine-specific sibling-checkout pattern this repo's own contribution rules forbid in build files. - `tool-preflight` reports `scheme: … MISSING` as a first-class concept — the server's health model *requires* a Chez install that AGENTS.md says must never be used directly. ## Defect 4 — stale jmcp binaries silently hide documented writer tools Today's investigation of "writers missing from the catalog": - Current `mcp/server.ss` `critical-tools` **includes** `jerboa_howto_add` and `jerboa_error_fix_add` (so hybrid mode should list them). - The installed `~/.local/bin/jerboa` (built Sep 10 13:13) predates that source (16:38) and serves **45 tools without either writer**, while `jerboa_eval`, `jerboa_anti_pattern_add`, etc. are present. - Both opencode (`~/.config/opencode/opencode.json`) and codex (`~/.codex/config.toml`) run jmcp with only `JERBOA_MCP_REPO`/`JERBOA_MCP_DATA_DIR`; `JERBOA_MCP_MODE` is unset → default `hybrid`. Hybrid is fine — the binary was just stale. Consequences (logged `~/mcp-issues.md` 2026-09-10 ~15:30): AGENTS.md-mandated knowledge saves (verified incremental SHA-256 recipe, `no entry for "fcntl"` error-fix) could not be written; sessions lose hours discovering the catalog shrank. The dispatcher still routes to registered-but-hidden tools (`jerboa` tool + `tool: "jerboa_howto_add"` works), but nothing tells the client its binary is stale, and nothing in tools/list indicates server build/version. ## Requested changes 1. **Route ALL code execution through the Jerboa toolchain** — `jerboa eval '<expr>'`, `jerboa run`, `jerboa test` (or the internal jerbuild runtime the CLI itself uses). Never spawn `scheme` from the MCP server. Delete the sibling-path fallbacks in `scheme-path` and the scheme-centric `tool-preflight`. 2. **Preflight the runtime once at startup** (and on demand): if no legal execution entry point exists, fail fast with a clear `isError: true` message naming the missing piece and the env var to set — never return success-shaped results for subprocess failures (exit 126/127, killed, etc.). 3. **Fix error classification** for missing-binary exits so the advisor stops suggesting `missing-libdirs` for exit 127, and suppress the 30s slow-watchdog block when the process already exited. 4. **Expose server version/build in initialize + tools/list** (e.g. `serverInfo.version` = jerboa VERSION + build timestamp) so clients can detect stale jmcp binaries before losing writer tools. 5. Consider documenting `JERBOA_MCP_MODE` (`full`/`hybrid`/`mini`) and the dispatcher-can-route-to-hidden-tools behavior in the repo docs, since AGENTS.md tells agents to use writer tools that hybrid+stale-binary combinations can hide. ## Related - Feature suggestion `eval-runtime-preflight-and-status` (catalog `t:xwei4k`, 2026-09-09) — covers the preflight/isError part; this issue adds the AGENTS.md violation, write-blocking, misattribution, and staleness angles. - `~/mcp-issues.md` entries: 2026-09-09 (writer internal error), 2026-09-10T23:30 (6 blocked verify=true writes + fallback verification manifest `/Users/user/mcp-import-examples/verification.json`), 2026-09-10 ~15:30 (writers not exposed). ## Workarounds in current use - `jerboa_eval`: none possible — tool unusable on this machine (verified again today). - `jerboa_write_file`: always pass `verify: false`, verify separately via `jerboa_check_balance` + a `jerbuild`-through-jerboa exec, or project `make` targets. - Writer tools: call through the compact dispatcher by name (`jerboa` tool with `tool: "jerboa_howto_add"`) — works even when hidden from tools/list.
ober closed this issue 2026-09-10 22:09:27 -04:00
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ober/jerboa#53
No description provided.