FreeBSD jsh coreutils commands fail: [freebsd-main-weak] jsh_ls / jsh_hostname on every coreutil #86

New issue

Open

opened 2026-09-16 21:55:21 -04:00 by ober · 0 comments

ober commented

2026-09-16 21:55:21 -04:00

Owner

Summary

On FreeBSD, every coreutils command (ls, cat, cp, mv, hostname, etc.) fails when run through the jsh-built-in path. Instead of executing the command, jsh prints a message of the form [freebsd-main-weak] jsh_ls (or the relevant jsh_<util> name) and returns success (exit 0) without doing anything.

This affects:

./jsh-freebsd -c 'ls'
./jsh-freebsd -c 'hostname'
Running ls, hostname, and every other coreutil interactively inside the FreeBSD jsh binary
Both when connecting to the FreeBSD host via ssh biggus and running the shell locally on the host

Environment

Host: FreeBSD (reported from host biggus)
Binary: jsh-freebsd built via gmake jsh-freebsd with JSH_FEATURES=all
The FreeBSD build does NOT link the Rust uutils archive (libjsh_coreutils.a) — coreutils are provided as weak C stubs.

Symptom / Reproduction

$ ./jsh-freebsd
user@host:~/jsh (master)
➜ ls
[freebsd-main-weak] jsh_ls
➜ hostname
[freebsd-main-weak] jsh_hostname
➜ echo $?
0

command -V ls inside the broken build reports the command as a jsh builtin rather than /bin/ls.

Root Cause

The (jsh coreutils) module (vendor/jerboa-shell-extras/jerboa-src/src/jsh/coreutils.ss) registers ~90 coreutils commands as shell builtins via register-coreutils!. Each registration is guarded by (when (cdr entry) ...), which checks whether the FFI binding for the command is non-#f.
Each binding is produced by define-coreutil-ffi
```
(define binding
  (and (foreign-entry? symbol)
       (foreign-procedure symbol (int void*) int)))
```
so the binding is truthy whenever Chez's (foreign-entry? symbol) returns true for that symbol (e.g. "jsh_ls").
On FreeBSD the Rust uutils library is not linked. The build scripts instead emit __attribute__((weak)) long jsh_ls() { fprintf(stderr, "[freebsd-main-weak] jsh_ls\n"); ... return 0; } stubs AND register those stub addresses with Sforeign_symbol(...) in the generated C main:
- support/gen-jsh-freebsd-main.ss — jsh_* symbols were in weak-stub-symbols, which is registered via Sforeign_symbol.
- build-jsh-freebsd.ss — emitted stub functions returning 127 and unconditionally registered them via Sforeign_symbol.
- build-jsh-freebsd-cross.ss — same as gen-jsh-freebsd-main.
Result: (foreign-entry? "jsh_ls") returns true, so register-coreutils! installs every coreutil as a jsh builtin pointing at the stub. Once registered, the executor's builtin lookup (builtin-lookup) wins and the normal external-command resolution via which()/ffi-fork-exec (which would find /bin/ls on PATH) is never reached.

Fix (merged as PR #85)

PR: #85 (merged into master)

Changes:

Split the jsh_* coreutils command symbols into a dedicated coreutils-weak-symbols list in the FreeBSD build-C generators.
Weak C stubs are still emitted (required so the linker can resolve the symbols), but they are no longer registered with Sforeign_symbol.
With no Sforeign_symbol registration, (foreign-entry? "jsh_ls") returns false → register-coreutils! skips those commands → the executor falls through to PATH-based external execution (which → ffi-fork-exec → /bin/ls).
build-jsh-freebsd.ss now only registers jsh-coreutils-commands with Sforeign_symbol when has-rust-coreutils? is true (i.e. a real uutils archive is present).

Files changed:

support/gen-jsh-freebsd-main.ss
build-jsh-freebsd.ss
build-jsh-freebsd-cross.ss
VERSION

Verification (on the FreeBSD host)

Rebuilt from the fix branch on biggus:

$ ssh biggus
biggus$ cd ~/jsh && JSH_FEATURES=all gmake jsh-freebsd
biggus$ ./jsh-freebsd -c 'ls'        # real directory listing, exit 0
biggus$ ./jsh-freebsd -c 'hostname'  # prints "biggus", exit 0
biggus$ ./jsh-freebsd -c 'command -V ls'        # "ls is /bin/ls"
biggus$ ./jsh-freebsd -c 'command -V hostname'  # "hostname is /bin/hostname"

ls now lists the directory, hostname prints the host name, and both are resolved to the system binaries on PATH.

A related PR (#67, fix/all-features-followup, "fix: make all-features extras bootstrap and cross builds reliable") originally built on top of the same area and had merge conflicts against master after #85 landed (conflicts in VERSION, support/extras-recorder-raw-control.patch, support/patch-extras-build.sh, test/test-meta-commands.sh). The conflict in support/extras-recorder-raw-control.patch involved master's sigaction-based recorder_relay_job_signals_ignored versus the branch's mutex-locked version; the resolved patch keeps the thread-safe mutex version and applies cleanly to the vendored extras ffi-shim.c. As of writing, that PR was being rebased/updated to resolve the conflicts and pass CI; the required-ci gate additionally runs a full pristine features-all bootstrap + binary build + smoke checks on a FreeBSD runner, which takes a long time.
Root-cause note for future builds: the weak-stub emission in the FreeBSD C generators should never be paired with Sforeign_symbol registration for symbols that Scheme probes with foreign-entry? for capability detection. foreign-entry? is a "does an entry exist" probe, not a "is this entry functional" probe — stub registration defeats the (when (cdr entry) ...) guard.

## Summary On FreeBSD, every coreutils command (ls, cat, cp, mv, hostname, etc.) fails when run through the jsh-built-in path. Instead of executing the command, jsh prints a message of the form `[freebsd-main-weak] jsh_ls` (or the relevant `jsh_<util>` name) and returns success (exit 0) without doing anything. This affects: - `./jsh-freebsd -c 'ls'` - `./jsh-freebsd -c 'hostname'` - Running `ls`, `hostname`, and every other coreutil interactively inside the FreeBSD jsh binary - Both when connecting to the FreeBSD host via `ssh biggus` and running the shell locally on the host ## Environment - Host: FreeBSD (reported from host `biggus`) - Binary: `jsh-freebsd` built via `gmake jsh-freebsd` with `JSH_FEATURES=all` - The FreeBSD build does NOT link the Rust uutils archive (`libjsh_coreutils.a`) — coreutils are provided as weak C stubs. ## Symptom / Reproduction ```sh $ ./jsh-freebsd user@host:~/jsh (master) ➜ ls [freebsd-main-weak] jsh_ls ➜ hostname [freebsd-main-weak] jsh_hostname ➜ echo $? 0 ``` `command -V ls` inside the broken build reports the command as a jsh builtin rather than `/bin/ls`. ## Root Cause 1. The `(jsh coreutils)` module (`vendor/jerboa-shell-extras/jerboa-src/src/jsh/coreutils.ss`) registers ~90 coreutils commands as shell builtins via `register-coreutils!`. Each registration is guarded by `(when (cdr entry) ...)`, which checks whether the FFI binding for the command is non-`#f`. 2. Each binding is produced by `define-coreutil-ffi` ```scheme (define binding (and (foreign-entry? symbol) (foreign-procedure symbol (int void*) int))) ``` so the binding is truthy whenever Chez's `(foreign-entry? symbol)` returns true for that symbol (e.g. `"jsh_ls"`). 3. On FreeBSD the Rust uutils library is not linked. The build scripts instead emit `__attribute__((weak)) long jsh_ls() { fprintf(stderr, "[freebsd-main-weak] jsh_ls\n"); ... return 0; }` stubs AND register those stub addresses with `Sforeign_symbol(...)` in the generated C main: - `support/gen-jsh-freebsd-main.ss` — `jsh_*` symbols were in `weak-stub-symbols`, which is registered via `Sforeign_symbol`. - `build-jsh-freebsd.ss` — emitted stub functions returning 127 and unconditionally registered them via `Sforeign_symbol`. - `build-jsh-freebsd-cross.ss` — same as gen-jsh-freebsd-main. 4. Result: `(foreign-entry? "jsh_ls")` returns true, so `register-coreutils!` installs every coreutil as a jsh builtin pointing at the stub. Once registered, the executor's builtin lookup (`builtin-lookup`) wins and the normal external-command resolution via `which()`/`ffi-fork-exec` (which would find `/bin/ls` on PATH) is never reached. ## Fix (merged as PR #85) PR: https://git.jerboa.sh/ober/jerboa-shell/pulls/85 (merged into master) Changes: - Split the `jsh_*` coreutils command symbols into a dedicated `coreutils-weak-symbols` list in the FreeBSD build-C generators. - Weak C stubs are still emitted (required so the linker can resolve the symbols), but they are **no longer registered with `Sforeign_symbol`**. - With no `Sforeign_symbol` registration, `(foreign-entry? "jsh_ls")` returns false → `register-coreutils!` skips those commands → the executor falls through to PATH-based external execution (`which` → `ffi-fork-exec` → `/bin/ls`). - `build-jsh-freebsd.ss` now only registers `jsh-coreutils-commands` with `Sforeign_symbol` when `has-rust-coreutils?` is true (i.e. a real uutils archive is present). Files changed: - `support/gen-jsh-freebsd-main.ss` - `build-jsh-freebsd.ss` - `build-jsh-freebsd-cross.ss` - `VERSION` ## Verification (on the FreeBSD host) Rebuilt from the fix branch on `biggus`: ```sh $ ssh biggus biggus$ cd ~/jsh && JSH_FEATURES=all gmake jsh-freebsd biggus$ ./jsh-freebsd -c 'ls' # real directory listing, exit 0 biggus$ ./jsh-freebsd -c 'hostname' # prints "biggus", exit 0 biggus$ ./jsh-freebsd -c 'command -V ls' # "ls is /bin/ls" biggus$ ./jsh-freebsd -c 'command -V hostname' # "hostname is /bin/hostname" ``` `ls` now lists the directory, `hostname` prints the host name, and both are resolved to the system binaries on PATH. ## Related / Follow-up - A related PR (#67, `fix/all-features-followup`, "fix: make all-features extras bootstrap and cross builds reliable") originally built on top of the same area and had merge conflicts against master after #85 landed (conflicts in `VERSION`, `support/extras-recorder-raw-control.patch`, `support/patch-extras-build.sh`, `test/test-meta-commands.sh`). The conflict in `support/extras-recorder-raw-control.patch` involved master's sigaction-based `recorder_relay_job_signals_ignored` versus the branch's mutex-locked version; the resolved patch keeps the thread-safe mutex version and applies cleanly to the vendored extras `ffi-shim.c`. As of writing, that PR was being rebased/updated to resolve the conflicts and pass CI; the `required-ci` gate additionally runs a full pristine `features-all` bootstrap + `binary` build + smoke checks on a FreeBSD runner, which takes a long time. - Root-cause note for future builds: the weak-stub emission in the FreeBSD C generators should never be paired with `Sforeign_symbol` registration for symbols that Scheme probes with `foreign-entry?` for capability detection. `foreign-entry?` is a "does an entry exist" probe, not a "is this entry functional" probe — stub registration defeats the `(when (cdr entry) ...)` guard.

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ober/jerboa-shell#86

No description provided.

Rows
Columns