#82 - FreeBSD 15.1 amd64 ta6fb Chez 10.4.0 compile-program hangs with large import closure — full root-cause work plan - ober/jerboa

ober commented

2026-09-17 20:45:02 -04:00

Owner

Corrected investigation record

This issue tracks the unresolved Chez 10.4.0 ta6fb compiler divergence observed on FreeBSD 15.1 amd64 (plaisio) while building ober/jerboa-code.

Live reproduction

On plaisio:

cd ~/mine/jerboa-code
PATH=/home/user/bin:/usr/local/bin:/usr/bin:/bin gmake binary

The build reaches [1/5] Program compile. With the old shadowed Jerboa 0.12.6 binary, the fresh jerboa runtime cross-wpo-compile.ss child stays at about 99% CPU, grows to roughly 4.6 GB RSS, and leaves program.so and program.wpo at zero bytes. Forcing the installed Jerboa 0.12.10 binary selects:

[ta6fb workaround] in-process compile path (issue #78)

but hangs identically: about 4.7 GB RSS, 99% CPU, and zero-byte output. Thus PR #79's in-process workaround is ineffective for this real jcode closure and was not validated on FreeBSD before merge.

An lldb attach during the spin showed sweep_generation -> S_gc_ocd_entry -> S_do_gc. Raising collect-trip-bytes as high as 2^35 let RSS exceed 38 GB without output; optimize-level 0 still hangs; preloading all 197 compiled library .so files still hangs. These controls do not establish a simple GC-threshold fix.

Reduction achieved

The temporary source-bisection harness reduced the failure as follows:

gmake binary [1/5]
  -> (jcode ui cli) alone
  -> (jcode ui tui) alone
  -> all four quarters of tui's imports compile independently
  -> same 188-form library with empty exports compiles in ~1 second
  -> exporting tui-main makes it hang
  -> tui-main's first two top-level expressions compile
  -> adding its with-tui expression hangs
  -> the large let* prefix through clause 17 compiles
  -> adding clause 18 hangs; clause 18 calls event-loop
  -> replacing event-loop with (void) compiles
  -> small synthetic loop/guard/try bodies compile
  -> the real event-loop definition remains the trigger

The relevant real shape is:

(def (event-loop state)
  (let loop ()
    (guard (e [else ... (unless (app-state-quit? state) (loop))])
      ... tick/memstats/sidebar updates ...
      ... agent-event draining ...
      ... tb-peek-event and multi-branch dispatch ...
      ... draw/present ...
      (unless (app-state-quit? state) (loop)))))

The same source closure compiles on macOS arm64 in about two minutes. This currently points to a ta6fb/x86-64 Chez compiler-path defect, not a proven application runtime hang.

Eval hypothesis was tested and rejected as a sufficient fix

The jcode function:

(def (maybe-stop-jcode-repl!)
  (guard (e (#t (void)))
    ((eval 'stop-jcode-repl! (interaction-environment)))))

was transformed in the generated source to a guard around (void). The full exported tui-main compile still hung because the real event-loop remained reachable. Isolated synthetic eval, interaction-environment, and guard forms compiled quickly. Do not claim that removing this eval fixes the compiler hang without a fresh, source-faithful proof; the current evidence says the event-loop compiler path is the stronger trigger.

Ready toolchain and next investigation

A native ta6fb toolchain was built successfully on plaisio:

~/work/jerboa-fbsd-trace/bin/jerboa
~/work/jerboa-fbsd-trace/.chez/bin/scheme

Checkout HEAD used for the build was 51512333; gmake build completed with 22 compiled and 0 skipped.

Next steps:

Reproduce in a clean object directory with the native toolchain, avoiding the shadowed 0.12.6 binary and stale zero-byte artifacts.
Instrument s/compile.ss around the cfh0 read/expand loop (compile.ss:646-702), compile-file-help1/2, and the expand call. Add flushed low-allocation phase/form counters.
Instrument s/syntax.ss chi-body and library-body-to-IR paths, then s/cpnanopass.ss around lambda/letrec/guard/continuation lowering. Instrument s/x86_64.ss only after proving earlier passes complete.
Compare the final marker and lldb/procstat stack against macOS at the same source/toolchain revision.
Reduce the real event-loop body while preserving its guard handler and recursive binding. Validate every generated source and avoid stale scratch harnesses.
Once a standalone Chez form reproduces the loop, add a Chez regression test, patch the vendored Chez source, rebuild ta6fb, and rerun jcode.
Add a required FreeBSD gmake binary CI job; current FreeBSD CI runs gmake build but does not exercise this binary path.

No repository source was changed during this investigation. The durable handoff is ~/fix-jerboa-fbsd-trace-hangs-eval.md.

## Corrected investigation record This issue tracks the unresolved Chez 10.4.0 `ta6fb` compiler divergence observed on FreeBSD 15.1 amd64 (`plaisio`) while building `ober/jerboa-code`. ### Live reproduction On plaisio: ```sh cd ~/mine/jerboa-code PATH=/home/user/bin:/usr/local/bin:/usr/bin:/bin gmake binary ``` The build reaches `[1/5] Program compile`. With the old shadowed Jerboa 0.12.6 binary, the fresh `jerboa runtime cross-wpo-compile.ss` child stays at about 99% CPU, grows to roughly 4.6 GB RSS, and leaves `program.so` and `program.wpo` at zero bytes. Forcing the installed Jerboa 0.12.10 binary selects: ```text [ta6fb workaround] in-process compile path (issue #78) ``` but hangs identically: about 4.7 GB RSS, 99% CPU, and zero-byte output. Thus PR #79's in-process workaround is ineffective for this real jcode closure and was not validated on FreeBSD before merge. An lldb attach during the spin showed `sweep_generation -> S_gc_ocd_entry -> S_do_gc`. Raising `collect-trip-bytes` as high as `2^35` let RSS exceed 38 GB without output; `optimize-level 0` still hangs; preloading all 197 compiled library `.so` files still hangs. These controls do not establish a simple GC-threshold fix. ### Reduction achieved The temporary source-bisection harness reduced the failure as follows: ```text gmake binary [1/5] -> (jcode ui cli) alone -> (jcode ui tui) alone -> all four quarters of tui's imports compile independently -> same 188-form library with empty exports compiles in ~1 second -> exporting tui-main makes it hang -> tui-main's first two top-level expressions compile -> adding its with-tui expression hangs -> the large let* prefix through clause 17 compiles -> adding clause 18 hangs; clause 18 calls event-loop -> replacing event-loop with (void) compiles -> small synthetic loop/guard/try bodies compile -> the real event-loop definition remains the trigger ``` The relevant real shape is: ```scheme (def (event-loop state) (let loop () (guard (e [else ... (unless (app-state-quit? state) (loop))]) ... tick/memstats/sidebar updates ... ... agent-event draining ... ... tb-peek-event and multi-branch dispatch ... ... draw/present ... (unless (app-state-quit? state) (loop))))) ``` The same source closure compiles on macOS arm64 in about two minutes. This currently points to a ta6fb/x86-64 Chez compiler-path defect, not a proven application runtime hang. ### Eval hypothesis was tested and rejected as a sufficient fix The jcode function: ```scheme (def (maybe-stop-jcode-repl!) (guard (e (#t (void))) ((eval 'stop-jcode-repl! (interaction-environment))))) ``` was transformed in the generated source to a guard around `(void)`. The full exported `tui-main` compile still hung because the real event-loop remained reachable. Isolated synthetic `eval`, `interaction-environment`, and `guard` forms compiled quickly. Do not claim that removing this eval fixes the compiler hang without a fresh, source-faithful proof; the current evidence says the event-loop compiler path is the stronger trigger. ### Ready toolchain and next investigation A native ta6fb toolchain was built successfully on plaisio: ```text ~/work/jerboa-fbsd-trace/bin/jerboa ~/work/jerboa-fbsd-trace/.chez/bin/scheme ``` Checkout HEAD used for the build was `51512333`; `gmake build` completed with 22 compiled and 0 skipped. Next steps: 1. Reproduce in a clean object directory with the native toolchain, avoiding the shadowed 0.12.6 binary and stale zero-byte artifacts. 2. Instrument `s/compile.ss` around the `cfh0` read/expand loop (`compile.ss:646-702`), `compile-file-help1/2`, and the `expand` call. Add flushed low-allocation phase/form counters. 3. Instrument `s/syntax.ss` `chi-body` and library-body-to-IR paths, then `s/cpnanopass.ss` around lambda/letrec/guard/continuation lowering. Instrument `s/x86_64.ss` only after proving earlier passes complete. 4. Compare the final marker and lldb/procstat stack against macOS at the same source/toolchain revision. 5. Reduce the real event-loop body while preserving its guard handler and recursive binding. Validate every generated source and avoid stale scratch harnesses. 6. Once a standalone Chez form reproduces the loop, add a Chez regression test, patch the vendored Chez source, rebuild ta6fb, and rerun jcode. 7. Add a required FreeBSD `gmake binary` CI job; current FreeBSD CI runs `gmake build` but does not exercise this binary path. No repository source was changed during this investigation. The durable handoff is `~/fix-jerboa-fbsd-trace-hangs-eval.md`.

fix-jerboa-freebsd-forever-issues-no-78.md

13 KiB