jerboa-semgrep

Pure Jerboa Semgrep port scaffold.

This repository is intentionally library-first. The current executable slice supports a compatibility-oriented Semgrep subset:

• YAML/JSON-compatible Semgrep config parsing for rules:, including config directories.
• Search rules with pattern, pattern-regex, pattern-either, and patterns.
• Structural metavariables like $X, repeated metavariable consistency, and Semgrep-style ... and $...ARGS wildcard matching for unfielded child sequences.
• A limited regex-backed generic matcher for generic text targets, including plain ellipsis, $X, $...X, repeated metavariable consistency, and comment-tolerant whitespace separators.
• as: $X captures for positive patterns entries.
• patterns filters for pattern-not, pattern-not-regex, pattern-inside, pattern-not-inside, metavariable-regex, and metavariable-pattern.
• Safe metavariable-comparison support for direct string comparisons and numeric int($X)/float($X) comparisons.
• A minimal taint mode for ordered source/sink/sanitizer pattern lists with shared-metavariable compatibility.
• Language aliases for the supported languages, including python3, python2, py, js, jsx, ts, and tsx.
• Rule metadata in JSON output, rendered metavariable messages, rendered fix: and fix-regex, string-level autofix application, and formula-local focus-metavariable range focusing.
• A generated Semgrep language registry from upstream lang.json, with parser support still limited to the wired tree-sitter grammars.
• Target parsing via jerboa-treesitter for JSON, Python, JavaScript, JSX, and TypeScript/TSX parsed through the JavaScript grammar as a fallback.
• Recursive directory targets with conservative default source-file filtering.
• Descriptor-relative target/config walks that skip symlinks, reject hard links, pin directory identities, enforce cumulative traversal budgets, and commit autofixes only if the originally read inode is still in place.
• A first .semgrepignore pass with ! negation for scanned directory roots, plus line-level nosemgrep suppression, including rule-specific comments.
• CLI compatibility for semgrep scan --json|--sarif|--text --config RULES TARGET, plus --include, --exclude, --severity, --autofix, and stdin target - with --lang.
• Semgrep-like finding records and JSON/SARIF/text output, including extra.metavars for structural captures.

Unsupported Semgrep rule forms fail explicitly instead of pretending to be compatible. Tree-sitter grammars are provided by jerboa-treesitter, which uses statically linked grammars only.

Build, Test, And Release Gates

make test

JERBOA_TREESITTER_HOME defaults to ../jerboa-treesitter. That sibling checkout must be built first because parser FFI is isolated there. CI instead fetches the exact commit and tree in dependency-lock.tsv. The network Jerbuild bootstrap is deliberately fail-closed until a producer signing identity, signed manifest, and asset digest are provisioned in support/jerbuild-bootstrap.lock and support/jerboa-release-signers; CI never executes a merely versioned, same-origin download.

Run the release/security gates:

make security
make malformed-corpus
make verify
make target-evidence
make release-evidence

make verify runs static security checks, dependency audit, import checks, the 512-case hostile rule/source corpus, and the smoke/regression test suite. make release-evidence writes dist/release-evidence, including Semgrep SBOM/source hashes, refreshed sibling jerboa-treesitter grammar/runtime evidence, and a repeated-transpile reproducibility report for tracked generated Scheme artifacts, normalized Jerbuild hash metadata, repeated corpus output, and target scanner proof status.

Production support requires marker-complete target proof through JSEMGREP_TARGET_PROOF_FILE with JSEMGREP_REQUIRE_TARGET_PROOF=1; without it, target-only oracle, compatibility, review, autofix, and output-redaction gates remain blocked.

Input size caps are enforced before rule configs and targets are fully read:

• JSEMGREP_MAX_CONFIG_CHARS, default 4194304
• JSEMGREP_MAX_TARGET_CHARS, default 8388608

Untrusted pattern-regex, pattern-not-regex, nested regex formulae, and metavariable-regex constraints use the bundled Rust finite-automata engine. General look-around and backreferences fail closed; the bounded compatibility layer supports only a leading capture-free (?!literal) metavariable filter. Patterns are capped at 8192 characters, capture groups at 128, and findings at 50000 per rule scan. fix-regex also uses the linear engine with a 50000 replacement/16 MiB output cap; replacement text is currently literal, and capture-template $ syntax fails closed. make adversarial-fs covers external-file links, directory cycles, hard links, linked configs, and autofix link rejection.

This repository is expected to remain network-free, process-execution-free, and free of direct Scheme FFI. Native parser linkage belongs in jerboa-treesitter and is audited by make audit.

Regenerate the Semgrep language registry from the sibling checkout:

make generate

Run the focused compatibility oracle against a current Semgrep binary:

make oracle

make oracle looks for a sibling Semgrep checkout binary, then semgrep on PATH. Set SEMGREP_CURRENT=/path/to/semgrep to override it. If no current Semgrep binary is available, the oracle skips cleanly. The current local baseline passes 42/42 normalized oracle cases against packaged Semgrep 1.164.0.