jerboa-aigit

jerboa-aigit scans Git history for evidence that commits or changed lines may have been AI-assisted. It separates recorded provenance from heuristics:

• Git AI notes in refs/notes/ai are reported as recorded authorship evidence.
• Agent/tool metadata in author identity, commit messages, and notes is reported separately.
• Message, code, structure, history, baseline, and SimHash heuristics are scored as probabilistic evidence, not proof.

The current implementation is offline-only. It never calls an LLM or sends repository data over the network.

Usage

./bin/jerboa-aigit scan /path/to/repo --count 25
./bin/jerboa-aigit scan /path/to/repo --format json
./bin/jerboa-aigit scan /path/to/repo --config aigit.json
./bin/jerboa-aigit scan /path/to/repo --from main~20 --to HEAD --file src/app.ss
./bin/jerboa-aigit explain HEAD /path/to/repo --format markdown
./bin/jerboa-aigit stats /path/to/repo --count 100
./bin/jerboa-aigit verify-authorship /path/to/repo --count 25

During development, the same entrypoint can be run directly:

jerboa main-binary.ss scan /path/to/repo --format json

Supported options are --config FILE, --count N, --all, --from REV, --to REV, --commit REV, --file PATH, --include PATH, --exclude PREFIX, --first-parent, --all-parents, --min-lines N, --max-files N, --max-added-lines N, --max-note-bytes N, --threshold HUMAN,AI, --threshold HUMAN,UNCERTAIN,AI, --no-llm, --llm, --provider NAME, --format table|json|jsonl|markdown, --metadata-only, and --heuristics-only.

--config FILE reads guarded JSON. Supported scan keys are path, count, format, from, to, file, exclude, min_lines, metadata_only, and heuristics_only. Supported scoring keys are human_threshold, ai_threshold, weight_text, weight_code, weight_structure, weight_similarity, weight_history, weight_baseline, max_files, max_added_lines, max_note_bytes, git_timeout_seconds, and provider_timeout_seconds. Later CLI flags override earlier config values.

Threshold CLI values override the active config for the current invocation. The scanner has two operative cutoffs: scores below HUMAN are likely-human-style, scores from HUMAN to below AI are mixed-uncertain, and scores at or above AI are likely-ai-assisted. The three-value form is accepted for compatibility with human,uncertain,ai wording; the middle value is retained as a label boundary only and does not create a separate verdict.

--no-llm is the default. --llm --provider local enables an optional local provider adapter when either local_provider_command is set in the JSON config as an argv array or JERBOA_AIGIT_LOCAL_PROVIDER points at an executable. The scanner sends one bounded JSON payload as the final argv item and expects strict JSON on stdout with a numeric score plus optional reason and evidence. The provider executable must be an absolute canonical path outside the scanned repository. Repo-local, symlinked, missing, and group/world-writable provider commands are refused so scanning never executes repository code or unsafe local adapters. Local provider output is reported as a secondary model signal; it never overwrites recorded authorship metadata. Remote/network providers are not implemented in this release, and network_used remains false.

stats reports commit count, distinct authors, recorded tools, recorded AI line counts, metadata/heuristic counts, score-band counts, and signal-category coverage for the selected revision range. Use stats --format json for the same summary as a stable machine-readable object.

explain prints the selected commit's thresholds, raw signals, weighted signal contributions, corroboration bonus, counter-evidence from zero-score signals, and warnings.

Merge commits default to first-parent diffs. Use --all-parents to inspect per-parent merge diffs via Git's -m mode; merge warnings state which mode was used.

By default, scans skip vendor/, generated/, dist/, node_modules/, and .git/ paths. Use --include PATH or --file PATH to inspect one of those paths explicitly. --include and --file accept repository-relative pathspecs; absolute paths and .. components are refused before invoking Git. UTF-8 paths and paths containing spaces, tabs, newlines, and non-ASCII characters are covered by fixture tests using NUL-delimited Git parsing where supported.

Resource limits default to 500 changed files per commit, 20,000 added lines analyzed per commit, and 50,000 bytes per AI note. When a limit is hit, output keeps the bounded data and includes a warning.

Git subprocesses and local provider subprocesses are timeout-bound. Git timeout warnings mean partial scan data may be missing; provider timeout warnings mean the scanner fell back to offline heuristics.

Baseline signals are reported as contextual evidence with zero default weight. The author baseline uses same-author addition history; the repository baseline uses median/MAD-style addition-count deviation across the selected scan window.

Warnings also call out root commits with no parent baseline, merge commits where the scanner intentionally uses the first-parent diff, missing parent objects in shallow history, rename/copy changes, and binary file changes that are skipped by text-line heuristics.

What It Reads

The scanner runs Git commands against the requested repository using fixed argument lists, not shell-interpolated command strings. It reads commit metadata, first-parent diffs, --numstat, added patch lines, and AI authorship notes from refs/notes/ai. It does not execute repository code and does not write to the scanned worktree, refs, notes, hooks, or config.

Output Contract

JSON output includes:

• detector_version
• config_hash
• analysis provenance fields: analysis_provider, provider, llm_used, and network_used
• repository path
• commit and parent IDs
• author/committer-facing metadata
• changed files, additions, deletions, and added line count
• recorded AI note presence and excerpt
• structured recorded_attribution entries for supported Git AI note line ranges (source, tool, model, session, path, start, end)
• structured recovered_attribution entries for inferred identity evidence (source, agent, confidence, recorded_provenance:false, evidence)
• file_findings entries that attach matching recorded note ranges to changed files without duplicating commit-level heuristic scores
• unmatched_recorded_attribution for recorded ranges whose path was not part of the changed-file set
• metadata hits
• raw signal scores, weights, evidence, and limitations
• aggregate score and verdict
• warnings for missing or unavailable evidence

Verdicts are intentionally conservative:

• recorded-ai-authorship
• metadata-indicated-agent
• likely-ai-assisted
• mixed-uncertain
• likely-human-style
• insufficient-evidence

Only recorded authorship metadata is high-confidence provenance. Heuristic verdicts can be wrong for generated scaffolds, formatter-only changes, disciplined commit conventions, bot commits, and large refactors.

Evaluation and Measured Limitations

The committed evaluation corpus is in eval/labels.tsv, with generated fixtures from eval/generate-fixtures.sh. Regenerate the current report with:

eval/report.py --output eval/sample-report.md
eval/sensitivity.py --output eval/sensitivity-report.md
eval/differential.py --output eval/differential-report.md

Current sample metrics are committed in eval/sample-report.md. On the small generated corpus, the latest report shows overall precision/recall/F1 of 1.000, recorded provenance precision/recall/F1 of 1.000, metadata F1 of 0.667, and heuristic-only F1 of 1.000. These numbers are guardrails, not a broad accuracy claim: the corpus has 10 labeled cases, includes one ambiguous case excluded from rate denominators, and is intentionally fixture-sized.

Known false-positive classes:

• generated/vendor or lockfile updates if explicitly included;
• formatter-only or mechanical refactor commits with broad regular diffs;
• bot commits whose metadata resembles AI-agent identity;
• disciplined commit-message templates that match generated prose patterns;
• repetitive framework scaffolds written by humans.

Known false-negative classes:

• AI-assisted changes with no recorded notes, no agent metadata, and ordinary human-style diffs;
• small edits below meaningful heuristic thresholds;
• copied AI-generated code edited enough to avoid similarity/style signals;
• unsupported provenance formats outside the documented Git AI note shapes;
• remote/model evidence when no explicit local provider is configured.

The differential report in eval/differential-report.md compares overlapping fixture cases with installed sniff and treats git-ai refs/notes/ai as the recorded-provenance reference. Cadence comparison is recorded when CADENCE_BIN points at a prebuilt executable; otherwise it is reported as unavailable rather than building or modifying the sibling checkout.

Development

make test
make verify
make binary

All Jerboa source is in main-binary.ss. Per AGENTS.md, edit .ss files only with Jerboa MCP balanced tools and run balance plus verification after every change.