Complete durability and performance validation #13

Closed

ober wants to merge 26 commits from fix/dtrace-perf-complete into master

pull from: fix/dtrace-perf-complete

merge into: ober:master

ober:master

ober:fix/close-result-cache

ober:fix/dtrace-perf-remediation

ober:jpkg-declare-jsqlite-executable

ober:feat/dtrace-sql-semantics

ober:docs/agents-checkout-hygiene

ober:docs/agents-fj-cheatsheet

ober:fix/global-cache-thread-safety

ober:fix/schema-btree-root-split

ober:fix/integrity-virtual-table-schema

ober:fix/st02-st04-storage-durability

ober:chore/local-mine-authority-agents-20260803

ober:feature/agents-cicd-check

ober:faster

ober:jerbuild

Conversation 0 Commits 26 Files changed 23 +1296 -119

ober commented 2026-09-14 15:10:39 -04:00

Owner

Copy link

Implements the remaining durability, integrity, concurrency, benchmark, and DTrace validation work from dtrace-perf.md.

Highlights:

• Fail-closed database locking and monotonic busy deadlines; commit/release durability ordering.
• Full index/table correspondence checks for rowid and WITHOUT ROWID tables, including partial and expression indexes.
• Runtime trace identity/span-parent improvements with lazy FFI loading safe for static startup.
• Persistent multi-process rollback/WAL commit ledger with barriers, reader sampling, checksums, and upstream validation.
• Reproducible cold/persistent benchmark matrix across batch sizes and journal modes.
• DTrace run harness and strict provenance/result validator.
• Pinned vendor oracle documentation and version bump to 0.1.11.

Verification:

• make binary: pass on macOS arm64.
• make unit: 1203 checks, 0 failures, 0 errors.
• make diff: 199 checks, 0 failures, 0 errors against pinned oracle.
• make robustness: corruption, DML fuzz, concurrency, binary, ledger, and query fuzz lanes pass.
• Benchmark smoke (N=100 REPS=1): all 10 matrix scenarios pass with checksums.
• Changed-line security scan: no findings.

The live DTrace collector smoke was also exercised; this host's macOS System Integrity Protection prevents attachment, and the harness retained/validated the failure evidence rather than reporting a false trace success.

Implements the remaining durability, integrity, concurrency, benchmark, and DTrace validation work from `dtrace-perf.md`. Highlights: - Fail-closed database locking and monotonic busy deadlines; commit/release durability ordering. - Full index/table correspondence checks for rowid and WITHOUT ROWID tables, including partial and expression indexes. - Runtime trace identity/span-parent improvements with lazy FFI loading safe for static startup. - Persistent multi-process rollback/WAL commit ledger with barriers, reader sampling, checksums, and upstream validation. - Reproducible cold/persistent benchmark matrix across batch sizes and journal modes. - DTrace run harness and strict provenance/result validator. - Pinned vendor oracle documentation and version bump to 0.1.11. Verification: - `make binary`: pass on macOS arm64. - `make unit`: 1203 checks, 0 failures, 0 errors. - `make diff`: 199 checks, 0 failures, 0 errors against pinned oracle. - `make robustness`: corruption, DML fuzz, concurrency, binary, ledger, and query fuzz lanes pass. - Benchmark smoke (`N=100 REPS=1`): all 10 matrix scenarios pass with checksums. - Changed-line security scan: no findings. The live DTrace collector smoke was also exercised; this host's macOS System Integrity Protection prevents attachment, and the harness retained/validated the failure evidence rather than reporting a false trace success.

ober added 1 commit 2026-09-14 15:10:40 -04:00

Complete durability and performance validation 76358e6714

ober added 1 commit 2026-09-14 15:15:48 -04:00

Make differential oracle failures fatal 487ce374d9

ober added 1 commit 2026-09-14 15:20:57 -04:00

Fetch oracle before differential checks 46680f2479

ober added 1 commit 2026-09-14 15:22:14 -04:00

Validate successful DTrace result bundles 4cdf60088c

ober added 1 commit 2026-09-14 15:29:03 -04:00

Resolve fetched oracle paths before make prerequisites ffad25f69c

ober added 1 commit 2026-09-14 15:34:44 -04:00

Report benchmark latency and allocation metrics 7a3e7a5775

ober added 1 commit 2026-09-14 15:36:33 -04:00

Verify extracted DTrace result archives 23efeb5b44

ober added 1 commit 2026-09-14 15:40:37 -04:00

Honor benchmark workload overrides 274ab5b3d4

ober added 1 commit 2026-09-14 15:42:08 -04:00

Exercise rollback phases in commit ledger 6b3e92f295

ober added 1 commit 2026-09-14 15:56:14 -04:00

Complete streaming and scaling benchmark coverage d533e58c73

ober added 1 commit 2026-09-14 16:00:17 -04:00

Wire semantic spans to optional DTrace 533e00df93

ober added 1 commit 2026-09-14 16:04:14 -04:00

Include CLI streaming in required verification 5e390649ec

ober added 1 commit 2026-09-14 16:07:21 -04:00

Verify committed mutation effects in ledger 8a3a1ebddc

ober added 1 commit 2026-09-14 16:18:11 -04:00

Cover persistence boundary failures 755ecf1ff5

ober added 1 commit 2026-09-14 16:29:09 -04:00

Add controlled validation harness 5ee49808b0

ober added 1 commit 2026-09-14 16:30:05 -04:00

Allow required CI to finish on FreeBSD ae3d45be25

ober added 1 commit 2026-09-14 16:33:22 -04:00

Configure Jerboa builder in CI f5c5e4716f

ober added 1 commit 2026-09-14 16:42:03 -04:00

Install SQLite oracle headers in CI e02eb3d597

ober added 1 commit 2026-09-14 16:47:33 -04:00

Pass FreeBSD SQLite oracle flags d1aba14ea1

ober added 1 commit 2026-09-14 16:49:18 -04:00

Avoid unsupported CI environment files c60672351e

ober added 1 commit 2026-09-14 16:52:11 -04:00

Use pinned SQLite amalgamation for oracle 053e33ae9b

ober added 1 commit 2026-09-14 17:00:49 -04:00

Match CI oracle extension configuration 217b1c17d9

ober added 1 commit 2026-09-14 17:10:08 -04:00

Exercise durable fault hook during direct writes 5d3e116098

ober added 1 commit 2026-09-14 17:17:32 -04:00

Rebuild CI oracle from a clean checkout 6c77a9853c

ober added 1 commit 2026-09-14 17:19:05 -04:00

Allow pinned SQLite oracle warnings 46394282df

ober added 1 commit 2026-09-14 17:21:22 -04:00

Force rebuild of the test oracle 7c2bec164d

ober closed this pull request 2026-09-14 17:24:41 -04:00

Some checks failed

Hide all checks

version-policy / required (pull_request) Successful in 3m48s

Required

Details

required-ci / required (pull_request) Failing after 3m58s

Required

Details

Pull request closed

Please reopen this pull request to perform a merge.

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ober/jerboa-sqlite!13

No description provided.