Add exact typed decimal aggregate states #44

Merged

ober merged 7 commits from feat/typed-decimal-aggregates into main 2026-09-19 18:00:36 -04:00

Conversation 0 Commits 7 Files changed 13 +309 -31

ober commented 2026-09-15 01:01:36 -04:00

Owner

Copy link

Summary

• implement exact scaled-integer state for ordinary SUM(DECIMAL) and AVG(DECIMAL)
• return DECIMAL(38,input_scale) from decimal SUM while ordinary AVG finalizes once to DOUBLE
• add the pinned decimal_average(DECIMAL) aggregate, preserving input width/scale with signed half-to-even rounding
• enforce signed HUGEINT intermediate bounds with explicit overflow errors
• compose dictionary-vector and group/filter selections instead of rejecting filtered aggregate inputs
• cover grouped, filtered, DISTINCT, prepared, NULL/empty, negative tie, overflow, and multi-chunk execution
• advance the package and CLI version to 0.21.0

This lands parity-ledger reproductions R08 and R09 as a typed decimal aggregate slice. The broader aggregate family, parallel state combination, and spill-aware integration remain explicitly outstanding.

The 0.21.0 version is ordered after open PR #40 (#40), PR #41 (#41), PR #42 (#42), and PR #43 (#43).

Pinned-source evidence

• DuckDB source pin: 117e1a46be1c903c5a36ee3c881c125597f93c60
• decimal_avg.cpp defines exact scaled-integer sum/count state, signed HUGEINT overflow, input-type-preserving output, and half-to-even rounding
• pinned decimal aggregate fixtures require SUM output widened to DECIMAL(38,scale)

Verification

• all changed .ss files pass delimiter balance checks
• changed modules compile under jerboa_verify; only established lexical pre-scan warnings remain
• focused typed-decimal, kernels, engine, and planner tests pass
• make build
• make unit
• make verify (653 local SQLLogic checks and 62 upstream subset checks; stress, native analysis, and docs all pass)
• make binary
• make binary-smoke
• changed-line security scan: no findings
• git diff --check

Review notes

• Aggregate selection composition is required for correctness whenever a filtered scan yields dictionary vectors; this fixes the prior aggregate input has unexpected selection failure for all aggregate kinds.
• SQL DECIMAL values are accumulated by their stored scaled integers, avoiding per-row floating conversion.
• Reusable exact-decimal and nested-selection lessons are recorded in the repository cookbook and anti-pattern data.

## Summary - implement exact scaled-integer state for ordinary `SUM(DECIMAL)` and `AVG(DECIMAL)` - return `DECIMAL(38,input_scale)` from decimal SUM while ordinary AVG finalizes once to DOUBLE - add the pinned `decimal_average(DECIMAL)` aggregate, preserving input width/scale with signed half-to-even rounding - enforce signed HUGEINT intermediate bounds with explicit overflow errors - compose dictionary-vector and group/filter selections instead of rejecting filtered aggregate inputs - cover grouped, filtered, DISTINCT, prepared, NULL/empty, negative tie, overflow, and multi-chunk execution - advance the package and CLI version to 0.21.0 This lands parity-ledger reproductions R08 and R09 as a typed decimal aggregate slice. The broader aggregate family, parallel state combination, and spill-aware integration remain explicitly outstanding. The 0.21.0 version is ordered after open PR #40 (https://git.jerboa.sh/ober/jerboa-duckdb/pulls/40), PR #41 (https://git.jerboa.sh/ober/jerboa-duckdb/pulls/41), PR #42 (https://git.jerboa.sh/ober/jerboa-duckdb/pulls/42), and PR #43 (https://git.jerboa.sh/ober/jerboa-duckdb/pulls/43). ## Pinned-source evidence - DuckDB source pin: `117e1a46be1c903c5a36ee3c881c125597f93c60` - `decimal_avg.cpp` defines exact scaled-integer sum/count state, signed HUGEINT overflow, input-type-preserving output, and half-to-even rounding - pinned decimal aggregate fixtures require SUM output widened to `DECIMAL(38,scale)` ## Verification - all changed `.ss` files pass delimiter balance checks - changed modules compile under `jerboa_verify`; only established lexical pre-scan warnings remain - focused typed-decimal, kernels, engine, and planner tests pass - `make build` - `make unit` - `make verify` (653 local SQLLogic checks and 62 upstream subset checks; stress, native analysis, and docs all pass) - `make binary` - `make binary-smoke` - changed-line security scan: no findings - `git diff --check` ## Review notes - Aggregate selection composition is required for correctness whenever a filtered scan yields dictionary vectors; this fixes the prior `aggregate input has unexpected selection` failure for all aggregate kinds. - SQL DECIMAL values are accumulated by their stored scaled integers, avoiding per-row floating conversion. - Reusable exact-decimal and nested-selection lessons are recorded in the repository cookbook and anti-pattern data.

ober added 1 commit 2026-09-15 01:01:37 -04:00

feat: add exact typed decimal aggregates 96ef8d24f1

ober added 1 commit 2026-09-15 01:02:26 -04:00

docs: record typed aggregate review state f9c63b9140

ober added 9 commits 2026-09-19 16:34:52 -04:00

feat: stream CSV copy with reader options 4a1d1f8de1

test: update CLI smoke version 91287762b3

test: cover WAL and checkpoint crash boundaries a0d0fb996d

feat: prune persisted row groups with zone maps b4b1defc21

Merge main and resolve CSV feature conflicts 21b9c959e8

Retrigger CI after runner-side failure c1d1808916

Stack WAL recovery feature after CSV changes 8a3cf26794

Stack zone-map pruning after WAL recovery changes b1c353b4a1

Stack decimal aggregates after zone-map changes 0282a75bec

ober added 4 commits 2026-09-19 17:10:00 -04:00

Bootstrap pinned Jerboa toolchain in CI 6eb1db2e51

Merge remote-tracking branch 'origin/feat/csv-incremental-options' into feat/wal-checkpoint-faults 7feef13435

Merge remote-tracking branch 'origin/feat/wal-checkpoint-faults' into feat/lazy-zone-map-scan 19b4a5e54b

Merge remote-tracking branch 'origin/feat/lazy-zone-map-scan' into feat/typed-decimal-aggregates f4081a5ea6

ober added 4 commits 2026-09-19 17:14:34 -04:00

Make close-race assertion platform-neutral 03b9b03fe1

Merge remote-tracking branch 'origin/feat/csv-incremental-options' into feat/wal-checkpoint-faults 3d654e0b2f

Merge remote-tracking branch 'origin/feat/wal-checkpoint-faults' into feat/lazy-zone-map-scan 4438b69640

Merge remote-tracking branch 'origin/feat/lazy-zone-map-scan' into feat/typed-decimal-aggregates 7a7f03e1ec

ober added 4 commits 2026-09-19 17:32:13 -04:00

Make native race smoke self-contained 363975ce4e

Merge remote-tracking branch 'origin/feat/csv-incremental-options' into feat/wal-checkpoint-faults 5f3a053eaa

Merge remote-tracking branch 'origin/feat/wal-checkpoint-faults' into feat/lazy-zone-map-scan 44944ceb7f

Merge remote-tracking branch 'origin/feat/lazy-zone-map-scan' into feat/typed-decimal-aggregates 4e37d52e29

ober added 4 commits 2026-09-19 17:46:41 -04:00

Build complete FreeBSD standalone binaries 37bdd45520

Merge remote-tracking branch 'origin/feat/csv-incremental-options' into feat/wal-checkpoint-faults 6e752fa236

Merge remote-tracking branch 'origin/feat/wal-checkpoint-faults' into feat/lazy-zone-map-scan 447da45cad

Merge remote-tracking branch 'origin/feat/lazy-zone-map-scan' into feat/typed-decimal-aggregates 00eaae6fc7

ober merged commit e2326ecb93 into main 2026-09-19 18:00:36 -04:00

ober referenced this pull request from a commit 2026-09-19 18:00:39 -04:00

Merge pull request #44

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ober/jerboa-duckdb!44

No description provided.