010. Restore selected metadata subtrees in structured JSONB bindings

• Status: Measured
• Date: 2026-10-13

Problem and mechanism

With JSONB type metadata attached to a document inside a flattened struct or relation item, a fixed nested field read previously restored the complete bound document before selecting the field. The candidate recognizes literal field chains that cross those physical bindings and extracts the selected JSONB subtree before restoring its numeric type and value. It carries parent presence and null state through the projection and keeps metadata pointers relative to the original document. It adds no binding API and does not validate stored JSON or metadata; their consistency remains the caller's responsibility.

Direct metadata-backed BindJSON already supported selected-subtree restoration. This change extends that lowering to eligible flattened and relation-item paths. Whole-document observations and chains whose root or access depends on a guard, computed value, or dynamic index retain the general restoration path. The most-specific physical binding remains authoritative.

Applicability

The operation preserves the existing parent missing/null/no-value handling, callback-scope isolation, and caller-owned version-1 JSON Pointer metadata contract. It does not inspect or verify stored data during Build.

Reproduction

The baseline is 3ab3913. The candidate was measured as source snapshots:

The placement benchmark harness was unchanged between baseline and candidate:

The six Build cases crossed Direct, Flattened, and Relation placements with a whole-document :type="object" query and a nested .nested.price>10.5 query. Database execution used 100- and 1,000-row fixtures. The relation fixture had one child row per parent and an index on parent_id. Each row contained a Float price, five Float list items, and unused nested payload fields; null documents and negative zero were included. The nested query was rooted at doc, profile.doc, or the relation callback's x.doc, respectively. Each execution sample checked exact result IDs against local HFF matching before measuring the PostgreSQL plan. The correctness matrix included relation items with both a directly bound JSONB document and a flattened struct item; the relation benchmark used the flattened struct item.

The host was Darwin 25.2.0, arm64, Apple M3 Max with 64 GiB RAM. Go was 1.27.1 with default GOMAXPROCS=16. PostgreSQL was 18.6 in a Debian aarch64 container using image digest sha256:5a5a84b19854a9ffaa54082c166ff4ec27473a361e496e5ea167f298f2da9722. There were no explicit CPU, memory, or cpuset limits. JIT was off, parallel workers were disabled, custom plans were forced, the statement timeout was 60 seconds, and the database directory used tmpfs.

Build command:

GOTOOLCHAIN=go1.27.1 go test ./backends/sql -run '^$' \
  -bench '^BenchmarkPostgreSQLMetadataPlacementBuild$' \
  -benchmem -benchtime=200ms -count=3

PostgreSQL execution command (use separate plan directories for baseline and candidate):

HFF_POSTGRES_CONTAINER=<container-name-or-id> \
HFF_POSTGRES_PLAN_DIR=/private/tmp/hff-structured-subtree-plans \
GOTOOLCHAIN=go1.27.1 go test -tags=integration ./backends/sql -run '^$' \
  -bench '^BenchmarkPostgreSQLMetadataPlacementExecution$' \
  -benchtime=1x -count=3

Raw logs, SQL, plans, exact IDs, and the summary are outside Git in /private/tmp/hff-structured-subtree-evidence/ on the measurement host. The tables use the final three-sample pair after an earlier exploratory pair.

Results

Build time is median ns/op with the three-sample min–max. B/op is cumulative allocation, not peak or retained memory. allocs/op, parameter count, and SQL bytes are reported separately.

The database timings below are median milliseconds from three single-execution samples, with min–max ranges. Planning and execution are separate PostgreSQL measurements; plan-node counts and SQL bytes are baseline → candidate. Matched rows are the unchanged expected count.

For 1,000-row nested-scalar cases, measured execution medians changed from 131.4 to 6.014 ms for Flattened and 132.7 to 6.694 ms for Relation. Direct BindJSON already had selected-subtree restoration; its nested-scalar execution control measured 5.406 to 5.835 ms. Whole-document controls and direct-binding SQL were unchanged. These measurements are specific to these fixtures, container, and PostgreSQL version; they do not predict application latency or guarantee a plan for other data.

Correctness and tradeoffs

The full PostgreSQL integration suite passed in 312.940 seconds. It included 492 comparisons in the new structured-subtree matrix, covering parent state, key and pointer paths, guards, captures, shadowing, and result-type observations. Correctness tests cover relation items with both directly bound JSONB documents and flattened struct items; the relation performance fixture uses flattened struct items. SQL/local result parity also passed for each benchmark sample. A binding-configuration test rejects overlapping ancestor/descendant mappings. The full repository check and integration-tagged vet passed.

Build timings increased on most controls; the candidate does not establish a general Build-time improvement. For eligible nested paths, SQL became 121 bytes shorter and the final plan used four fewer nodes in these fixtures, at the cost of one additional parameter. No retained-memory or peak-memory measurement was made. Metadata restoration still assumes the caller keeps stored documents and metadata consistent.

Decision and limits

Retain selected-subtree restoration for fixed literal field chains within flattened and relation-item JSONB bindings. Whole-document queries and guarded, computed, or dynamic-index access continue to use the general path. The measurement does not add a per-attribute metadata API and does not establish a benefit for other query shapes, databases, row counts, or document sizes.