Finite-limit review

HFF exposes finite resource limits as service defaults. Their current values and counting contracts are listed in the local backend configuration guide. This page tracks benchmark evidence for reviewing those defaults; it does not turn operation counts into latency guarantees.

Reproduce the selected boundary workloads

Run the benchmark with Go 1.27.1 and allocation reporting:

go test -count=5 -benchtime=100ms -run '^$' -bench '^BenchmarkDefaultLimitWorkloads$' -benchmem ./engine

The query-byte case pads a bare items predicate with whitespace to exactly the configured byte boundary, then adds one byte to verify rejection. It measures byte scanning rather than a dense syntax tree; the parser and compiler benchmarks separately cover conjunctions up to 1,000 terms. The input case validates a root object and list with exactly MaxInputValues counted values, then adds one value to verify rejection. Runtime cases include successful and aborted evaluations: a nested search over 600 items completes, identity-map creates 99,999 generated values within the 100,000-value limit, and separate cases verify MaxSteps and MaxGeneratedValues aborts. They check the expected result or typed limit failure. Query construction, record construction, filter building, and input validation are outside the runtime timed loops. These cases test selected boundaries and abort paths, not every limit or application workload.

One five-run baseline on Go 1.27.1, Darwin arm64, Apple M3 Max, with a 100 ms benchmark time per case produced these medians:

These measurements describe only the benchmark's records and queries. Compare them with representative service workloads on the target Go version and hardware before using them to justify a default. A benchmark result is not a wall-clock deadline or a performance promise. B/op reports cumulative allocation during a benchmark iteration, not peak or resident memory; the identity-map case's roughly 13 MB reflects that allocation churn.

Verify default depth boundaries

Run the engine boundary tests with the race detector:

go test -race ./engine -run '^TestDefaultDepthBoundaries$'

The tests accept and reject the exact configured boundaries for syntax depth, input depth, callback depth, and JSON parse/serialization depth, and assert the typed limit errors. They also verify that a deeply nested callback expression can hit the default syntax-depth limit before its separate callback-depth limit. The callback-depth cases raise syntax depth to isolate the callback limit. JSON serialization uses an input-depth allowance of 65 so the JSON depth limit is the one exercised. These are correctness checks, not latency or throughput measurements.

Measure selected depth-limit workloads

The engine benchmark measures completed work at the default syntax, callback, input, and JSON depth ceilings. The callback case raises MaxSyntaxDepth to 512 so it measures MaxCallbackDepth independently; under the normal syntax default, syntax depth can reject deeply nested callbacks first. The input case times Validate; the JSON cases use a previously validated Input and time MatchInput.

go test -count=5 -benchtime=100ms -run '^$' -bench '^BenchmarkDefaultDepthWorkloads$' -benchmem ./engine

One five-run baseline on Go 1.27.1, Darwin arm64, Apple M3 Max, with a 100 ms benchmark time per case produced these medians:

These cases verify completed operations at selected depth ceilings. Exact acceptance and rejection immediately around the defaults are checked by TestDefaultDepthBoundaries; neither benchmark is a latency guarantee or evidence that the defaults fit every workload.

Verify the compilation-work boundary

Run the compile-step boundary test with the race detector:

go test -race ./engine -run '^TestDefaultCompileBudgetForRepeatedStructuralChecks$'

The test compiles the same shallow, 3,999-byte query containing 1,000 repeated x=y predicates against schemas whose x and y values each contain a struct. With 400 integer fields in each operand, compilation completes within the default MaxCompileSteps. With 600 fields, it returns a span-bearing LimitError and no partial program. Raising only MaxCompileSteps to 2,000,000 lets that same query and schema compile. The query remains below the byte ceiling and the syntax is shallow, so this isolates semantic compile work from query-byte and syntax-depth limits. It is a correctness boundary check, not a timing measurement or evidence for tuning the default.

Measure a compile workload at the default step ceiling

Run the benchmark with Go 1.27.1 and allocation reporting:

go test -benchtime=3x -count=3 -run '^$' -bench '^BenchmarkDefaultCompileWorkCeiling$' -benchmem ./engine

The workload repeats x=y 1,000 times in a 3,999-byte, shallow query. It compares schemas with 496 and 497 integer fields in each operand and disables ListEquality, so compilation must perform the structural permission checks. The required compile-work boundary is calibrated outside the timed loop with the public limit: the 496-field case completes with 998,002 required steps under the default 1,000,000; the 497-field case requires 1,000,002 and returns a typed limit error.

One baseline on Go 1.27.1, Darwin arm64, Apple M3 Max, with three iterations per case and three benchmark runs produced these medians:

The high B/op is cumulative allocation for this deliberately repeated structural-check workload, not peak or resident memory. This one schema/query shape demonstrates behavior near the default compile-work ceiling; it does not establish that the default suits every service or query workload.

After reducing repeated field-slice copies and reusing the per-compilation visited map, the same benchmark produced these medians:

For this workload, required steps remained exactly 998,002 and 1,000,002. The optimization does not memoize structural checks or change compile-step accounting, and the public Type.Fields() copy behavior remains unchanged. This before/after comparison is specific to the benchmark's repeated structural checks; it does not predict gains for other compiler workloads.

Measure input admission at the default value ceiling

Run the benchmark with Go 1.27.1 and allocation reporting:

go test -benchtime=100ms -count=3 -run '^$' -bench '^BenchmarkDefaultInputValueCeiling$' -benchmem ./engine

The record has schema {items: list[int]} and the query items:exists, which does not inspect individual list elements. The benchmark compares exactly 100,000 values with 100,001 values; the root object and list count toward that total. Record construction, compilation, and filter building are outside the timed loop. Validate includes its result-compatibility check, while Match includes full input admission before evaluation.

One baseline on Go 1.27.1, Darwin arm64, Apple M3 Max, with a 100 ms benchmark time and three runs produced these medians:

This measures input-admission work for one flat list shape, not record construction, retained input memory, database or transport cost, or every service workload. It is evidence for this boundary only, not a universal budget-safety or latency claim.

Measure a generated-value workload at the default ceiling

Run the local matcher benchmark with Go 1.27.1 and allocation reporting:

go test -benchtime=100ms -count=3 -run '^$' -bench '^BenchmarkDefaultGeneratedValueCeiling$' -benchmem ./backends/local

The query items:flatmap(x=>[x,x]):size=2*n creates an intermediate three-value list for each input item, then a flattened two-entry result and its outer list. The benchmark compares 19,999 items (99,996 generated values, completed) with 20,000 items (100,001 values, rejected). Validation, compilation, and Build happen outside the timed loop; each iteration calls MatchInput on the previously validated input.

One baseline on Go 1.27.1, Darwin arm64, Apple M3 Max, with a 100 ms benchmark time and three runs produced these medians:

This is one flatmap workload. Allocation volume is cumulative B/op, not peak or resident memory. MaxGeneratedValues counts logical HFF values, not bytes or peak memory. This timing and allocation sample does not describe other queries or establish a universal limit-safety or latency guarantee.

Evidence and coverage matrix

The table separates what each limit counts from the workloads checked so far. Benchmark figures above are single-host samples; they describe those inputs, not generally acceptable service latency or a reason to change a default.

These checks establish counter behavior for the named cases. They do not prove that defaults are safe for every service, constrain SQL execution, or predict database planning, transport, peak memory, or caller-specific latency. SQL backends reject explicit execution-budget requirements they cannot enforce; local runtime limits do not become database limits.

Compare padded and dense queries at the byte ceiling

Run the compilation benchmark with Go 1.27.1 and allocation reporting:

go test -benchtime=100ms -count=3 -run '^$' -bench '^BenchmarkDefaultQueryByteCeiling$' -benchmem ./engine

The benchmark compiles against {x: bool}. It compares a 65,536-byte source containing one predicate padded with spaces with a same-size dense source of 32,768 whitespace-conjoined x predicates. Both compile under the defaults; a 65,537-byte source is rejected by byte admission before parsing. Engine and source setup are outside the timed loop.

One Go 1.27.1, Darwin arm64, Apple M3 Max baseline with a 100 ms benchmark time and three runs produced these medians:

Generated-parser scratch uses sync.Pool, so repeated-run allocation figures vary with retained scratch and garbage collection. B/op is cumulative allocation, not a fixed per-query memory budget or peak/retained memory. The rejection result measures this admission path, not general scanning throughput. This comparison isolates source density at one byte length; it does not establish a safe maximum query size for other schemas or services.

Default review outcome

Reviewed 2026-10-12: retain the current finite defaults for the shared HFF configuration. The intended baseline is a service evaluating human-authored filters against ordinary schemas. Existing boundary tests and selected workload measurements show that the configured limits accept and reject their specified cases; no representative service SLO or workload evidence supports changing the values. The measurements below are the reason to keep the current limits as a conservative common starting point, not proof of universal safety.

The benchmarks cover selected boundary shapes rather than a distribution of production filters, schemas, or records. They do not establish universal latency, peak-memory, or service-SLO guarantees. Applications should tune limits against their own input contracts and SLOs; future representative workload evidence may justify revisiting these defaults. No additional synthetic benchmark is required merely to retain the present shared defaults.