api #

zod object schema with optional .meta({aliases}) on fields

args object from CLI parser (mri, minimist, etc.)

zod object schema with optional alias metadata

optional zod schema to extract aliases for short form preference

zod object schema with optional alias metadata

raw argument array (typically process.argv.slice(2))

async function expected to reject

optional regex to match against the error message

Add a benchmark task.

this Benchmark instance for chaining

Remove a benchmark task by name.

this Benchmark instance for chaining

Run all benchmark tasks.

Tasks execute in add() order. The first task runs against a colder runtime than subsequent ones (uncompiled JS, cold caches) — a property of in-process benchmarking that matters more when an early task has aggressive overrides like low warmup_iterations or min_iterations. If first-position bias is a concern, put a throwaway warm-up task first, or call run() twice and use the second result set.

Format results as an ASCII table with percentiles, min/max, and relative performance.

Format results as a Markdown table.

formatted markdown string

Format results as JSON.

JSON string

Get the benchmark results.

shallow copy of the results array (prevents external mutation)

Get results as a map for convenient lookup by task name.

fresh Map of task name to benchmark result (prevents external mutation)

Reset the benchmark results. Keeps tasks intact so benchmarks can be rerun.

this Benchmark instance for chaining

Clear everything (results and tasks).

this Benchmark instance for chaining

Get a quick text summary of the fastest task.

human-readable summary string

Check if the benchmark has been run and has results.

comparison options including regression threshold and staleness warning

comparison result from benchmark_baseline_compare

comparison result from benchmark_baseline_compare

optional task name to use as baseline for comparison (defaults to fastest)

optional task name to use as baseline for comparison (defaults to fastest)

first benchmark stats (or any object with required properties)

second benchmark stats (or any object with required properties)

function to warmup (sync or async)

if provided, use this instead of detecting

Minimum speedup ratio to consider a regression. For example, 1.05 means only flag regressions that are 5% or more slower. Default: 1.0 (any statistically significant slowdown is a regression)

Number of days after which to warn about stale baseline. Default: undefined (no staleness warning)

Minimum percentage difference to consider meaningful, as a ratio. Passed through to benchmark_stats_compare. See BenchmarkCompareOptions. Default: 0.10 (10%)

Coefficient of variation (std_dev / mean) at or above which a comparison is flagged with noise_warning: true. Doesn't affect bucketing — the task still routes to regressions / improvements / unchanged as the Welch math dictates — but tells the formatter (and any custom consumer) that the underlying measurement is noisy enough that a "significant" call should be read with skepticism. The default is calibrated for system-noise/thermal/background-load contamination, not microbenchmark floor effects; lower it (e.g. 0.15) for sub-microsecond functions where any cv signal matters, raise it for inherently noisy workloads where 0.3 fires on every run. Default: 0.30

Outlier ratio (fraction of iterations rejected by MAD outlier removal) at or above which the comparison is flagged with noise_warning: true, OR-gated with the cv check. Catches the case where the post-cleaning cv looks tight because outlier removal already deflated the spread — a third of the iterations being tail events is itself a noise signal, even if the surviving samples cluster cleanly. Set higher (e.g. 0.2) for benchmarks where outliers are expected (allocator-bound, async I/O), lower (e.g. 0.05) for tight CPU loops. Default: 0.10

Whether a baseline was found

Timestamp of the baseline

Git commit of the baseline

Age of the baseline in days

Whether the baseline is considered stale based on staleness_warning_days option

Node.js version recorded in the baseline (null if no baseline).

Node.js version of the process that produced the current results.

True if the baseline's node version differs from the current process. Informational only — surfaced in the comparison header so readers can apply the caveat across all task comparisons; does not affect per-task classification (Node bumps affect every task uniformly).

The metadata bag persisted in the baseline file, or null if the baseline didn't have one (or no baseline was found). Returned verbatim — fuz_util doesn't validate the shape or diff it against any "current run" metadata. If the consumer needs a diff, they pass options.metadata (current-run context) and walk the two records themselves.

Individual task comparisons for tasks where Welch's math is meaningful — i.e., budget unchanged between baseline and current. Methodology-changed tasks are NOT in here; they live in methodology_changed. Excluding them here keeps aggregate reads of c.comparison.faster / c.comparison.speedup_ratio / c.comparison.recommendation safe — those fields are still computed for methodology-changed rows but answer a counterfactual question (the math is correct over mismatched sample sizes) and would mislead consumers iterating this array.

Tasks that regressed (slower with statistical significance), sorted by effect size (largest first)

Tasks that improved (faster with statistical significance), sorted by effect size (largest first)

Tasks with no significant change

Tasks whose effective budget differs between baseline and current. Excluded from regressions/improvements/unchanged because the Welch comparison is contaminated by sample-size or warmup differences — the math is correct but answers a different question than the reader thinks. Re-save the baseline after intentional methodology changes to surface any genuine drift that was masked.

Tasks in current run but not in baseline

Tasks in baseline but not in current run

Directory to load baseline from (default: '.gro/benchmarks')

Directory to store baselines (default: '.gro/benchmarks')

Git commit hash (auto-detected if not provided)

Git branch name (auto-detected if not provided)

Opt-in passthrough for run-level context (corpus identity, dependency versions, binary sizes, hardware notes). Round-trips on _load and is accessible via BenchmarkBaselineComparisonResult.baseline_metadata, but is *not* interpreted by _compare — consumers decide what to do with mismatches. Keep values JSON-serializable; this is written verbatim to the baseline file.

Welch comparison of baseline vs. current means.

When the row is in methodology_changed, every field of this object is contaminated by sample-size or warmup differences — not just recommendation. The Welch math runs end-to-end and populates faster, speedup_ratio, significant, p_value, percent_difference, effect_size, effect_magnitude, ci_overlap, and recommendation, but it's comparing distributions produced under different methodologies. Treat the result as diagnostic ("how dramatic is the contamination?"), not authoritative.

The built-in formatters (benchmark_baseline_format, benchmark_baseline_format_json) never render comparison.* for methodology-changed rows for this reason — they show only the budget diff. Custom consumers reading this field directly should check methodology_changed first.

True if the effective time budget differs between baseline and current. When set, the task is routed to methodology_changed instead of regressions/improvements/unchanged, and the comparison field's contents become unreliable per the doc above.

True when measurement noise is high enough to undermine the significance call on this row. OR-gated across two signals: - max(baseline.cv, current.cv) >= noise_warning_cv_threshold where cv = std_dev_ns / mean_ns (post-outlier-removal). - `max(baseline.outlier_ratio, current.outlier_ratio) >= noise_warning_outlier_ratio_threshold`, since outlier removal deflates cv — a high outlier ratio is itself a noise signal even when the cleaned cv looks tight. Independent of methodology_changed. The Welch math still ran; this flag tells the reader to take comparison.significant with skepticism on this row. Cv is recomputed at comparison time from persisted mean_ns/std_dev_ns; outlier_ratio is persisted directly.

The larger of the two coefficients of variation across baseline and current, exposed so consumers can render the actual noise level. Recomputed at comparison time from persisted mean_ns and std_dev_ns — not a persisted field.

The larger of the two outlier ratios across baseline and current. Read directly from the persisted entries. Exposed so consumers can render the actual outlier rate alongside noise_warning.

Significance level for hypothesis testing (default: 0.05)

Minimum percentage difference to consider practically meaningful, as a ratio. Below this threshold, differences are classified as 'negligible' and significant is forced to false, regardless of p-value. This prevents the t-test's oversensitivity at large sample sizes from flagging system-level noise (thermal throttle, OS scheduler, cache pressure) as meaningful differences.

Effect magnitude thresholds scale from this value: negligible < min, small < min*3, medium < min*5, large >= min*5.

Default: 0.10 (10%).

Which benchmark is faster ('a', 'b', or 'equal' if difference is negligible)

How much faster the winner is (e.g., 1.5 means 1.5x faster)

Whether the difference is both statistically and practically significant

P-value from Welch's t-test (lower = more confident the difference is real)

Percentage difference between means as a ratio (0.05 = 5%, 1.0 = 100%)

Cohen's d effect size (informational — not used for classification)

Interpretation of practical significance based on percentage difference

Whether the 95% confidence intervals of the two means overlap.

Informational only — do not use this field to infer significance. Two means with overlapping 95% CIs can still differ significantly at p<0.05 (overlap of up to ~25% of CI width is consistent with significance); conversely, non-overlapping CIs are slightly *stronger* than the standard p<0.05 bar but the relationship is not strict. Use significant and p_value for classification. This field is exposed for consumers who want to render CIs side-by-side and need the visual overlap check.

Note: the underlying CIs are computed with a z-score (1.96) rather than the strict t-score, so at small n the CIs are slightly narrow — overlap is reported less often than a t-based CI would. Effect is ~2-3% at the n=30 floor after Bessel's correction on std_dev_ns.

Human-readable interpretation of the comparison

Target measurement duration per task in milliseconds. The loop runs at least min_iterations iterations *and* at least this long, with max_iterations and on_iteration abort() as hard ceilings. Default: 1000ms

Number of warmup iterations before actual measurements. Warmup helps stabilize JIT compilation and caches. Default: 10

Cooldown time between tasks in milliseconds. Lets the runtime settle GC and (partially) thermal state between tasks. Fixed regardless of the previous task's duration or allocation footprint — if a suite mixes heavy and light tasks, raise this so light tasks downstream don't run in the prior task's GC shadow. Default: 100ms

Minimum number of iterations to run. The loop continues past duration_ms if needed to reach this floor, so raising it in a slow suite extends wall-clock past duration_ms.

The default (30) is sized so the Welch's-t DOF approximation used by benchmark_stats_compare stays stable on the floor case. Lowering it below ~15 produces statistically unreliable significance calls on slow tasks that hit the floor exactly — the math still runs but comparison.significant becomes a coin flip on noisy outliers.

Default: 30

Maximum number of iterations to run. Prevents infinite loops on extremely fast functions, and also sizes the pre-allocated timings array — set this only as high as you expect to fill, since oversized caps waste memory and add GC pressure during measurement. Default: 100000

Custom timer to use for measurements. Default: timer_default (auto-detects environment)

Callback invoked after each iteration completes. Useful for triggering garbage collection, logging progress, early termination, or custom instrumentation.

Note: The callback time is NOT included in iteration measurements - it runs after the timing capture. However, frequent GC calls will slow overall benchmark execution time.

Callback invoked after each task completes. Useful for logging progress during long benchmark runs.

Whether to pretty-print (default: true)

Whether to include raw timings array (default: false, can be large)

Group results by category using filter functions.

Display name for the group

Optional description shown below the group name

Filter function to determine which results belong to this group

Task name to use as baseline for the "vs" column. When specified, ratios are computed against this task instead of the fastest. If the baseline task is not found in the group, falls back to "vs Best" with a warning.

Task name

Statistical analysis of the benchmark

Number of iterations executed

Total wall-clock time for the task (setup + warmup + measurement + teardown) in milliseconds

Raw timing data for each iteration in nanoseconds. Length equals iterations (the array is right-sized after measurement). Useful for custom statistical analysis, histogram generation, or exporting to external tools.

Effective per-task time budget after applying overrides to suite defaults. Persisted into baselines so benchmark_baseline_compare can detect methodology drift — a min_iterations bump between baseline and current shifts Welch's DOF and produces "regressions" that are sample-size artifacts, not real drift.

Mean (average) time in nanoseconds (over MAD-cleaned samples)

50th percentile (median) time in nanoseconds (over raw samples)

Standard deviation in nanoseconds (over MAD-cleaned samples)

Minimum time in nanoseconds (over MAD-cleaned samples)

Maximum time in nanoseconds (over raw samples)

75th percentile in nanoseconds (over raw samples)

90th percentile in nanoseconds (over raw samples)

95th percentile in nanoseconds (over raw samples)

99th percentile in nanoseconds (over raw samples)

Coefficient of variation (std_dev / mean)

95% confidence interval for the mean in nanoseconds

Array of detected outlier values in nanoseconds

Ratio of outliers to total samples

Number of valid samples after outlier removal (population for central tendency and min_ns)

Number of input samples before filtering invalid/outlier values

Operations per second (NS_PER_SEC / mean_ns)

Number of failed iterations (NaN, Infinity, or negative values)

Format stats as a human-readable string.

Name of the task (for display)

Function to benchmark (sync or async). Return values are ignored.

Optional setup function run before benchmarking this task. Not included in timing measurements.

Mutations to fn and name made here are honored by the measurement loop — useful for dynamic configuration that depends on async state resolved during setup.

Optional teardown function run after benchmarking this task. Not included in timing measurements.

If true, skip this task during benchmark runs. Useful for temporarily disabling tasks during development.

If true, run only this task (and other tasks marked only). Useful for focusing on specific tasks during development.

Hint for whether the function is sync or async. Auto-detected during warmup if not set; async: false skips per-iteration promise checking for sync functions, while async: true forces an await on every measurement iteration.

Setting async: true on a function that returns sync forces an unnecessary microtask per iteration — for sub-microsecond functions this adds measurable bias. Prefer leaving async undefined (auto- detected during warmup) unless the conditional-async hazard below applies.

Required for conditional-async fns — without async: true, a first call that happens to return synchronously locks in the sync code path and any later Promise returns leak as unhandled rejections.

Override the suite's duration_ms for this task only. Useful when one task is much slower than others and needs a longer (or shorter) measurement window than the suite default.

Override the suite's warmup_iterations for this task only. Slow tasks may want fewer warmup iterations to keep wall-clock reasonable; tight/complex functions may want more so TurboFan reaches steady state.

Override the suite's min_iterations for this task only. Raise this on slow tasks so percentile and CI math have enough samples. Wins over duration_ms (see the suite field) — a slow task with a raised floor extends wall-clock until the count is reached.

Prefer raising this over duration_ms when you need more samples: per-iteration noise (GC, scheduler, thermal) is a time-rate process, so a longer wall-clock window proportionally inflates exposure to rare tail events. Raising the sample floor fixes the statistical-power problem without that inflation.

Override the suite's max_iterations for this task only. Cap a fast task to a fixed sample count, or raise the ceiling on a slow task that would otherwise be limited by the suite default. Also sizes the per-task pre-allocation — avoid extreme caps that won't actually be filled. When this differs sharply across tasks in the same suite, the larger allocation can leak GC pressure into subsequent tasks (cooldown_ms is fixed regardless of prior task footprint).

the parsed Svelte AST root node

The source text of the test expression, or null for the final else branch.

The resolved static string value for this branch.

Nodes to execute.

Execute a node. Throw on failure.

Called after a node fails. For observability — the error is already recorded.

Called when a node is skipped (pre-skip or dependency failure).

Return true to skip a node without executing. Dependents still proceed.

Maximum concurrent executions. Default: Infinity.

Stop starting new nodes on first failure. Default: true.

Skip internal graph validation (caller already validated).

Whether all executed nodes succeeded.

Per-node results.

Number of nodes that completed successfully.

Number of nodes that failed.

Number of nodes that were skipped.

Total execution time in milliseconds.

Error message if any nodes failed.

Signal to send.

Timeout in ms before escalating to SIGKILL. Must be non-negative. Useful for processes that may ignore SIGTERM. A value of 0 triggers immediate SIGKILL escalation.

the original/current content

the new/desired content

maximum number of concurrent operations

optional AbortSignal to cancel processing