49 Commits

Author SHA1 Message Date
Miguel de Benito Delgado
a61c659221 ♻️ Share renderer benchmark error handling
Share construction of comparison storage failures and browser phase errors.
Preserve occurrence order, operation context, causes and stale propagation.

Use shared GPU setup guidance in command validation and runtime checks.
Document how supplied outcome fields override default errors, exit codes
and log paths.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
8eff426479 🐛 Make renderer benchmark diagnostics bounded and resilient
Retain build failure occurrences in failed manifests and preserve the
manifest-write error alongside the original preparation failure.

Bound diagnostic text, escape terminal controls, and keep Unicode pairs
intact. Validate optional traces and partial evidence before encoding, with
explicit diagnostics for unsupported or excessive data.

Preserve completed phase evidence and owner cleanup when bridge encoding
fails. Retain raw numeric observations, including NaN and infinities, and
keep the newest log-tail evidence after escaping.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
91a8bc8f1e 🐛 Bound process cleanup times and keep all failures
- Improve timeout handling, error status preservation and artefact
  saving after failures and interruptions
- Some refactoring for readability
- Improve tests

AI-assisted-by: gpt-6.1-sol, claude-5.5-sonnet, deepseek-flash
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
e16b7589ed ♻️ Refactor benchmarks server
Clean up, add tests

AI-assisted-by: claude-5.5-sonnet, deepseek-flash
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
e6fcbe5969 ✨ Collect full benchmark error traces
Keeps track of all errors in a vector.

AI-assisted-by: gpt-6.1-sol, deepseek-flash
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
5212d62369 🔧 Nicer render-bench launcher
As suggested by @elenatorro.
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
b96bf0e5c6 ✨ Improve failure reporting
- Each recorded failure gets one diagnostic entry.
- Distinct metadata events retain separate entries even when
  their messages match.
- Run and comparison reports expose screenshot warnings, files
  and skip reasons.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
d46db53d2f 🐛 Explain saved renderer A/B failures
Print compact A/B failure summaries from the last saved comparison when
the command returns status 1 with an empty error list. Include reasons
and run, build or comparison references, with bounded fields and an
eight-entry limit.

Read interruption state after execution and cover saved configuration
failures, unacknowledged differences and signal output in tests.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
6c7f8aec5c 🐛 Preserve renderer benchmark failure context
Retain saved-run input paths and roles in compare and merge errors.
Carry bounded build identities and process details through saved
 diagnostics and terminal reports. Preserve browser startup context
through failure wrappers and quote string paths for terminal output.

Cover file reads, compiler exits, catalog identities and browser startup
with behavioral tests across persistence and CLI reporting.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
f3549a4072 💄 Prefer shorter form lambdas in obvious places 2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
964b0da1ac ♻️ Consolidate error diagnostics
AI-assisted-by: gpt-6.1-sol, deepseek-flash
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
fbba049372 ✨ Complete renderer benchmark command catalogue
- Improve readme
- Migrate scene cases to measured commands and remove obsolete descriptors.
- Preserve exact camera targets, including very small positive scales.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
1820ed47b7 ✨ Enforce benchmark case deadlines
Stop failed cases and preserve their execution evidence.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
201eb11589 ✨ Save raw benchmark phases and reports
Record raw phases and report statistics per execution.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
483bd6e350 ✨ Integrate renderer benchmark case phases
- Run preparation, main and reset through the shared command executor.
- Refactor preparation into named operations and pure data projections.
- Fully document the browser bridge.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:07 +01:00
Miguel de Benito Delgado
5e0cc27150 ♻️ Reorganize benchmark runner, reporting and cli 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
9b995bab39 ✨ Add asynchronous renderer benchmark commands
Add private completion and failure callbacks to renderer commands. Preserve
renderer endpoints and raw metric traces across asynchronous delivery.

Supply the complete hook set through the runtime context and validate it
at command boundaries. Remove per-command hook dependency declarations.

Cover callback ownership, duplicate completion, and camera timing, and
teach clj-kondo about asynchronous command declarations.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
a6091637bb 📚 Improve readme, some docstrings, simplify 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
88c2132a5d ✨ Complete benchmarking cli redesign
AI-assisted-by: gpt-6.1-sol, deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
1156ced7d5 ✨ Add defcommand macro and expose render_sync
AI-assisted-by: gpt-6.1-sol, deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
a74ec5fb1a ♻️ CLI command execution refactor
AI-assisted-by: gpt-6.1-sol, deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
dee26e5b47 ♻️ Command specific parsing
AI-assisted-by: gpt-6.1-sol, deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
15bc266ee8 ♻️ Use per-command schemas in benchmark cli 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
d14432c010 ✨ Fix error msg in ab command 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
1fc898af54 ♻️ Syntactic sugar for the benchmarks
AI-assisted-by: deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
c7a2e29679 ♻️ Consolidate benchmark output dir creation 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
62619ea248 ✨ Implement run merging
AI-assisted-by: deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
2d5880e14d ♻️ Clean up some duplicated code 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
6db80536c4 📚 Add guide to the wasm renderer benchmarks 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
b26d28e9df 🔥 Delete some dead code 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
2973259fc5 ♻️ Consolidate case declaration macros and helpers
AI-assisted-by: deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
38e3325d85 ♻️ Consolidate build identity macros and helpers
AI-assisted-by: deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
6349cfb56d ♻️ Consolidate scene building macros and helpers
- Shared macro expansion for shapes and containers.
- Shared runtime construction for rectangles, paths, and circles.
- Centralized container attribute validation.
- Simplified imports in the seven callers.
- Added behaviour tests for seeded IDs, defaults, expression
  evaluation order, and container failures.

AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
9ae9f853bc ♻️ Consolidate error helpers in benchmark runners
AI-assisted-by: gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
af1d4f7680 🚧 WIP 29.5, refactor error handling
AI-assisted-by: claude-haiku-5.5
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
cf2bc75089 🚧 WIP 29.4, introduce error handling helpers 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
1c822901db 🚧 WIP 29.3, fix fingerprint generation 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
ec0b228abb 🚧 WIP 29.2, refactor feature comparisons 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
37a3a57af9 🚧 WIP 29.1, add options schema 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
928d312bf3 🚧 WIP 29, clean up run.cljs 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
b80b65cb64 🚧 WIP 28, enable GPU support 2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
b6e648817d 🚧 WIP 27, improve logging and console reporting
AI-assisted-by: deepseek-flash, gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
3b572be54c 🚧 WIP 26, A/B tests and a bunch of fixes
AI-assisted-by: deepseek-flash, gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
2a80b9dbfd 🚧 WIP 25.3 Clean up
- remove manual schema versioning
- clean docstrings

AI-assisted-by: deepseek-flash, gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
cd03d21c86 🚧 WIP 25.2 Safer deletion and symlink handling
AI-assisted-by: deepseek-flash, gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
c775521f3f 🚧 WIP 25.1 Clean up
More consistent args.
Avoid importing unnecessary namespaces.
Improve file write error handling.

AI-assisted-by: deepseek-flash, gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
d0f195899e 🚧 WIP 25 complete runner, drop pilot
AI-assisted-by: deepseek-flash, gpt-6.1-sol
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
6db7bceec9 ✨ Add retained renderer benchmark builds and host
Add prepared build artifacts, browser host and the build and
purge commands for the CLJS renderer benchmarks.

Each preparation attempt gets a build ID and a retained directory with
its manifest, log, Node case catalog, browser bundle, renderer files
and host page. Ready builds load their matching catalog in process and
serve only their own directory through an owned loopback host.

Own Chromium through a bounded process group, verify the compiled build
identity before execution and retain failed preparations for diagnosis.
Wire the diagnostic pilot to the same build, catalog, server and
browser APIs.

Add real boundary tests for artifacts, processes, servers, browsers and
CLI parsing, and register them with the performance test runner. Rename
the benchmark lint and test scripts to match the suite.

AI-assisted-by: deepseek-flash
2026-11-11 15:27:06 +01:00
Miguel de Benito Delgado
1720d6ecbc 🚧 WIP 24 Relocate all code, delete old benchmarks 2026-11-11 15:27:06 +01:00