6 Commits

Author SHA1 Message Date
Tsai Yuan
966aa9496a
fix(benchmark): validate sandbox timing options (#7651)
* fix(benchmark): validate sandbox timing options

* fix(benchmark): guard infinite idle reaping

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-28 20:47:59 +08:00
xiaodu55
2483962484
fix(sandbox): classify speculative prewarm acquires by event state, not coarse clock ties (#7045)
* fix(sandbox): classify speculative prewarm acquires by event state, not coarse clock ties

* test(sandbox): pin the prewarm tie classification with a quantized monotonic clock
2026-10-25 13:30:13 +08:00
ShiYuan Xiong
75df1be6b7
fix(benchmark): reject negative idle timeouts before provider creation (#7363)
* fix(benchmark): reject negative idle timeout before provisioning

* test(benchmark): cover registered providers for idle timeout validation
2026-10-23 23:56:44 +08:00
ShiYuan Xiong
2254ec6ed5
feat(benchmark): add OpenSandbox provider coverage (#7059) 2026-10-19 07:45:59 +08:00
Totoro
16d36ccd21
feat(sandbox): add bounded AIO prewarming and lifecycle benchmarks (#6998)
* fix(benchmark): validate sandbox lifecycle and concurrent measurements

* feat(sandbox): prewarm likely AIO tool turns

(cherry picked from commit 3a1017ed032ed9331987bc3dc61f23dcdac4e905)

* fix(sandbox): bind bounded prewarming to execution leases

* docs: trim frontend guidance below its existing size limit

* fix(benchmark): separate failed lifecycle stage timings

* test(sandbox): measure prewarm overlap and unused resources through agent graphs

* fix(sandbox): finish lease and sampler cleanup after cancellation or errors

* docs(benchmark): distinguish initialization wait from model overlap

* docs(helm): align the configuration example with version 61

* fix(sandbox): preserve synchronous agent graphs when prewarming is enabled

* test(cargo): isolate native fixtures from inherited make jobserver flags

* fix(sandbox): require managed cleanup before prewarming

* fix(sandbox): log denied prewarming as an expected skip

* fix(benchmark): filter provider summary input records

---------

Co-authored-by: nothingyuancando <1579035512@qq.com>
2026-10-18 08:10:56 +08:00
Vanzeren
e01173d8b2
bench(checkpoint): production-shaped full/delta benchmark with configurable snapshot frequency (#4467)
* feat(checkpoint): production-shaped full/delta benchmark with configurable snapshot frequency

- Group benchmark scripts into per-family folders (checkpoint/, sandbox/)
- Extract shared benchmark infrastructure into checkpoint_bench_common.py
- Add checkpoint_delta_snapshot_frequency config (default 1000, process-frozen);
  freeze it in make_lead_agent and DeerFlowClient; key the state-schema
  adaptation cache by resolved frequency
- New bench_production.py: per-case child processes run N ainvoke turns through
  the real lead-agent graph (scripted deterministic model, real AsyncSqliteSaver),
  then measure GET /state + POST /history through the real Gateway route stack
  in one event loop (httpx ASGITransport), cold/warm accessor-cache split,
  cross-mode digest gates
- New summarize_production.py: delta/full ratios plus decision metrics
  (snapshot_write_spike, cache_effect_ms, checkpoint_write_share,
  auto-discovered history per-limit ratios)

* fix(checkpoint): address production benchmark review
2026-07-27 11:47:49 +08:00