6 Commits

Author SHA1 Message Date
hataa
453ca834cf
chore(ci): refresh the shard duration baseline (#8264)
Regenerated on ubuntu-latest with the same runner image, Python, uv,
--extra postgres install and Postgres/Redis services as
backend-unit-tests.yml, via 'make test-shard-durations' against
main @ 085b52b5f.

- 41998 entries (was 31912): 31345 carried, 10653 added, 567 dropped
- selection matches the core shards: not live and not bundled_extension
- format unchanged (4-space JSON, sorted keys); content-only diff
2026-11-15 20:17:20 +08:00
Rei
9cb2bde557
test(ci): run bundled extension behavior suites outside the core shards (#8180)
Bundled examples keep their behavior tests in backend/tests/, so the core
backend workflow ran optional extension behavior on every pull request. Mark
those suites with pytest.mark.bundled_extension, exclude them from the core
shards and from the duration baseline, and run them in a dedicated workflow
through make test-bundled-extensions. Tests of host mechanisms stay in the
core shards even when an example package is the payload.

The marked suites import deerflow.agents, config, runtime, knowledge and
app.gateway, so the workflow triggers for backend/packages/**,
backend/app/gateway/** and the marked test modules; a backend test pins those
paths, the make target and both marker selections, and fails when a marked
module is missing from the path list. Host/extension compatibility checks — a
host API gate and authorization of contributed routes — live in
test_extension_host_contracts.py so they still fail core CI. The committed
duration baseline drops the 482 entries for the excluded suites.

Refs #8173

Co-authored-by: REI-330 <173389910+REI-330@users.noreply.github.com>
Co-authored-by: willem-bd <143703838+willem-bd@users.noreply.github.com>
Co-authored-by: Eilen6316 <136898293+Eilen6316@users.noreply.github.com>
2026-11-14 09:12:25 +08:00
Xinmin Zeng
44c6077801
refactor(provisioner): remove ignored legacy skills arguments (#7555)
* refactor(provisioner): remove ignored legacy skills arguments

* chore(provisioner): document legacy flag and align test durations
2026-10-26 17:01:51 +08:00
Dragon ze
33a3970c6b
chore(ci): refresh the shard duration baseline (#6963)
* chore(ci): refresh the shard duration baseline

backend/.test_durations was last regenerated on 2026-09-22 (61ae228b0, #5656)
and now covers 40.1% of the collected suite, so `least_duration` plans 19,415
tests from the imputed average instead of measured cost. Replaying CI's 4-way
split with the committed file yields shard loads of 203.5/217.4/234.3/222.9s
(1.15x heaviest-to-lightest); with this refresh the same replay is 219.6s each.

Regenerated with the documented target: `make test-shard-durations`, i.e.
`pytest -m "not live" --ignore=tests/blocking_io --store-durations
--clean-durations --durations-path=.test_durations tests/ -q` on main at
8d994cc1e. Data-only diff; the shard jobs keep reading the file unchanged.

* chore(ci): cover the four tests added by 6b60903d7

The checkpointer/postgres-schema helper suites gained 4 cases while this
refresh was in flight, which would have left the baseline 4 entries short of
the base tip. Re-ran just those two files through the same store path, so the
file now covers 32,396/32,396 collected tests with no gap. Re-measuring those
files also refreshed 113 neighbouring entries.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-16 07:10:06 +08:00
Roli Bosch
61ae228b0e
fix(skills): align Vercel deploy name and mounted script path (#5656)
* fix(skills): align Vercel deploy path

* test: align shard duration key with vercel-deploy rename

* docs: align demo transcript with vercel-deploy mount path

* test: keep duration baseline surgical after skill rename

* test(skills): align Vercel demo and strengthen path coverage

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-22 22:39:49 +08:00
spud
fe379c4486
feat(ci): split backend unit tests into parallel shards (#5137)
* feat(ci): split backend unit tests into parallel CI shards

Split the single offline backend `make test` job into four GitHub Actions
matrix shards (SPLITS=4, GROUP=1..4) via pytest-split, so the ~12k-test suite
runs in parallel instead of in one 15-minute job. Each shard runs on its own
runner with its own Postgres/Redis services; fail-fast: false lets a failing
shard report its owned tests without cancelling its peers.

`make test` stays the canonical full-suite entry point; CI now calls the new
`make test-shard SPLITS=4 GROUP=N`. tests/blocking_io remains owned solely by
the dedicated blocking-I/O workflow (excluded via --ignore), extending #5105.

Fixes #5088

* test(ci): make backend test shards duration-aware and pin the contract

Make `make test-shard` an explicit least_duration split that READS
backend/.test_durations (read-only for shards, so concurrent CI jobs never
race writes on it), and add `make test-shard-durations` to regenerate that
file from the full offline suite. Update the CI unit-test workflow contract to
call `make test-shard SPLITS=4 GROUP=<n>` and assert the shard command carries
--splits 4, --group 2, -m "not live", --ignore=tests/blocking_io and
--splitting-algorithm least_duration. Verified on the real 13,140-test normal
suite that the four shards are pairwise disjoint and their union equals the
unsplit suite.

Refs #5088

* test(ci): fail fast when the duration baseline is missing

`make test-shard` now requires backend/.test_durations and exits with a clear
error instead of letting pytest-split silently degrade to an even (count-based)
split. Harden the CI contract test to pin `--durations-path=.test_durations` and
to assert the repo ships the committed duration baseline.

Refs #5088

* docs: trim backend/AGENTS.md within guidance budget

* test(ci): add backend test duration baseline

Add the duration baseline generated by a full offline backend run on a
GitHub-hosted ubuntu-latest runner (the same runner type the shards use), so
`make test-shard` balances the four matrix shards by real wall-clock cost.

Refs #5088

* test(ci): make the duration writer honor DURATIONS_FILE

`test-shard-durations` now writes `--durations-path=$(DURATIONS_FILE)` instead of a
hard-coded .test_durations, so the reader and writer stay consistent when the
path is overridden.

Refs #5088

* test(ci): address sharding review feedback

* test: isolate subagent execution capacity state

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-09-02 11:54:40 +08:00