`Paths.delete_thread_dir()` called `shutil.rmtree()` directly. Windows
refuses to unlink a file that carries the read-only attribute
(`WinError 5`) where POSIX only needs a writable parent, so one read-only
file under `user-data/` made the whole thread undeletable: `DELETE
/api/threads/{id}` answered 500 `Failed to delete local thread data`,
`rmtree` had already removed everything it walked before that entry, and
every later retry failed on the same file.
Retry at the call site: clear the read-only attribute tree-wide and call
`rmtree()` once more. Only `PermissionError` is retried, the retry still
raises when it cannot help, so a file another process holds open stays a
real error instead of becoming a silent skip. A `onexc` handler cannot do
this portably: POSIX `_rmtree_safe_fd()` catches the handler's own
`PermissionError`, re-dispatches it as a `scandir` error and leaves the
deletion to fail later as `ENOTEMPTY`.
Co-authored-by: Aling33 <along308@users.noreply.github.com>
* test(channels): assert the blocked-rename error portably on Windows
Rebased onto the channels-store refactor. The suite pins that a failed
atomic-rename save must surface the blocked-rename error, never the
cleanup error. Replacing a file over a non-empty directory is EISDIR
(errno 21) on POSIX but WinError 5 (errno 13, PermissionError) on
Windows, so the assertions now accept the platform's blocked-rename
error via a shared helper instead of hard-coding errno 21, which broke
on Windows dev hosts.
Verified by the maintainer on macOS (15/15 green on the POSIX arm) and
run here on Windows (15/15 green on the nt arm); POSIX regression
signal is unchanged.
* style: ruff format 0.15.12
---------
Co-authored-by: Jay0130-a <309071492+Jay0130-a@users.noreply.github.com>
* fix(subagents): preserve test summaries before evidence truncation
* fix(subagents): honor Node counters and cache harvested summaries
* fix(subagents): retain warning counts and Go case evidence
* fix(subagents): retain long pytest sibling summaries
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): make doctor.py --help print usage without running diagnostics
scripts/doctor.py never parsed its arguments: `python scripts/doctor.py
--help` printed no usage and ran the whole diagnostic suite (subprocess
probes, config resolution, provider checks) and exited 0; any unrecognized
flag was silently ignored the same way. Closes#7386.
- Parse arguments with argparse.ArgumentParser in the __main__ entry point:
--help prints usage and exits 0, unknown flags exit 2.
- The no-argument invocation used by `make doctor` is unchanged, and main()
stays directly callable from the existing unit tests (they run in-process
under pytest, whose argv must not reach argparse).
- Add backend/tests/test_doctor_script.py: offline subprocess cases for
--help (usage, no diagnostics run), an unknown flag (exit 2), and the
no-argument run (report footer present, exit 0 or 1).
* test(scripts): isolate doctor CLI subprocess checks
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The analyzer prints ensure_ascii=False JSON, but the parent pipes used
the ambient locale codec and the child inherited the ambient stdio
codec: non-ASCII skill facts crashed the review pass on cp1252
Windows and mojibake'd on cp936. Pin both sides to UTF-8.
Fixes#7387
* fix(readability): preserve <a> links when Readability.js silently degrades on Windows
On Windows, simple_json_from_html_string(use_readability=True) invokes the
Readability.js subprocess, which succeeds without error but returns the
unextracted document instead of the extracted article. The pure-Python
fallback (use_readability=False) also drops every <a> destination. The net
effect is that web_fetch returns link-free Markdown on Windows.
Fix: after extraction, check whether the output retained the <a> elements
that the resolved HTML carried. When they are all gone, fall back to
converting the resolved HTML directly with markdownify — which preserves
<a> as [text](href) — so links survive regardless of the Readability.js
outcome.
Measured: 17 of 22 test_web_fetch_relative_links.py failures resolved;
43/48 pass on Windows (5 remaining are pre-existing malformed-markup
extraction tests, unrelated to this change).
Closes#6459
* test(readability): add degraded-extraction and scope-pinning tests
* test(readability): detect unwanted article fallback
* test(readability): classify HTML body in authority-tag census
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(config): pin explicit app config path across get_app_config
Fixes#7086. reload_app_config(config_path) now records an
_app_config_source pin (mirroring _extensions_config_source) so
get_app_config and DeerFlowClient(config_path=...) keep following the
explicit file instead of ambient default resolution.
* fix(config): synchronize app config cache and absolute pins
Serialize singleton reads, reloads, resets and injections so a concurrent call cannot publish a config under another file's pin. Keep explicit paths absolute across working-directory changes while preserving ambient discovery and failed-reload behavior.
Add deterministic concurrency and relative-path regressions. Validation: 33,875 offline tests and 262 blocking-I/O tests passed; frontend check, backend formatting and agent guidance passed.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(uploads): make companion record removal best-effort
* style(uploads): format the new test and log the markdown-reference removal
* fix(uploads): report the orphaned record through the module that owns it
* fix(persistence): auto-create the Postgres database only when it is missing
init_engine retried bootstrap through CREATE DATABASE whenever the error
text contained "does not exist". Postgres words missing relations,
columns, functions and roles the same way, so a bootstrap that failed
inside an existing database (for example a migration reading a dropped
column) went on to CREATE DATABASE. Startup then failed with
DuplicateDatabaseError, or with "permission denied to create database"
for a role without CREATEDB, and the original error appeared nowhere in
the traceback.
Recognize a missing database by SQLSTATE 3D000 (invalid_catalog_name)
instead, following __cause__ for a wrapped copy, and re-raise every
other bootstrap error unchanged. A live PostgreSQL test covers both the
missing-database path and an existing database whose migration fails.
* docs(changelog): reference #7410 for the Postgres auto-create fix
* fix(persistence): walk orig, __cause__ and __context__ for the missing-database SQLSTATE
Match _is_unique_violation's chain walk, with its cycle guard, so a wrapper
that keeps the driver error only on `orig` or `__context__` still triggers
auto-create. Drop the live test's English message assertion; the SQLSTATE
check already proves the original error surfaced.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(uploads): never truncate an existing upload on same-name collision
Same-name uploads silently destroyed the earlier file on disk: the shared
write path (open_upload_file_no_symlink) opened the destination with
O_WRONLY|O_CREAT (no O_EXCL) and ftruncate(0), so re-uploading report.pdf
opened and emptied the report.pdf already stored for the thread, and two
concurrent same-name uploads could race on one inode. The Gateway
ingestion pipeline already refuses overwrites (staged no-overwrite link
commit), but the embedded client upload path (copy_upload_file_no_symlink)
and the channel ingress paths (write_upload_file_no_symlink) still went
through the truncating open.
Open every upload destination with O_EXCL. On FileExistsError, claim the
next free _N suffix against the directory's current contents via the
existing claim_unique_filename helper and retry — the same deterministic
behavior the Gateway pipeline documents (name_1.ext). The existing file is
never opened, so its bytes survive intact under any future naming policy.
Embedded-client and channel inbound-file responses now report the name the
bytes actually landed under instead of the requested name.
* test(uploads): cover concurrent same-name writers
* fix(uploads): skip known copy collisions before staging
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scheduler): reconcile running occurrences from terminal durable runs
A completion hook can fail after the durable run has already committed its terminal status (transient DB, observer, or notification error). Single-instance run_once only recovered expired launching claims, so the occurrence stayed running: a recurring schedule stopped firing and a once task never reached its terminal parent state until a restart.
Add a non-destructive ScheduledTaskRepository.reconcile_terminal_runs pass. It only selects running occurrences whose linked runs row is already terminal and whose scheduled metadata matches, then finalizes through the existing finalize_occurrence path with the run's status, error, and goal verdict. Live runs are never taken over or interrupted; multi-instance reconciliation already handles terminal runs through lease ownership. Single-instance run_once calls it after expired-launch recovery. No schema change.
Tests: a terminal success finalizes, a live run is left alone, error/timeout/interrupted map correctly, and a success with an unsatisfied goal verdict becomes unmet. Full scheduler suite: 1041 passed / 130 skipped; guidance 0 errors / 0 warnings; ruff clean.
* fix(scheduler): isolate terminal-run reconciliation failures per occurrence
A poisoned occurrence (the same observer or database failure that made the sweep necessary) ran in the same transaction as every other candidate, so one raised exception rolled back the whole pass and escaped run_once before admission and queue draining. Give each candidate its own session/transaction: a failure rolls back only that occurrence, logs its id, and the remaining candidates still commit. The service also catches a remaining pass-level failure so the rest of the poller runs.
Extend coverage: scheduled-trigger finalization (the motivating case), a second reconcile returning 0 with no state change, once-task completed projection, and a poisoned-observer candidate ordered before a healthy one whose success still commits. Full scheduler suite 1044 passed / 130 skipped.
---------
Co-authored-by: Tong-bit-art <Tong-bit-art@users.noreply.github.com>
The quote-delimiter tables only knew ASCII quotes, backticks and curly
quotes, so `已生成「/mnt/user-data/outputs/季度 报告.pdf」。` captured the
handle `/mnt/user-data/outputs/季度` (the path pattern stops at the space)
and `已生成『…』。` kept the closer inside `real_ref`; either handle is dead
on a later read.
Register `「」` and `『』` as wrappers: a paired closer stays inside a
filename, an unpaired one is trimmed from a bare token, and corner-trailed
URLs and refs are no longer cut.
The persistence regression (capture -> registry round-trip -> resolution ->
real file read with the tool result gone) is ported from #7376 by
@yangjj-iso, per review.
* feat(projects): search shelf documents by keyword
Add search_project_documents: two-tier name/summary then bounded content scan, case-insensitive substring matching with original-text offsets. Never converts on the fly; reports stop_reason, content_scanned, skipped reasons, and capped unscanned/truncated ids.
* fix(projects): derive unscanned count without paging after budget trip
Review feedback: once the content budget trips, stop paging list_active to enumerate unscanned ids — cap ids at the trip page and derive the honest total from one count_active query (tier 1 is exhaustive, so active - tier1 hits - processed = unscanned). Also add a tool-level isolation test pinning that search never crosses projects or users.
* fix(projects): scan the shelf over one stable snapshot, not offset pages
Review feedback: offset paging re-read the live shelf per page, so a concurrent upload/trash between pages could duplicate a match or silently skip one while still claiming complete coverage. Tier 1 now materializes the whole shelf in a single SELECT (one read snapshot) and tier 2 iterates those same rows; the budget-trip unscanned count is derived exactly from the snapshot with no extra query. Mid-scan upload/trash regression tests pin both failure modes.
* fix(sandbox): close pipe write fds once when Popen fails
The command runners closed the pipe write descriptors twice on the
spawn-failure path: the except block released all four pipe fds, then
the finally block closed the two write fds again. In a busy gateway
another thread can claim a freed descriptor number between the two
closes, so the second os.close silently closed an unrelated descriptor
(a concurrent command's capture pipe or an HTTP socket). Guard the
parent's write-end close on a successful spawn.
* fix(sandbox): release pipe fds on BaseException and scope the Windows runner test
Review follow-up on #7394:
- Catch BaseException in both command runners' spawn-failure cleanup so a
KeyboardInterrupt mid-spawn still releases each pipe fd exactly once;
the guarded finally keeps the no-double-close guarantee.
- Skip the _run_windows_command failure-path param on non-Windows
runners: it reads subprocess.CREATE_NEW_PROCESS_GROUP when building
the call, which raises AttributeError on POSIX and would fail Linux CI.
- Add a KeyboardInterrupt regression test (fd-per-fd close accounting).
---------
Co-authored-by: Zhangchenhao-111 <285911435+Zhangchenhao-111@users.noreply.github.com>
* fix(mcp): let ordinary users save bundled personal connections
The personal MCP policy read `server.cwd`, which McpServerConfigResponse
does not declare. A bundled launcher definition (DingTalk, WeCom,
HubSpot) never carries `cwd`, so for a non-administrator the check
raised AttributeError and every create, edit or enable of one returned
500. The web UI installs catalog connections with scope "user", so an
ordinary user could not install any of the three.
Read a supplied `cwd` from the model's extras instead. A bundled
launcher with a working directory still requires an administrator.
* docs(changelog): reference #7400 for the bundled personal MCP fix
* docs(changelog): say #7400 makes bundled personal installs work, not work again
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): bound run event type filters
* test(gateway): cover run event filter cardinality
* style(gateway): group run event filter limit with module constants
configure.py did not parse its arguments, so `--help` printed no usage and
ran the bootstrap as a side effect (creating config.yaml, .env and
frontend/.env in a fresh checkout), and any unrecognized flag was silently
ignored the same way. Parse arguments with argparse before touching the
filesystem: --help now prints usage and exits 0, unknown flags exit 2, and
the no-argument invocation used by `make configure` is unchanged.
Extend the regression tests from #7102 with two offline subprocess cases
covering both new behaviors.
Fixes#7382
* fix(stream): tolerate ~~~-fenced code and \-prefixed closer when decoding escaped newlines
_decode_escaped_newlines_outside_fences only recognised ``` fences, so replies
that used ~~~ fences (e.g. code blocks rendered by the model with
~~~python ...) kept literal backslash-n sequences escaped instead of decoding
the ones outside fences, and a fence whose closer was split as backslash+tilde
across chunks was misread mid-stream. Fixes#7262.
- detect the opening fence with (?:```|~~~) and remember which delimiter the
fence was opened with, so closing requires the same delimiter
- while inside a fence, hold back a closer that begins with a backslash so a
fence ending in backslash+tilde mid-stream is not misread; the held tail is
revisited on the next push and normalised on flush
- keep backtick hold semantics unchanged
* fix(mindie): preserve unclosed fences during newline decoding
Match one-shot decoding to streaming by choosing the earliest backtick or tilde opener and preserving the remainder when its matching closer is absent. Correct the string-return docstring and cover mixed delimiters across every two-chunk split and character-by-character streaming.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(examples): keep the extension example's tests passing as the contract grows
The README's package-test commands failed one test on main:
isinstance(registry, ExtensionRegistry) is False because the test's
FakeRegistry does not implement knowledge_provider(), which
extension-api 0.2.7 added to the runtime-checkable protocol. The same
assertion broke earlier when plugin() was added. Nothing in CI runs
these tests, so each addition to the contract broke them silently.
FakeRegistry now records knowledge_provider() and plugin() calls, and
the test asserts the example registers neither, as its README says.
Extension Package Check, which already runs when the contract changes,
now installs each bundled example that has a tests/ directory next to
the checkout's extension-api and runs the README's pytest and ruff
commands. A backend test pins that step and checks that every tested
example declares those tools in its dev extra.
The example README and the extensions guide now count nine
contribution kinds, including knowledge providers.
* docs(changelog): reference #7364 for the extension example test fix
* test(extensions): name the missing package-test step in the wiring pin
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
`render-config`, `secrets` and `nginx-conf` write `--out` through
`os.replace`, so an existing directory surfaced as a raw `PermissionError`
traceback, and a file ancestor as `OSError` from `mkdir`.
Validate the first existing ancestor once in `main()` before any writer runs,
and state the file contract in `--help` for all three commands.
* fix(ci): retarget label docs at triage.yml and guard label declarations
.github/labels.yml still described the auto-labeling setup as the three
workflows triage.yml replaced (pr-labeler.yml / pr-triage.yml /
issue-triage.yml), and its section headers pointed at the deleted
.github/labeler.yml and pr-triage.yml. A reader following those pointers
lands on 404s.
The same pair had an unguarded failure mode: triage.yml applies labels by
name, and GitHub rejects the whole addLabels batch with 422 when any name is
missing from the repository. Nothing checked that a label added to the
workflow also reached the declarative source of truth.
Point the comments at triage.yml and add scripts/check_label_declarations.py,
which fails when the workflow applies a label labels.yml does not declare.
* fix(ci): trace applied labels and cover the declaration guard
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Document that a failed write, fsync, or replace in
_publish_bytes_atomically leaves the previous file intact, and that
temporary removal must not replace that error, including interruptions.
Add site-specific regression tests for unlink and warning-log failures.
The cleanup_failed_atomic_write call itself already landed on main in #7200.
Co-authored-by: wannT-1 <200890097+wannT-1@users.noreply.github.com>
* fix(gateway): answer malformed run bodies with 4xx instead of 500
RunCreateRequest types config, context and input as free-form dicts, so
some bodies it accepts still failed during run admission with an
exception nothing mapped, and the run routes answered 500:
- a non-object config.configurable (build_run_config's dict.update, and
.get in the stateless thread selector)
- a non-object config.context (build_run_config's own ValueError)
- an assistant_id that does not normalize to letters, digits and hyphens
- a non-string agent_name in config.configurable, config.context or
context (_load_scope_agent_config)
- an AI message whose tool_calls is a string, object or list of strings
(AttributeError from LangChain's validator)
The request model now requires config.configurable and config.context to
be objects or null (422 invalid_run_config_section), and
build_run_config checks configurable as it already checked context.
start_run maps build_run_config's ValueError to 422, as it does for
validate_thread_id. Message conversion also maps AttributeError to its
existing 400, and a non-string agent_name gets the existing 422 for an
unresolvable agent. All of these fail before admission, so no run is
created.
* docs(changelog): reference #7362 for the malformed run body fix
* docs(changelog): note the configurable shapes #7362 now rejects
Unsafe filenames and unnamed parts were dropped silently: the response kept
`success: true` with an empty `files` list, so the #7122 frontend gate never
blocked a chat send whose attachment had been rejected (#7160).
Both drop arms now record the rejected name in `skipped_files`, and `success`
follows `bool(uploaded_files)`: an all-rejected batch fails, a mixed batch
succeeds and lists what was dropped.
* fix(skillscan): keep a live client handle through a branch that rebinds it later
The instance-client signal dropped every name a compound statement could bind
before walking any of its parts, so an outbound call that really happened first
disappeared when a later statement in that same body rebound the name
(issue #4296, case 1).
Dropping up front is what keeps a benign skill from being hard-blocked for the
parts that can *follow* a sibling's rebinding -- a `finally` after a handler
replaced the handle, an `else` after a loop body -- so those keep it. Parts that
cannot follow a sibling (an `if`/`elif`/`else` branch, the one `match` case that
runs, a `try` body, a loop body) are now walked with the enclosing handle intact
and apply their own bindings in source order, the rule the top level already
uses. Names the statement binds as it dispatches -- a loop target, an
`except ... as` name, a `match` capture -- stay dropped before every part.
* fix(skillscan): walk a compound header before dropping its dispatch bindings
A loop target, an `except ... as` name and a `match` capture bind only after
that header has been evaluated, so dropping them first hid a real sink written
in the header itself: `for s in s.post(...)` with `s` a live handle reported
nothing, while the same call under a differently named target was found. Move
the drop to after the header loop and before the parts, which is all
`_branching_parts` needs; the body still starts with those names gone, so the
loop-value case stays unreported.
Raised in review on #7128. Pins the two header shapes, the inverse that keeps
the body conservative, and the three paths the review called out as unpinned:
the mutually-exclusive two-level `if`/`elif` case, a `while` body, and a `try`
`else` after a body rebind.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(plugins): add sectioned long-form report writing example
A standalone example extension whose save_section model tool stores a
万字-scale report section by section (SQLite, per-user and per-thread)
and assembles every section into one markdown document on a final
section_index=999 call. The assembled document is returned as a tool
artifact with only a short status line re-entering the model context, so
the full report never consumes the context budget.
Assembly drops a section's leading heading when it restates the section
title (models habitually open each section with '# <title>' while the
assembler already emits '## <title>'), which otherwise doubles every
heading in the document. The outline section passes through unwrapped
and keeps its own heading as the document title.
* fix(frontend): register report sections in extension catalog
* docs(report-sections): correct the test command path
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
`shell-curl-pipe-shell` hard-codes `bash|zsh|dash|fish|sh` in the download-pipe
matcher while the structural `env` fallback reads `_SHELL_NAMES`. The two lists
drift: `curl ... | ksh` (and csh/tcsh/ash/mksh) pipes a fetched script in
silence, even though sandbox_audit_middleware already treats ksh as a shell.
Derive the alternation from `_SHELL_NAMES` and widen both to the same ten
names, so a shell added once is visible to both detectors.
* fix(extensions): keep atomic-write cleanup from replacing the save error
Extension snapshot rollback, the managed plugins rewrite, and skill-archive
upload copies now use cleanup_failed_atomic_write so a failed unlink or
warning log cannot mask the original publication error.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
* fix(skills): keep upload and storage cleanup from replacing the save error
Route the skill-archive upload discard, custom-skill file writes, and
per-user skill-state writes through cleanup_failed_atomic_write. Close the
extension rollback descriptor on fdopen failure, and record the plugins
temporary path before the write so a short write is still removed.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
* fix(skills): keep form close and snapshot delete from masking the save error
Discard the uploaded skill archive before closing the multipart form, and
suppress close errors so a flush failure cannot replace the install outcome.
Snapshot rollback of a missing original now uses cleanup_failed_atomic_write,
and skill-state writes clean up on BaseException including KeyboardInterrupt.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
* test(extensions): assert logging cleanup cannot replace the delete-branch error
The log-handler check only ran for the unlink parameter, where the handler
is never patched. Move it under the logging failure so a handler exception
must stay out of the surfaced install error.
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
---------
Co-authored-by: wannT-1 <200890097+wannT-1@users.noreply.github.com>
Co-authored-by: wannT-1 <wannT-1@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: honor checkpoint object in thread state updates
Prefer the checkpoint object over the deprecated checkpoint_id, validate
its thread_id, and preserve checkpoint namespace when updating state.
Add regression coverage for checkpoint-object forks and thread mismatch.
* fix: preserve client context handling in checkpoint PR
* fix: support state updates in checkpoint namespaces
* fix: reject unknown checkpoint namespaces
* fix(gateway): isolate subgraph titles from thread metadata
Only synchronize state titles into conversation metadata for the root checkpoint namespace. Cover persisted child updates with and without checkpoint IDs in full and delta modes, and retain root title synchronization and branch sequence cleanup.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
