* fix(skillscan): read Python secret assignments from the AST
The `secret-env-assignment` rule swept every text file with a
`name[:=]value` regex, which misreads Python syntax in two ways:
- `def __init__(self, token: Optional[str] = None):` — the captured
"value" is a type annotation, not embedded secret material.
- `api_key = os.getenv("MINIMAX_API_KEY")` — reading a secret from the
environment is this rule's own documented remediation, yet it was
reported as a hardcoded credential.
Both are HIGH severity, so they map to a review `error` and fail the
Skill Review gate. Two bundled public skills therefore failed CI on an
unchanged checkout:
- skills/public/github-deep-research/scripts/github_api.py:56
- skills/public/music-generation/scripts/generate.py:27
Python sources now go through the AST instead of the line-oriented
sweep, keeping only real literal values. The text sweep is unchanged for
config, shell, YAML, and Markdown. Annotated assignments are still
reported, and now at the literal rather than at the annotation.
Tests: `secret-env-assignment` previously had no coverage anywhere in
backend/tests. Added six tests, including a regression test that scans
every bundled public skill script. Verified red on main and green here.
* fix(skillscan): keep text coverage for unparseable Python
Reviewer feedback on #5648: when `ast.parse` failed, the rule returned no
findings at all. One syntax error -- or a NUL byte, which `ast.parse` rejects
with the same exception -- therefore silenced the HIGH-severity
`secret-env-assignment` rule for the whole file, where `main` still swept the
raw text and reported it. For a review-gate rule that is a trivial evasion.
The line-oriented sweep moves into `_scan_secret_assignments_by_text`, which
the non-Python path now calls and which `_scan_python_secret_assignments` falls
back to when the file will not parse. Parseable files keep the precise AST
semantics this change introduces; unparseable ones keep main-level coverage
instead of losing the rule entirely.
Tests: both fallback paths added (syntax error, NUL byte); both fail before this
commit and pass after.
* fix(sandbox): drop ignored directories from remote list_dir
Remote providers listed node_modules/.git and similar entries through the
shared remote_list_dir parser, while the local list_dir and the remote
glob/grep implementations all skip them via should_ignore_path. Apply the
same rule in the shared parser so remote listings match both, filtering
after the empty-output check so an all-ignored directory returns an empty
list instead of a missing-path error.
* fix(sandbox): apply ignore patterns relative to the listing root
should_ignore_path checked every component of the absolute entry path, so an
ancestor of the listing root whose name matches a pattern (`build`, `env`,
`logs`, …) hid the root's contents: `ls /srv/build/workspace` returned []
even though the directory had files. The local walk only filters descendants
of the requested root and still returns that file.
Match patterns against the path relative to `resolved` instead, keep the
requested root itself, and keep entries that cannot be placed relative to the
root (a symlinked root is printed resolved by `find -H`) rather than dropping
them. Regression tests cover an explicitly requested ignored root, ignored
ancestors outside the root, and both cases through the real find pipeline.
* fix(uploads): convert the bytes we wrote, not the name they landed under
Document conversion re-opened the upload by name after it was already
visible in the thread's uploads directory: the Gateway converted the
committed file_path, and DeerFlowClient converted the copy it had just
placed there. That directory is writable from local and AIO sandboxes,
so a process watching it can replace the name with a symlink in the
window between the upload landing and the converter opening it. The
converter then reads whatever host file the link points at and writes
that content back into the thread as the .md companion, which the
sandbox can read. Reproduced end to end on both paths with a real xlsx:
the companion came back holding the host file's rows.
The Gateway now duplicates the descriptor of the staged file before the
link-commit, copies those bytes into a private directory outside the
uploads tree, and converts there. A descriptor cannot be redirected by
replacing a name, so the conversion input is the content this request
wrote. The client converts the caller's own source file instead of the
copy in uploads; the source is the file the caller handed in, which the
sandbox cannot reach.
Both already wrote the companion without following a symlink, so only
the read side changes. The uploads directory still receives exactly the
same files.
* docs(changelog): note upload conversion source fix (#5611)
* fix(uploads): close the conversion descriptor when staging its copy fails
Review follow-up. The private directory for the conversion copy was
created before the try that owns the duplicated descriptor, so a failure
there — a full or unwritable temporary filesystem — propagated without
closing it. The upload's own cleanup only unlinks the committed name and
releases the sandbox lease, so the descriptor stayed open for the life of
the process and kept the unlinked staged bytes allocated with it; repeated
failures accumulated both.
Directory creation now happens inside that try, and the finally removes
the directory only once it exists.
* fix(uploads): keep the conversion descriptor owned across cancellation
Review follow-up. run_file_io cannot interrupt its worker, so cancelling
the await around os.dup only abandoned the result: the duplicate was
created moments later with nothing left to close it, and it pinned the
staged bytes of an upload whose name the cleanup had already unlinked.
Cancellation after the duplication was just as leaky, because the
commit-path handler caught Exception and CancelledError is not one.
The duplication now runs as its own task, shielded from the caller's
cancellation, and closes its own result when the caller is gone by the
time the worker finishes. The commit path catches BaseException, closing
the descriptor it already owns before re-raising.
Both windows are pinned: one test stalls the duplication worker after it
allocates and cancels ingestion, the other stalls the commit so the
cancellation lands while the descriptor is owned.
* fix(uploads): drain the conversion copy so its descriptor always closes
Review follow-up. The copy worker owns the duplicated descriptor and
closes it in its own finally, but a bare await let a cancellation cancel
the executor job while it was still queued: the worker never ran, so that
finally never ran either, and the enclosing scope had already handed
ownership away and saw None. Draining also keeps a late worker from
writing into a private directory this scope has since removed.
The copy now goes through await_drained, the shield-and-drain helper the
Gateway already uses for offloads that must not be abandoned mid-flight.
Pinned by a test that holds the copy job queued, cancels ingestion, then
releases it and requires the descriptor to come back closed.
* fix(frontend): read web_fetch titles that start with blank lines or indented headings
* docs(frontend): describe the indented-code guard as it actually behaves
* fix(frontend): reject mixed code indentation in web-fetch titles
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(doctor): validate CLI credential contents
* fix(doctor): keep provider results when CLI auth state is undecodable
A credential file holding non-UTF-8 bytes made read_text raise
UnicodeDecodeError, which _load_json_object did not catch. check_llm_auth
wraps the whole model loop in one handler, so a single bad file replaced
every provider result with one generic failure and dropped the
provider-specific fix hints. Catch it alongside the malformed-state cases
and pin the behaviour with a two-model regression test.
Also cover the acceptance branches the change claims but did not test
(malformed JSON, non-object JSON, a directory as the auth path, blank
tokens, non-numeric expiresAt) and record credential_loader.py as the
source of truth for the mirrored rules, including the one place doctor is
deliberately stricter.
* fix(middlewares): end length-capped turns cleanly, prevent todo re-engagement, annotate write_file budget
When a model hits its per-response output cap (finish_reason=length) while
emitting a write_file tool call, ModelLengthFinishReasonMiddleware suppresses
the truncated call and stamps model_length_termination. TodoMiddleware must
not re-engage (jump_to=model) on such a capped turn -- doing so re-emits the
same oversized call into the same cap, producing up to 3 futile responses
with junk fragments instead of a clean truncation notice.
Changes:
- TodoMiddleware.after_model: skip completion reminder jump when
additional_kwargs.model_length_termination is present (follows the existing
deerflow_error_fallback precedent).
- ModelLengthFinishReasonMiddleware: always append the length notice when
tool calls were suppressed, even when partial text survived (collapses the
visible-content ternary). Fixes a latent bug in append_visible_text that
silently dropped string content.
- tools.get_available_tools: annotate write_file's model-visible description
with the model's configured max_tokens output budget. Guarded extraction
safely handles missing or non-numeric tokens, and the tool is cloned via
model_copy to keep module-level singletons immutable across assemblies and
prevent guidance leakage to unbudgeted models.
- release_policy_parameters() updated for both middlewares.
- AGENTS.md chain entries (#20, #35) and module docstrings updated within
AG002 guidance limits.
- Tests: 8 new/focused unit tests + 1 updated pin + 1 real create_agent()
integration test reproducing the incident (thread b1723286).
* fix(tools): use effective model cap for write_file guidance
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): stop E2B reconciliation from reviving warm-pool sandboxes
Periodic reconciliation probed every discovered remote sandbox with
Sandbox.connect() before the locality check, and the check itself only
consulted _sandboxes, not _warm_pool. A sandbox parked by release() was
therefore adopted back to active on the first pass, and because the SDK
normalizes connect(timeout=None) to its 300s default and the control
plane extends a running sandbox's expiry when now+timeout is later,
each 60s pass kept pushing the expiry forward — idle warm sandboxes
never hit their configured idle_timeout.
Treat _sandboxes and _warm_pool ids as locally tracked up front: skip
probing them (no timeout-mutating connect), keep them canonical, and
route only genuinely remote candidates through the duplicate-reap path.
Extend the post-probe adoption recheck to _warm_pool so a release that
lands mid-probe cannot be promoted back to active either.
Fixes#5550
* fix(sandbox): keep active E2B VMs alive and sweep expired warm entries
Address review on #5562:
- Reconciliation now refreshes the remote TTL of locally active
sandboxes through their cached client (never connect()), restoring
the keepalive for turns that outlive idle_timeout without reviving
warm-pool VMs.
- Warm-pool entries parked longer than idle_timeout are dropped during
reconciliation — their VMs are expected to be reaped by the control
plane — releasing the ownership lease and the capacity slot they
would otherwise pin until reclaim, eviction, or shutdown.
- Remove the now-dead thread-local canonical sort; locally tracked ids
are skipped unconditionally, so the ordering hint had no effect.
* fix(sandbox): preserve active E2B keepalive and shared capacity
* fix(sandbox): serialize E2B reconciliation lifecycle transitions
* fix(sandbox): fence E2B ownership and timeout lifecycle writes
* fix(sandbox): isolate ownership heartbeats from E2B timeout IO
---------
Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(mcp): re-scope to MCP task claim lifecycle only
Keep PR #4966 a small, closed MCP lease/cancellation state-machine change and
move RunJournal and Run lifecycle work into dedicated follow-ups. This branch
contains only the MCP task claim lifecycle:
- mcp task release/snapshot fencing by owner + per-claim lease token
- phase-level single-flight poll/cancel/notification owners with retained handoff
- routine cancellation no longer persisted as a task failure diagnostic
- bounded ordinary release ownership retention past the drain deadline
- 0018_mcp_task_lease_tokens migration + migration/bootstrap head assertions
- wait_for_task_until helper (MCP uses it); worker-specific capture helper moved
to the run-finalization follow-up
RunJournal (journal.py + test_run_journal.py) and run lifecycle
(manager/worker/store/run sql + run tests) are preserved on
backup/cancellation-safety-full and will be raised as separate follow-ups.
* fix(mcp): unblock claims after ambiguous handoff resolves
A phase-level single-flight owner only guards an ambiguous claim outcome. Once
the claim resolves, the phase owner is released immediately; the handoff may
continue releasing returned rows as bounded, service-owned background work
(transferred to _compensation_tasks on timeout). Per-claim token fencing rejects
a late release against a newer claim generation, so a stuck release no longer
locks the whole phase until process restart.
- README: drop the stale progress-snapshot sentence from the bounded ordinary
release description.
- service: pop the identity-checked phase owner as soon as the claim outcome is
known, then release returned rows with the bounded path; carry the release in
_compensation_tasks if it exceeds the drain deadline.
- mcp/AGENTS.md: document that only an unresolved claim outcome (not the handoff)
blocks later phase scans, and that returned-row releases may continue in the
background once the owner is released.
- tests: pin that the phase owner is released before a stuck release finishes
while the release stays service strong-owned.
* refactor(mcp): remove unused single-record claim wrappers
_poll_one, _cancel_one, and _notify_one are unreachable in production: the
worker always processes claimed records through _run_claimed_batch, so these
wrappers preserved a second, dead single-record lifecycle (state is None)
whose only observable behavior was a wrapper-specific cancellation release.
Remove the three wrappers and migrate the regressions that guarded their
cancel/release invariants to exercise the production _run_claimed_batch path
(operation=_*_one_claimed, release=_release_*_after_cancellation). The single
wrapper-only "state is None" contract (test_poll_release_hang_without_batch)
is deleted; all 11 remaining invariants (CancelledError preservation, repeated
cancellation, poll-only token-fenced lease release, notification claimed vs
dispatched phase release, hung compensation -> service ownership, and
background compensation exactly-once observation) are now covered through the
real batch lifecycle.
* fix(mcp): fence claim-owned mutations against stale generations
The per-claim token check in the ORM release/apply paths was only in the
SELECT; the final write went out by primary key. On SQLite (where
with_for_update() is a no-op) a mutation from an older claim generation
could therefore clear a claim that a newer generation had reclaimed after lease
expiry — the exact distributed lease-fencing failure the per-claim token was
meant to prevent.
Make every claim-owned mutation a single atomic conditional UPDATE with the
owner and per-claim token in the WHERE clause (rowcount 0 => stale, return
False, no mutation):
- release_claim: atomic fence; record the poll-failure event after the fence
wins (same transaction, holding the write lock).
- apply_snapshot / apply_cancel_snapshot: atomic fence; record the event after.
- finish_notification_run: atomic fence; use a CASE on event_version >>
dispatch_version to keep a newer event pending for redelivery instead of
swallowing it as delivered.
Add one regression per path: a stale generation's release/apply/finish after a
same-worker reclaim is rejected and never clears the newer claim.
* test(mcp): pin the migration chain head to the lease-token revision
0026_mcp_task_lease_tokens becomes the alembic head, so the chain-head pin in the 0025 repair test had to move on. Follow the 0023 precedent there (single head plus expected predecessor) instead of pinning a literal head, and give the new revision its own migration test, which owns the pin and covers the nullable claim-token columns on upgrade and their removal on downgrade.
* refactor(mcp): close cancellation cleanup leftovers
* fix(mcp): retain cancelled release diagnostics
* test(mcp): remove obsolete settled compensation case
* test(mcp): cover interleaved lease reclaim races
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The AIO sandbox's implicit persistent shell session hangs forever when a
command containing a bare 'exit' is executed: exit kills the session's
shell process, and the server's response path for that exec_command
request never completes. Verified against a standalone all-in-one-sandbox
1.11.0 container via the raw SDK:
shell.exec_command('seq 1 1000 | head -n 5') # OK (not a SIGPIPE issue)
shell.exec_command('echo x; exit 0') # HANGS every time
shell.exec_command('echo after') # OK (server recreates shell)
shell.exec_command('( echo x; exit 0 )') # OK, exit code propagates
remote_list_dir_command and remote_search_command both end their probe
scripts with a bare 'exit' (to propagate the find/grep status code), so
every list_dir/grep/glob call deterministically wedges the session —
this is the root cause behind parallel [ls, bash] tool calls deadlocking
an entire run (same defect family as #1433 and #5128).
Fix: keep 'set +e' as the outermost prefix (pinned by existing tests) and
wrap the rest of each probe script in a subshell, so exit only terminates
the subshell. Output and exit codes propagate identically.
Tests: 116 passed (backend/tests/test_aio_sandbox.py, test_remote_list_dir.py,
test_remote_search.py); one endswith assertion updated to the wrapped form.
Co-authored-by: mad_max <mad_max@coscoshipping.local>
* feat(authz): gate composer send on runs:create (Phase 4 follow-up, #4063)
Mirrors #5294's stop gating on the send side: both chat routes pass
canCreateRuns (from PERMISSIONS.RUNS_CREATE, lockstep with the backend
enum) into the shared composer. The gate sits at the top of
submitThreadMessage — the single choke point every composer entry
(submit button, Enter, goal-set-triggered run) funnels through — and
denies with a toast plus a rejected promise so PromptInput keeps the
text. The idle submit button is disabled and explains the boundary via
conditionally-spread aria-label/title (startTurnUnavailable, en/zh);
while streaming the button stays the runs:cancel stop affordance.
Also removes the unreachable kind === "stop" branch in handleSubmit
(flagged during #5294's review): the Enter path early-returns with the
streaming toast before the classifier runs.
* fix(frontend): reject denied goal starts before saving state
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): add WeChat QR login and binding recovery
* fix(channels): enforce single-worker WeChat QR login and preserve bot ID
Reject QR login endpoints when multiple Gateway workers are configured, while keeping manual token setup available.
Preserve the configured bot ID when the provider omits it or returns an empty value. Add regression coverage for worker guards and credential persistence.
* fix(channels): sync WeChat completion state on provider updates
Show the connected step when refreshed provider data confirms the binding, so the dialog no longer waits indefinitely after polling is cancelled.
Add regression tests for provider updates during pending poll and binding requests, including late responses and expiry.
* fix(channels): preserve WeChat pairing codes across waits and redirects
---------
Co-authored-by: YxinMiracle <“939157765@qq.com”>
* fix(memory): report non-mapping Honcho backend_config values as ValueError
failure_policy, workspace_overrides and user_peer_overrides were read with a
falsy-only `or {}` fallback, so a truthy non-mapping (a bare string, a YAML
list) reached .get/.items and escaped as a bare AttributeError from inside
backend construction. Route all three through one _mapping helper that keeps
falsy values meaning "unset" and names the offending key as a ValueError, the
posture the mem0 and OpenViking backends already take.
* fix(memory): name the Honcho numeric knobs that cannot be cast
timeout_seconds / connect_timeout_seconds / message_char_limit /
max_injection_chars still reached float() / int() with a YAML null or a
mapping, so the operator got a TypeError naming neither the key nor the
config file. Narrow all four through one _number helper that keeps falsy
values meaning "unset" the way the sibling api_key / storage_path scalars
already do, and reports a value that cannot be cast as a ValueError on the
key. Numeric strings keep parsing, since that is what float/int accept.
* fix(memory): report non-numeric backend_config knobs by name in mem0 and OpenViking
mem0 casts top_k, score_threshold, max_injection_chars and timeout_seconds, and
OpenViking casts timeout_seconds, max_seen_message_ids, retrieval.top_k and
retrieval.max_injection_chars, straight through int()/float(). A knob written
without a value in YAML therefore escapes as "TypeError: int() argument must be
a string..." from inside backend construction, naming neither which knob nor
which file is wrong, and a non-numeric value escapes as the equally anonymous
"could not convert string to float".
Both now resolve numeric knobs through a helper that keeps a value-less key at
its default, the way the same dicts already treat failure_policy and
allow_insecure_http, and turns an uncastable value into a ValueError that names
the knob. OpenViking's score_threshold keeps None as a meaningful value rather
than a default to fall back to.
* fix(client): stop embedded uploads from writing through symlinks
DeerFlowClient.upload_files copied each file with shutil.copy2 and let
convert_file_to_markdown write the companion straight into the uploads
directory. Local and AIO sandboxes can write to that directory, so a
symlink planted at an upload name or at the companion's name was
followed: the upload's bytes and the converted Markdown landed in
whatever host file the link pointed to, and the call reported success.
The Gateway refuses symlinked destinations and the IM channels write
through write_upload_file_no_symlink; the embedded client never adopted
either.
Uploads now go through copy_upload_file_no_symlink, a new helper next
to write_upload_file_no_symlink. It keeps copy2's content, permission
bits and timestamps, so files stay readable to Docker sandboxes, but
applies them to the descriptor opened with O_NOFOLLOW and opens the
source first so a missing source cannot truncate an existing upload.
As in the Gateway, a file with an unsafe destination is skipped and
listed in skipped_files, success turns false, and the message says how
many were skipped. The companion is converted inside a private temporary
directory and then written with write_upload_file_no_symlink; one whose
name is unsafe is left out like a failed conversion, and the original
upload is kept.
* docs(changelog): note embedded upload symlink fix (#5578)
* fix(client): keep copy2's same-file guard and companion permissions
Review follow-up. Two regressions in the previous commit.
copy_upload_file_no_symlink opened the destination before comparing it
with the source, and that open truncates. Passing a file that already
sits in the thread's uploads directory therefore copied an emptied file
over itself: the upload reported success with size 0 and the original
bytes were gone, where copy2 raised SameFileError and left the file
alone. The destination is now compared with the source through
os.path.samestat before anything is opened, so identity — including a
hardlink or another spelling of the same path — raises SameFileError as
before.
The Markdown companion was published with write_upload_file_no_symlink,
which creates a new file as 0600 and ignores the converted file's mode.
Under umask 022 the companion became 0600 while its own document stayed
0644, so a bind-mounted sandbox running as another uid could read the
upload but not the Markdown the response advertises. It now goes
through the same copy helper as the upload, which preserves the
converter's permission bits.
load_codex_cli_credential copied tokens.account_id straight into CodexCliCredential.account_id with no type check. A Codex auth file whose account_id is JSON null therefore propagated None into CodexChatModel._account_id, and model_post_init raised TypeError: 'NoneType' object is not subscriptable on the account prefix in its log line. A numeric or boolean account_id slipped through the same way, reaching the ChatGPT-Account-ID header as a non-string.
Treat any non-string account_id as absent and fall back to the credential's empty-string default, matching how a missing account_id already behaves.
* fix(gateway): classify Windows SVG MIME alias as active content
Treat Windows' image/svg alias like the standard image/svg+xml active content type.
* test(gateway): cover Windows SVG MIME alias
Pin image/svg classification independently of the host MIME database.
* docs(utils): document platform MIME aliases
Record the shared active-content classification invariant.
`_extract_claude_code_credential` copied `expiresAt` straight into
`ClaudeCodeCredential.expires_at`, so a credentials file whose `expiresAt` is a
string, null, list or object reached `is_expired` and raised
`TypeError: '<=' not supported between instances of 'str' and 'int'`. That
aborted the whole lookup instead of skipping the malformed source and moving on
down the documented order, the way the rest of the loader already behaves for a
malformed `claudeAiOauth` container.
Validate the field the way the sibling branches validate their input: log a
debug line and skip the source so the next candidate is tried.
* fix(sandbox): report an exactly-full search result as complete in the remote providers
`glob` and `grep` decide `truncated` twice: once for the raw output cap
(`parse_remote_search_output`, unchanged) and once for `max_results` after the
Python-side filters have run. The second decision returned as soon as
`max_results` matches had been collected, which cannot tell a search that held
exactly that many from one that held more — a tree holding exactly
`max_results` eligible matches came back flagged as cut off, and the tool then
told the model the result was incomplete.
These providers hold the whole listing (the raw stream is capped at
`max(max_results * 4, max_results + 50)` lines and reports its own cut-off), so
like AIO's `glob` branches they can look one match past the cap before
deciding: `AioSandbox.grep`, plus `glob`/`grep` in E2B, OpenSandbox, Tenki and
BoxLite now use the same `len(matches) > max_results` rule. This completes what
#5449 started for AIO's `glob`; the local provider's half is #5491.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(sandbox): let remote grep see one match past the per-file cap
E2B and OpenSandbox stopped each file's grep at max(max_results, 50)
matches, so a single file holding more than max_results hits — with a
raw stream far below its limit — ended the Python loop exactly at the
cap and reported the result as complete (#5534 review).
Retain one extra match per file so the one-match lookahead can observe
the overflow and report truncation. A single-file regression at
max_results=50 covers 50 matches (complete) vs 51 (truncated) for both
providers.
Co-Authored-By: Claude Code <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): rejoin active runs after reopening chats
* test(frontend): mock thread runs query in stream options test
* fix(frontend): avoid rejoining completed runs from stale cache
* fix(frontend): tighten active run recovery cleanup
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
load_codex_cli_credential called .get on the parsed ~/.codex/auth.json
(and $CODEX_AUTH_PATH) without checking that the top level is an object.
_load_json_file returns any valid JSON value, so an array or scalar payload
raised AttributeError out of CodexChatModel.model_post_init instead of the
documented 'Codex CLI credential not found' error. Guard the top level the
same way the sibling Claude loader and its own nested tokens guard do.
* fix(ragflow): batch validation for large document selections
* docs(ragflow): align documentation language with repository conventions
* docs(ragflow): preserve spacing before validation heading
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(middleware): add deterministic PII redaction for model-bound context
* fix(middleware): claim national IDs before cards, redact Command results, preserve ToolMessage fields
- Reorder detectors so checksum-gated national IDs run before the credit-card
detector; an 18-digit resident ID whose digit run also passes Luhn is no
longer mislabeled [CREDIT_CARD_n] (review finding, reproduced at 0a2a9d0)
- Redact ToolMessages carried in Command.update.messages, mirroring
ToolResultSanitizationMiddleware's dc_replace pattern
- Rebuild redacted ToolMessages via model_copy so artifact and
response_metadata survive
- Extend the numbered middleware chain in agents/middlewares/AGENTS.md
* fix(middleware): span one redactor per Command result; refresh stale AGENTS.md entry range
- Placeholder numbering now continues across every ToolMessage carried in a
single Command result (one _Redactor per _redact_result call) instead of
restarting per message
- The renumbered AGENTS.md chain still referenced entries 9-12 in the
ToolReceiptMiddleware entry; it now reads entries 10-13
* docs(agents): trim PiiRedactionMiddleware entry to fit the AGENTS.md chain budget
The main merge (fb36e0e) pushed the effective middlewares chain to 98341
bytes, 37 over the 98304 hard limit checked by agent-guidance (AG002).
Compress the entry while keeping the load-bearing facts: config gate, both
interception points incl. Command coverage, detector order rationale,
per-result numbering continuity, irreversibility, memory follow-up.
* fix(middleware): redact compaction input and reinjected summaries; harden detectors
Review round 3 on #5527:
- [P1] SummarizationMiddleware invokes its summary model directly from
before_model, outside PiiRedactionMiddleware's wrap_model_call, so raw
thread state reached the summary model and reinjected summaries carried
raw PII into model-bound context. Add a shared redact_text() seam: the
compaction prompt is redacted in _build_summary_prompt (app_config
already flows into the middleware) and DurableContextMiddleware redacts
summary_text at reinjection via a new pii_redaction_config knob wired
at both assembly sites.
- [P2] CUIT is 2+8+1 digits, not 2+10+1.
- [P2] Digit-anchored patterns use digit-aware lookarounds instead of
Unicode \b, which CJK characters defeat (身份证110105… / 手机号138…).
- [P2] The international-phone pattern no longer treats newlines as
separators, so a candidate cannot swallow the following numeric field
and then fail validation as a whole.
* fix(pii): redact title input and reserve summary placeholders
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(docker): add 'make prod-logs' entry point and hint when dev logs are empty
`make up` starts the production stack (deploy.sh: project `deer-flow`,
docker/docker-compose.yaml) while `make docker-logs` tails the dev stack
(project `deer-flow-dev`, docker-compose-dev.yaml), so after `make up`
it printed nothing (#5529), and no make entry point showed production
logs at all.
- scripts/docker.sh logs gains `--prod`: targets the stack deploy.sh
started, passes --env-file ../.env when present, and exports the same
interpolation defaults deploy.sh exports before every compose call —
without them the production volume specs fail to parse on checkouts
without .env.
- dev-only `logs` with no running containers now prints a hint pointing
at `make prod-logs` instead of staying silent.
- Makefile gains `prod-logs`, listed under Docker Production Commands.
New tests cover production targeting and the empty-state hint; they are
red on unfixed main and green here. Verified live: with a deer-flow
redis running, `logs --prod --redis` streams its logs.
Fixes#5529
* fix(docker): append --env-file after compose detection rebuilds COMPOSE_CMD
compose_preflight() probes the Compose binary and rebuilds COMPOSE_CMD,
so an --env-file appended before it was silently dropped. Append it
after preflight instead, and drive the regression test through the real
detection path (stub Docker Compose version v5.3.1, not require_compose_version)
so the append cannot regress silently.
Reviewed-in: #5538
* fix(utils): return empty text for content-less messages
message_content_to_text fell through to str(content), so a message whose
content is None yielded the truthy literal "None". Two call sites already
work around it with a local `or ""` and name the helper in the comment; the
subagent executor's `text if text else "No response generated"` fallback and
the archive's `if not text: continue` skip cannot work around it, so a
content-less terminal turn was reported as an answer of "None" and a
content-less LLM error fallback surfaced "None" instead of its error_detail.
* test(utils): cover non-None content compatibility and document fallbacks
* test(utils): cover contentless task history and refresh guard comments
---------
Co-authored-by: Lengshuang <90967079+Lesereingrape@users.noreply.github.com>
Co-authored-by: JasonH <4430962+yang0228@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(agents): close delegations a stopped run left in progress
Every task call is recorded in the delegation ledger as in_progress and
only moves on when its ToolMessage arrives. When the user stops a run
while a subagent is running, the task tool re-raises the cancellation
and no ToolMessage is written, so the entry stayed in_progress for the
rest of the thread and every later model call was told "already
delegated; do NOT delegate again; wait for or build on the result".
When a run starts with a new user message, mark entries that an earlier
run left in_progress and that have no ToolMessage as cancelled. Resumed
runs, which have no new user message, keep the current behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(agents): share the run-opening boundary between capture and closure
Also pin that in_progress entries without a run_id are never closed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(agents): clarify legacy delegation reply handling
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(knowledge): add verifiable RAGFlow source citations
* docs(knowledge): scope RAGFlow guidance to its own directory
* fix(knowledge): preserve citations through rendering and budgets
* fix(uploads): delete the requested upload, not a symlink's target
delete_file_safe resolved the requested path before unlinking it. The
uploads directory is writable from local and AIO sandboxes, so a
symlink planted under an upload name was followed: deleting alias.pdf
removed the victim.pdf it pointed to, and the companion cleanup then
removed victim.md, while the link itself survived and the call reported
"Deleted alias.pdf". A link resolving outside the directory was already
refused by the traversal check, so the damage stayed inside the
thread's uploads.
The function now checks and unlinks the requested entry itself and
treats a symlink as not found, the same way list_files_in_dir already
hides it. unlink() never follows the final component, so a file swapped
for a link between the check and the unlink removes only the link.
Tests cover the helper, the Gateway DELETE route, and
DeerFlowClient.delete_upload.
* docs(changelog): note upload delete symlink fix (#5547)
