* fix(skills): load SKILL.md saved as UTF-8 with a BOM
The front-matter anchor began at ^---, so the U+FEFF that Windows Notepad and
PowerShell's `Set-Content -Encoding UTF8` prepend made the match fail. parse_skill_file
then returned None and the catalog builder dropped the skill without logging, leaving an
author with a valid SKILL.md that simply never appears. split_skill_markdown reported
"No YAML frontmatter found" for the same file at install time.
Let the anchor consume one leading byte-order mark, and reuse the shared compiled pattern
in parser.py instead of the duplicated inline copy so the two cannot drift apart again.
* fix(skills): capture the description of a BOM-prefixed SKILL.md
The reader in skill_context anchored on ^---, so the byte-order mark that this change already
tolerates in the loader and validator made the match fail and build_skill_entry_metadata_from_read
capture an empty description. render_skill_context then listed the skill without the one line the
active-skill reminder shows for it.
Consume one leading byte-order mark there and pin the capture with a regression test.
or 0.5 cannot distinguish an explicit 0.0 from an unset field, so a discounted fact got the default confidence weight in both the FTS5 document and the substring fallback, while the same row indexed through FTS5Retrieval.index_fact kept its zero weight.
* fix(sandbox): resolve the async acquire lock path off the event loop
_discover_or_create_with_lock_async offloads every step of the acquire —
ensure_thread_dirs, the lock file, cache re-checks, backend discovery and
registration — except the one line that builds the lock path itself, which
resolves through Paths.base_dir and therefore runs a syscall on the loop.
The sibling anchor documented that as "a pre-existing blocking call in this
coroutine" and stubbed the path layer to get past it. With the resolution in
a worker thread the stub is no longer needed, so this removes it and pins the
behaviour with an anchor of its own.
* test(sandbox): word the offload anchor against the state after the stub removal
* test(sandbox): pin that the gate still catches an inline thread_dir call
The lock-path anchor only asserts execution reached the lock file, so it would
stay green if the gate stopped instrumenting the syscalls behind Path.resolve()
and the inline call were restored. Add a meta-check next to the store probe's:
call thread_dir() directly on the event loop under the same isolation and
assert BlockingError.
* test(checkpoint): cover delta-history cache migration and pin two open DeltaChannel defects
- Extend the full -> delta migration contract to {cache disabled, cold, warm}
with a differential raw-saver oracle; the warm state also asserts a
cache-served re-read with no extra inner history walks.
- Pin two still-unfixed upstream DeltaChannel defects as tripwire regressions:
#8382 (parallel-superstep delta replay order diverges from live order) and
#8448 (Postgres paged delta walk hydrates old checkpoints empty). Each
asserts the correct contract and skips, naming the issue and pinned version,
while the defect is present - so a later dependency bump that lands the fix
turns it into a live gate. The contract is never asserted as buggy behaviour
(see backend/tests/AGENTS.md).
* test(checkpoint): scope the #8448 skip to the postgres parameter
The paged stage-1 walk that langgraph#8448 poisons only exists in
langgraph.checkpoint.postgres; memory and sqlite never paginate. An
unscoped skip let a future regression in long-chain materialization on
those backends hide behind the Postgres-only upstream issue instead of
failing the InMemorySaver differential oracle. Keep the assertion live
there and trip only where the defect can occur.
* fix(search): coerce DDG result limits from environment variables
* fix(search): fall back for invalid DDG result limits
---------
Co-authored-by: kwsYegar <331533331+kwsYegar@users.noreply.github.com>
* feat(storage): add a content-addressed blob store contract (#4189 item 2)
Addresses the review on the previous head:
* the retention/dedup claim in the doc now states the deletion rule (liveness
from the durable references) instead of promising refcount-free deletion, and
the sidecar records the first writer as advisory provenance only;
* one validate_blob_kind() owns the kind grammar, so the message a producer sees
is the pattern that is enforced;
* adds the missing storage/AGENTS.md (registry entry included) that both
docstrings already pointed at;
* drops the dead _import_backend_module() by routing the folder scan through it;
* exports BlobNotConfiguredError from the package;
* renames _KIND_MAX_BYTES to _MAX_BLOB_BYTES (it caps one put, not a kind);
* delete failures raise the neutral BlobStoreError instead of BlobReadError.
Rebased onto current main so the guidance-budget test matches upstream.
Signed-off-by: zeng-bohan <zengbh1@gmail.com>
* fix(storage): restore the config module and backend packages dropped by the squash
The force-push to e72b435 silently lost four items the surviving code
imports, breaking `import deerflow.storage` and the test suite at
collection:
- deerflow/config/blob_storage_config.py (BlobStorageConfig +
get/set_blob_storage_config + load_blob_storage_config_from_dict,
mirroring memory_config's host-shared schema and hot-reload pattern)
- app_config.py wiring: the blob_storage AppConfig field, the import,
and the _apply_singleton_configs -> load_blob_storage_config_from_dict
call (memory stayed wired; blob did not)
- storage/backends/__init__.py (drop-in contract docstring)
- storage/backends/local_fs/__init__.py (LocalFsBlobStore re-export +
STORE_CLASS for _scan_backends)
Verified: pytest tests/test_blob_store.py tests/test_app_config_reload.py
tests/test_app_config_name_indexes.py -> 67 passed; ruff format/check
clean on all four files.
* fix(storage): name the failing digest check; document the 64 MiB per-put cap
- get_bytes: split the corruption message into size mismatch (truncation /
partial write) vs digest mismatch (same-size bit rot, reporting what the
content hashes to vs what the ref names) — the combined message reported
'got 1024 bytes, expected 1024' on same-size corruption and sent the
debugger in the wrong direction.
- docs/blob-storage.md + storage/AGENTS.md: state the _MAX_BLOB_BYTES cap
(64 MiB) and that it is backend-level defense-in-depth rather than part
of the BlobStore contract, so the externalized-tool-results follow-up
sees the limit before the first production put fails.
* test(storage): pin both get_bytes corruption branches separately
The pre-split test corrupted the blob with different-size bytes, so only
the size-mismatch branch ever ran and neither new message was asserted.
Now: different-size corruption asserts 'size mismatch', and a same-size
reversal (bit rot) asserts 'digest mismatch' — the branch the split was
actually about.
* docs(config): document blob_storage in config.example.yaml
Add the blob_storage section (enabled/backend/backend_config) mirroring the
memory section's shape, matching the Configuration section of
backend/docs/blob-storage.md, and bump config_version 46 -> 47 per the
example-file convention for schema additions. Default state is unchanged:
enabled: false, purely opt-in.
* fix(helm): align chart config version with example
* fix(storage): refresh blob store on config changes
---------
Signed-off-by: zeng-bohan <zengbh1@gmail.com>
Co-authored-by: zengbohan1 <310902929+zengbohan1@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: zeng-bohan <310902929+zeng-bohan@users.noreply.github.com>
* fix(sandbox): enforce execution authorization for image reads
* refactor(sandbox): share image injection authorization plan
* docs(agents): trim image middleware guidance to fit CI budget
---------
Co-authored-by: YxinMiracle <“939157765@qq.com”>
_start_channel built the default seen-event store path inline, and
Paths.base_dir resolves through realpath. Channel startup reaches that
wiring from POST /api/channels/{name}/restart, so every restart ran a
filesystem syscall on the Gateway event loop; the rest of the same
function already keeps its disk IO in a worker thread.
The existing wiring test stubs get_paths with a plain-string base_dir,
so it cannot see the syscall. The new anchor keeps the real Paths object
and isolates it with DEER_FLOW_HOME instead.
* docs(configuration): document runtime environment variables in English
* docs(configuration): correct env vars flagged in review
`LOG_LEVEL` is not read by DeerFlow: log verbosity comes from `log_level` in
`config.yaml`. `DEER_FLOW_ROOT` is not a process variable either — it is a
host-side value that only `docker/docker-compose-dev.yaml` interpolates, and it
has no fallback there, so `scripts/docker.sh` supplies one for the
`make docker-*` wrapper only.
Both rows are removed, and the table now lists only variables the process
actually reads: `DEER_FLOW_PROJECT_ROOT`, `DEER_FLOW_HOME`,
`DEER_FLOW_CONFIG_PATH`, `DEER_FLOW_SKILLS_PATH` and `AUTH_JWT_SECRET`. The
config-file home for log verbosity and the host-side home for `DEER_FLOW_ROOT`
are cross-referenced instead.
The zh page carried the same two incorrect rows; it is corrected the same way
so both locales agree.
* docs(configuration): scope the project-root rule to the variable that uses it
The blanket claim that relative path values resolve against
`DEER_FLOW_PROJECT_ROOT` is false for two rows in this table. At this head:
- `runtime_paths.runtime_home()` resolves a relative `DEER_FLOW_HOME` with
`Path(env_home).resolve()`, and `config/paths.py` resolves the state directory
the same way, so both land under the process working directory.
- `AppConfig.resolve_config_path()` builds `Path(os.getenv("DEER_FLOW_CONFIG_PATH"))`
directly, so a relative value is taken from the working directory as well.
`DEER_FLOW_SKILLS_PATH` is the only one that goes through `resolve_path()`, the
single project-root-aware helper (`base or project_root()`).
The prose now names that variable, states that the other two are used as given,
and asks operators to set them to absolute paths so writable state and the
loaded config file land where they expect. The `DEER_FLOW_HOME` row repeats the
absolute-path requirement, `DEER_FLOW_SKILLS_PATH` records the project-root
rule, and the `DEER_FLOW_PROJECT_ROOT` row no longer implies it governs relative
runtime paths in general. The zh page carries the same correction.
* docs(extensions): add an eleven-page extension developer manual
Add a harness/extensions/ section in English and Chinese documenting the
deerflow-extension-api 0.2.3 contract:
index when to write an extension vs a tool, MCP server or
skill; contribution kinds; loading; failure and
trust model; versioning
quick-start build, test, install and remove a working extension
runtime load sequence, API version rules, scopes and
ExtensionData, fail-open and cancellation, diagnostics
middleware placements and where each lands, scope, ordering,
observe-only wrap hooks, failure isolation
observers task lifecycle, system model calls, agent assembly
and fingerprint, context compaction
services-and-routes service lifecycle, router rejection rules, auth,
principals
run-evidence service and request-scoped readers (#5727), cursor
semantics, deletions, redaction, store differences
plugins experimental full-stack plugins, inline modules and
manifest-based browser assets (#5685)
operations extension manager CLI, sources, upgrade, rollback,
Docker, Helm, recovery
troubleshooting indexed by the exact log and error strings
reference every public name and the contract version history
Also fix stale descriptions: the contribution kinds listed in AGENTS.md
and deerflow/extensions/AGENTS.md (now eight, including plugins), the
run-evidence redaction claim (only auth_token is removed), and the plugin
mount return value in docs/full-stack-plugins.md ({ dispose }, not a
function). Changelog entries added in both languages.
* docs(agents): trim root AGENTS.md to keep inherited chains under the size limit
* docs(extensions): scope the manual to the 0.2.1 contract
Plugins and request-scoped run evidence are not in 2.1.x-dev yet; they
move to a stacked follow-up so this change can be cherry-picked.
* docs(extensions): cover full-stack plugins and request-scoped run evidence
Brings the extension manual up to deerflow-extension-api 0.2.3. Stacked on
the 0.2.1 manual so that part can be cherry-picked to 2.1.x-dev alone.
* docs(extensions): clarify PAT rejection on contributed routes
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs(extensions): add an eleven-page extension developer manual
Add a harness/extensions/ section in English and Chinese documenting the
deerflow-extension-api 0.2.3 contract:
index when to write an extension vs a tool, MCP server or
skill; contribution kinds; loading; failure and
trust model; versioning
quick-start build, test, install and remove a working extension
runtime load sequence, API version rules, scopes and
ExtensionData, fail-open and cancellation, diagnostics
middleware placements and where each lands, scope, ordering,
observe-only wrap hooks, failure isolation
observers task lifecycle, system model calls, agent assembly
and fingerprint, context compaction
services-and-routes service lifecycle, router rejection rules, auth,
principals
run-evidence service and request-scoped readers (#5727), cursor
semantics, deletions, redaction, store differences
plugins experimental full-stack plugins, inline modules and
manifest-based browser assets (#5685)
operations extension manager CLI, sources, upgrade, rollback,
Docker, Helm, recovery
troubleshooting indexed by the exact log and error strings
reference every public name and the contract version history
Also fix stale descriptions: the contribution kinds listed in AGENTS.md
and deerflow/extensions/AGENTS.md (now eight, including plugins), the
run-evidence redaction claim (only auth_token is removed), and the plugin
mount return value in docs/full-stack-plugins.md ({ dispose }, not a
function). Changelog entries added in both languages.
* docs(agents): trim root AGENTS.md to keep inherited chains under the size limit
* docs(extensions): scope the manual to the 0.2.1 contract
Plugins and request-scoped run evidence are not in 2.1.x-dev yet; they
move to a stacked follow-up so this change can be cherry-picked.
* docs(extensions): clarify PAT rejection on contributed routes
* feat(memory): add cognitiveStyle for collaboration preferences
Extend cross-session memory with user.cognitiveStyle and cognitive facts,
backward-compatible normalization, prompt injection as Thinking Style, and
settings UI. Add design notes and unit tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(memory): normalize legacy memory imports on frontend
Add normalizeMemoryPayload aligned with backend normalize_memory_data,
use it for Settings import and API reads, and add unit/regression tests.
Document schema-evolution steps and sync FACT_EXTRACTION_PROMPT cognitive category.
* fix(memory): harden legacy memory normalization
* fix(memory): keep import envelope validation strict
* fix(memory): keep unknown fields through frontend import normalization
normalizeMemoryPayload rebuilt the memory document from a section whitelist,
so every key it did not name was dropped. Because readMemoryResponse routes
all memory API responses through it, the loss showed up on read, export, and
import alike: revision vanished, and any future backend field would silently
break lossless export/import.
The Gateway models are extra="allow" and deermem normalizes additively, which
left the frontend as the only lossy hop in the chain. Spread before overriding
at each nesting level so normalization only validates and defaults
contract-owned fields, leaving the envelope check as strict as before.
Covered by a unit test on the strict import path, two backend tests for
extension preservation and caller immutability, and an E2E round-trip that
asserts on the captured import request body.
* docs(memory): state the normalization invariant that actually holds
The docstring cited `display` as a backend-driven field alongside
`revision`, but #4546 was closed unmerged and no such field exists in the
contract. It also promised lossless export/import, which the Gateway's
`extra="ignore"` models preclude for any undeclared field.
Replace both with the invariant the code is really defending: the frontend
must not be narrower than the Gateway contract, using the declared
top-level `revision` as the concrete example.
* fix(memory): preserve legacy migration and reject lossy imports
* fix(memory): align legacy fact normalization defaults
* test(memory): use authenticated app for import regression
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): keep custom-skill rollback filesystem IO off the event loop
Closes#5747. Three blocking sites remained on the Gateway event loop in
the rollback route and its shared response reader, all offloaded through
asyncio.to_thread following the #5729 pattern:
- storage.validate_skill_markdown_content() creates a temp dir and writes
SKILL.md into it (site 1)
- the post-scan read of the file being replaced (site 2, named by #5729's
test docstring)
- _read_custom_skill_response(): load_skills() walks every skill directory
and read_custom_skill() opens SKILL.md — O(installed skills) loop time
paid per accepted rollback and per GET /skills/custom/{name} (site 3)
A blocking_io regression drives the full accepted-rollback path (past the
security scan) under the strict gate; the two pre-existing cases only
covered the pre-scan branches.
* fix(gateway): keep the skill element type in the response-parts closure
Review nit on #5751: tuple[object, ...] erased the element type for the
_skill_to_response call; Skill is already imported in this module.
* fix(authz): cover create/run/memory/agent routes with permission checks; scope USER.md per user
Route permissions: the AuthorizationProvider model only applied to routes
carrying @require_permission, so POST /api/threads, /api/threads/search,
POST /api/runs/stream and /api/runs/wait (runs:create), every /api/memory
route and the custom-agent routes ran with authentication only — a
provider configured to deny threads:write/runs:create/... could not
enforce those decisions. Add the missing decorators (threads:write,
threads:read, runs:create, memory:read/write, agents:read/write) and
register the new permission names in authz.Permissions. No owner checks
are added: per-user data scoping stays in the repository layer.
User profile: GET/PUT /api/user-profile read and wrote a single global
{base_dir}/USER.md, so in a multi-user deployment any authenticated user
could overwrite the prompt context injected for everyone else (and read
it). Scope the file to the caller's bucket
({base_dir}/users/{user_id}/USER.md) like user-scoped skills; no other
consumer of the old global path exists in the tree (verified by grep and
the updated tests). test_put_user_profile asserted the wrong effective
user under the autouse conftest user fixture; fixed to test-user-autouse.
* fix(authz): bind positional args in require_permission; migrate legacy USER.md
Review follow-up on #4989 (willem-bd):
- require_permission's wrapper only looked for request in keyword
arguments, so direct positional calls like
create_thread(body, request) made the injected keyword stub collide
with the positional request (TypeError: multiple values). The wrapper
now binds the wrapped signature via inspect.signature().bind() and
honors a positionally-passed request (and thread_id) instead of
assuming kwargs; the test-stub injection only fires when request is
absent everywhere. The two positional callers in
tests/test_threads_router.py pass again (8/8 channel tests).
- migrate_user_isolation.py now claims the legacy global USER.md for
--user-id (default 'default') like the other unowned legacy
artifacts; without it, upgrading installs with an existing profile
would read content: null and later strand the old file beside the
new per-user one. Conflict handling mirrors migrate_memory (rename to
USER.legacy.md).
* style(scripts): keep migrate_user_isolation help within the line budget
* test(authz): give internal-request stubs realistic auth fields
The create_thread permission wrapper added in this PR authenticates
the direct calls in the internal-owner tests; their SimpleNamespace
requests had state.user but no auth_source and no cookies, so
get_current_user_from_request fell through to request.cookies.get and
raised AttributeError (verified introduced by this branch: both tests
pass on the base commit).
The stubs now carry cookies={} and
state.auth_source=AUTH_SOURCE_INTERNAL, which is exactly what
AuthMiddleware stamps on real internal requests, so state.user is
honored without the JWT path. All 80 tests in the file pass.
* fix(pat): keep memory/agent permissions out of PAT scopes by design
The route permissions exist (they guard the memory/agent routers) but PATs
govern the thread/run lifecycle only: _PAT_ROUTE_RULES default-denies those
routers for PAT callers regardless of scopes. The alignment invariant
becomes a subset check plus a pinned exclusion, and pat.py documents that
opening these scopes is a product decision requiring three synchronized
changes.
* test/docs: cover migrate_user_profile; drop the unclaimed USER.md injection wording
Five-scenario test class mirroring the sibling migration steps (move,
conflict-rename, noop, both under dry-run). The GET/PUT descriptions no
longer say USER.md is 'injected into agents' — nothing at this head consumes
it for prompts; the routes are storage/retrieval only. The memory AGENTS.md
migration pointer now enumerates skills/ and the global USER.md.
* docs(paths): align the USER.md layout comment with the injection disclaimer
* test(authz): align effective permission contracts
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* docs(subagents): restructure the subagent docs into an eleven-chapter user manual
Replace the single harness/subagents.mdx page (en + zh) with a
harness/subagents/ section of eleven chapters per language:
1. index concepts, delegation flow, inheritance, terminology
2. quick-start Ultra mode, task card, results, stop behaviour
3. catalog built-ins, config.yaml / managed sources, precedence,
Custom Agent delegation scope, ACP agents
4. delegation task parameters, snapshot context, acceptance criteria,
batch_task, skills / MCP / uploads inside a subagent
5. results terminal statuses, stop_reason, report contract,
receipt verification, acceptance checklist, ledger
6. limits every limit with key / default / range / behaviour,
runaway guards, subagent_runtime capacity
7. sandbox leases, per-subagent shell sessions, MAX_SHELL_SESSIONS,
middleware chain, execution isolation
8. observability task card, SSE + persisted events, metadata keys,
token attribution, Langfuse, trace ids, batch API
9. troubleshooting symptom-indexed FAQ with the fixing PRs
10. developers create_deerflow_agent, SubagentRuntime, contracts,
extensions, security boundaries, background registry
11. reference config keys, context keys, tool signatures, enums,
events, routes, June-September 2026 change log
The old page becomes the section index (asIndexPage), so existing links to
/docs/harness/subagents keep working; the two anchor links in
middlewares.mdx now point at limits#runaway-guards. Every chapter compiles
with @mdx-js/mdx and the docs link tests pass. Changelog entries added in
both languages.
* docs(subagents): fix the docs build and align the manual with the code
Remove the `index` key from both subagents `_meta.ts` files. The index page
is marked `asIndexPage`, so Nextra treats it as the folder itself; listing it
as a child failed `_meta` validation and returned 500 for every docs page.
Correct claims that disagreed with the backend, in both languages:
- GET /api/subagents is open to all users; only writes need an admin
- subagents use their own subagents.token_budget, not the Lead Agent's
- warn_threshold injects a model-visible warning, not just a log line
- [SUBAGENT LIMIT REACHED] and subagent_limit_capped fire only when the
per-run total was already exhausted before the response
- per-response concurrency defaults to subagent_runtime.max_running
- batch tools are not registered when a supplied runtime lacks a batch
service
- ask_clarification / present_files are default denies that a config.yaml
agent can lift
- smaller fixes to result text formats, event payloads, batch item states,
MAX_SHELL_SESSIONS handling, and UI labels
The changelog entry now says only page-level links survive the split.
* fix(skillscan): keep secret literals bound by kwargs, defaults and walrus in view
#5648 replaced the line-oriented `name[:=]value` sweep with an AST walk that
only inspected Assign/AnnAssign. A keyword argument, a parameter default and a
walrus all still read as `name=value` to that sweep, so moving an embedded
credential into a call silenced a HIGH-severity rule.
* fix(skillscan): fold constant-built secret values back into the AST scan
The pre-#5648 text sweep reported `API_KEY = "sk-" + "a1b2c3d4e5f6"`,
because its value capture stopped at the first closing quote of `"sk-`
and the remainder was never examined. Splitting the quotes does not
change the bound value, but `ast.Constant` alone now requires it to:
`+`, an adjacent literal run, and a placeholder-free f-string all bind
a compile-time constant that the HIGH-severity rule no longer sees,
while the same bytes in a file Python cannot parse still trip the text
fallback. `_python_secret_literal` folds those forms, so the AST pass
stays a precision-only change for values as well as binding forms.
Runtime-composed values (`os.environ[...] + "…"`, `%`-formatting) stay
unreported, which is the precision #5648 was after.
* fix(skillscan): fold a literal chain without reaching the recursion limit
The constant fold recursed once per operand, so a concatenation of ~1000
literals raised RecursionError on input ast.parse accepts. That exception
escaped past the per-file analyzer guard in scan_skill_dir, which drops every
finding collected for the file, so the deep chain cost the file its other
findings as well. The fold is now a stack walk that collects operands in
source order for either nesting, and the file's remaining rules keep reporting.
* fix(skillscan): report credential defaults on lambda parameters
_python_secret_bindings walked parameter defaults only for FunctionDef and
AsyncFunctionDef, so a credential moved into a lambda default (positional or
keyword-only) escaped the secret-env-assignment gate even though the
line-oriented sweep this rule replaced reported it. ast.Lambda exposes the same
ast.arguments structure, so add it to the tuple and pin both spellings.
Addresses review feedback on #5691.
---------
Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
_auto_create_postgres_db opens a throwaway engine against the server's
`postgres` database and disposes it in its finally, but that await was bare.
Startup cancellation landing there abandoned a half-disposed pool that nothing
else owned — the engine is a local, and the CREATE DATABASE has already
committed under AUTOCOMMIT, so the next boot does not take this path again.
Drain the dispose through the cancellation-safe await_drained() helper the
engine's own close path already uses, and pin it with a repeated-cancellation
regression.
* fix(logging): collapse space-carrying Redirecting slots (#5225 round 16)
A Redirecting slot was kept verbatim whenever a scheme matched at
position 0, but the generic absolute-URL pass it is handed to stops
both its `host` and `rest` groups at whitespace. A Location header
value carrying an interior space (legal field syntax a misbehaving
server can emit, and the shape round 13 deliberately kept matching)
therefore survived in the clear after the first space:
'Redirecting /private/x?token=Q -> https://cdn.example/other page?sig=LeakedSig'
redacted to '... -> https://cdn.example/<redacted> page?sig=LeakedSig'.
The slot is now kept only when a whitespace-free absolute URL fills it,
so the pass is guaranteed to consume it whole; space-carrying slots
collapse like every other shape neither pass could cover.
* fix(logging): ask the URL pass whether it consumes a Redirecting slot whole (#5225 round 16 review)
The kept-set regex ([a-zA-Z...]*://\S+) was a stand-in for the invariant
"the generic absolute-URL pass consumes the slot whole", and two
whitespace-free absolute shapes satisfied it while the pass actually
stopped early: a quote that reads as a closing mark (https://h/a')b?sig=…)
and an empty host before the first /?# (https:///path?sig=…), which its
host group never matches. Both left the signed tail in the log.
Replace the stand-in with the check itself: run the pass's own pattern
against the slot and keep it only when the match spans it end to end, so
the rule cannot drift from the pattern's stop conditions again.
---------
Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
* chore(checkpoint): bump to checkpoint 4.2.0 and drop the stale saver patch
langgraph-checkpoint 4.2.0 fixes the write dropped after a full -> delta
migration (upstream langchain-ai/langgraph#8526) and keeps its own
InMemorySaver override, so the compatibility patch in checkpoint_patches.py
could neither detect the fix nor stand down: it read langgraph's version
while InMemorySaver ships in langgraph-checkpoint, and it would have kept
shadowing the upstream implementation on 4.2.0. The postgres release 3.1.2
locates plain-value delta seeds (upstream #8535).
- floor langgraph-checkpoint>=4.2.0,<5.0 in the harness and raise the postgres
extra to >=3.1.2,<3.2; langgraph's own >=4.1.0 would otherwise let a
resolver reintroduce the bug now that the patch is gone
- delete the patch, its version guard, and the three implementation-assertion
tests; test_full_to_delta_migration_replays_on_same_thread stays as the gate
and fails on 4.1.1 without the patch (verified by a one-off unpatch run)
- langgraph stays 1.2.9 and langgraph-checkpoint-sqlite stays 3.1.1
Verification: make lint; delta checkpointer/cache/worker-resume suites green
with the migration gate passing on memory and sqlite (postgres params skip
without TEST_POSTGRES_URI). The full make test run before the packaging-pin
fix below was 13 failed / 18523 passed; 12 of those failures reproduce on main
with the same environment (git stash baseline, identical failure sets), 7 of
them because the local config.yaml runs checkpoint_channel_mode: delta. The
remaining one, the postgres-extra string pin in tests/test_checkpointer.py, is
updated here.
Upstream status for the surrounding gaps: cache parity across off/cold/warm
states, postgres pagination (langchain-ai/langgraph#8448) and parallel-superstep
replay order (#8382) are still open upstream, as is the abandoned-branch fork
fix (#8548) that the delta resume linearization in runtime/runs/worker.py works
around, so that path stays.
* docs(changelog): keep checkpoint upgrade in Unreleased
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The Gateway's staged upload commit links the `.part` file to its final name and
then unlinks the staged name. The conversion path duplicates a descriptor on the
staged inode *before* that commit (deliberately: it is what keeps conversion
reading the bytes this request wrote rather than whatever the committed name
points at afterwards), so the staged file still has an open handle when the
commit tries to remove it.
POSIX allows that; Windows does not. Uploading a convertible document
(`uploads.auto_convert_documents: true`, a PDF/DOCX/PPTX/XLSX upload) on a
Windows host therefore failed with a 500:
Failed to upload report.pdf: [WinError 32] The process cannot access the
file because it is being used by another process
Split publishing from removing the staged name: `_commit_upload_temp_no_overwrite`
now takes `unlink_staged`, and the ingestion service keeps ownership of the
staged path whenever it still holds the conversion descriptor, removing it once
that descriptor is released. The same best-effort removal replaces the
`os.unlink` in `_abort_upload_temp`, where an abandoned duplication worker can
hold the inode open and turn a cancellation into a secondary permission error.
Both removals are best-effort: a staged name that cannot be removed yet is
already hidden from every upload listing, and the Gateway sweeps leftovers on
startup.
Verified on Windows (Python 3.12):
- the new regression test reproduces the sharing violation portably (it pins the
staged name while the descriptor lives) and fails on main with the exact 500
above; it passes here
- `pytest tests/test_uploads_router.py tests/test_project_documents_promotion.py
tests/blocking_io/test_project_documents_promotion.py` -> 88 passed
(16 of those failed on main on this host)
- `ruff check` / `ruff format --check` clean
Co-authored-by: Shxiao101 <Shxiao101@users.noreply.github.com>
`CONTRIBUTING.md` documents `make docker-init` as "Build the custom k3s image
(with pre-cached sandbox image)" and lists building Docker images, installing
frontend/backend dependencies, and sharing the pnpm cache as its effects. None
of that is what the target does: `Makefile`'s `docker-init` runs
`scripts/docker.sh init`, which only checks `docker images` and runs
`docker pull` for the all-in-one sandbox image (skipping even that in local
sandbox mode). It never calls `docker build`, and no k3s image exists anywhere
in the repo. The root Makefile's own help text already says
"make docker-init - Pull the sandbox image".
The same misconception makes the registry-override note wrong: `UV_INDEX_URL`
and `NPM_REGISTRY` are consumed as build args by `backend/Dockerfile` and
`frontend/Dockerfile`, i.e. during the `docker compose up --build` that
`make docker-start` performs, so they cannot affect a pull-only target.
Correct the setup step, the command comment, and the registry note. No code
change.
Verified against the checkout:
- `Select-String -Path scripts/docker.sh -Pattern 'docker build|docker pull'`
-> only `docker pull "$SANDBOX_IMAGE"` (line 281); no `docker build`
- `Select-String -Path Makefile -Pattern 'docker-init'` -> line 57 help text
"Pull the sandbox image"; line 187 target -> `scripts/docker.sh init`
- `Select-String -Path backend/Dockerfile,frontend/Dockerfile -Pattern
'UV_INDEX_URL|NPM_REGISTRY'` -> both declared as build ARGs
- `backend/tests/test_client_live_policy.py::test_documentation_matches_live_test_commands`
(the only test that reads CONTRIBUTING.md) still passes; it asserts
"make test-live", the live opt-in variable, and "API", none of which this
touches.
Co-authored-by: Shxiao101 <Shxiao101@users.noreply.github.com>
* fix(skills): resolve the user-scoped install scan from its own config
UserScopedSkillStorage.ainstall_skill_from_archive re-implements the parent
body to redirect the install target into the per-user custom root, and the
copy dropped the app_config argument. _scan_skill_archive_contents_or_raise
therefore fell back to the process-global get_app_config() for the static
scan, while the archive preflight it inherits still read self._app_config --
so the two gates of a single install could consult different configs.
The divergence is reachable whenever the storage outlives a config edit:
DeerFlowClient snapshots get_app_config() at construction, and the Gateway
hands its per-request get_config() to get_or_new_user_skill_storage, while
get_app_config() hot-reloads a later edit of config.yaml. With the storage's
config enabling skill_scan and the global disabling it, the preflight ran,
the content scan silently skipped, and a CRITICAL archive installed.
* fix(skills): thread app_config into the per-file LLM scan
_scan_skill_archive_contents_or_raise takes app_config and uses it for the
static gate, but never forwarded it to _scan_skill_file_or_raise, so the
per-file LLM scan fell back to the process-global get_app_config() for
skill_evolution.moderation_model_name and the model it constructs. Under
the stale-snapshot scenario the content-scan fix addresses, the two halves
of one install still read two different configs.
Raised in review of #5703 by willem-bd. The gateway's own skill-write
routes already pass app_config to scan_skill_content; this brings the
install path in line.
* feat(image-search): expose the color and license_image filters
`_search_images` accepts `color` and `license_image` and forwards them into the
`f` filter payload that `ddgs`'s duckduckgo_images engine builds
(`duckduckgo_images.py:62-79`), so the provider already declares support for
both. But `image_search_tool` never took them as parameters and never passed
them, so no caller -- model or config -- could set either one. They were dead
parameters: declared, wired, and unreachable.
Both matter for this tool's stated purpose, which is sourcing reference images
for image generation: `color` narrows results to the palette being generated,
and `license_image` filters to license-cleared results when the reference will
be redistributed.
Expose them on `image_search_tool` alongside the existing `size` /
`type_image` / `layout` filters and pass them through. `_search_images` only
forwards truthy filters, so an unset filter still produces the same request as
before.
Adds two regression tests: one asserting both filters reach the DDGS call
(red on main with `KeyError: 'color'`), one pinning that unset filters stay out.
* fix(image-search): add a usage hint to the color filter docstring
The sibling filters each carry a usage hint ("Use \"Large\" for reference
images", "Use \"photo\" for realistic references"), but `color` only listed
its options. Since this docstring is the model's parameter surface, add a
short hint so the model knows when the filter applies, and note that
"color" means full-color rather than a meta-parameter.
---------
Co-authored-by: RXQ6 <RXQ6@users.noreply.github.com>
In grouped mode the sidebar indents a project's thread rows with a nested
SidebarMenu carrying ml-4, but SidebarMenu defaults to w-full. Width 100%
plus a 16px left margin overflows the sidebar by 16px, so the absolutely
positioned row kebab (right-1) lands past the visible edge and gets
clipped. The Archived group nests one level deeper and overflows 32px,
hiding its kebabs entirely.
Swap w-full for w-auto on both indented menus so the block-level flex
container fills the remaining width minus its margin. A dom test renders
ProjectsSection in grouped mode from seeded query caches and pins that
every indented menu drops w-full while the root menu keeps it.
Fixes#5681
_receive_single_file dropped an attachment with zero log lines whenever
_download_by_code returned None (or empty bytes) - each None reason
already has its accurate line inside the download function (non-200
exchange, missing downloadUrl, oversize abort, transport failure), but
nothing tied the skip to the file being received, and the empty-bytes
case landed in the same silent branch. The caller now logs a neutral
guard line naming the file - the same shape the WeChat channel's callers
and the manager reader use since #5225.
Both the None and empty-bytes paths are pinned by caplog tests;
reverting to the silent branch turns them red. Full DingTalk suite
135 passed / 1 skipped.
