* fix(skills): keep a slash skill activation when the model call is retried
SkillActivationMiddleware records the activated slash message in the run
context before it calls the model. LLMErrorHandlingMiddleware sits outside
it and retries a failed (429/503/timeout) or empty call by running this wrap
again on the same request; the retry found the run key already set and went
out without the SKILL.md body, while the slash source it had published kept
the skill's tool restrictions in force.
Record the activation itself next to the run key and replay it while nothing
has answered the slash message yet: a retry sees no AI message after the
target, a tool-loop follow-up call does. The replay carries the same reminder
and usage stamp without re-reading the skill or repeating the audit event and
usage record, which now sit with the other once-per-activation side effects.
Two existing tests sent the same unanswered request twice to model a
follow-up call; that is the retry shape, so their second call now includes
the first response, as the real graph does.
* docs(changelog): reference #6506
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skills): share skill cache invalidation across Gateway replicas
SkillStorage.load_skills() rescans disk and re-reads extensions_config.json
on every call, but the prompt layer in deerflow/agents/lead_agent/prompt.py
caches the enabled-skills list, the per-(app_config, user_id) entries and
the rendered <skill_system> section per process, and every skill mutation
in routers/skills.py only refreshed the process that handled it. With
several uvicorn workers or several Pods sharing one home volume, a skill
installed, edited, deleted, rolled back or toggled through replica A kept
being offered as the old skill set by replica B until B restarted, and
POST /api/skills/reload had to be sent to every Pod separately.
Reuse the MCP cache-reset mechanism. A new generic helper,
deerflow.config.shared_reset_marker, owns a hidden JSON marker beside the
resolved extensions config (.<config name>.<suffix>.json): publish()
replaces it atomically with a random generation under
extensions_config_write_lock + extensions_config_file_lock, chaining
previous_generation and an optional user_id; SharedResetMarkerTracker
polls it at most once per second (monotonic clock, never blocking behind a
concurrent poller), adopts the current state silently on its first poll,
and classifies a change as user-scoped only when exactly one publication
carrying that user happened since the last poll. deerflow/mcp/cache.py now
builds its .<config>.mcp-cache-reset.json marker through the same helper
with no behavior change.
prompt.py publishes .<config>.skills-cache-reset.json via
publish_skills_cache_reset(user_id=...) and checks the marker on every
cache lookup (global cache, per-config cache, prompt-section LRU, user
scoped entries), calling _invalidate_enabled_skills_cache() or
invalidate_user_skill_cache(). The publishing process remembers its own
generation so it does not retire the caches it just rebuilt. Every
mutation tail in routers/skills.py (install/upload, edit, delete,
rollback, enable/disable, reload) refreshes locally and then publishes the
marker off the event loop; /skills/reload answers scope=shared_config,
or scope=process when no extensions config path resolves, and a marker
publication failure is a 500 so a 200 never means only one process was
refreshed.
Docs: README, backend/docs/API.md, skills/config/mcp/gateway AGENTS.md,
and frontend skills.mdx (en/zh; also fixes the reference to a
non-existent skills/loader.py) describe the shared invalidation and drop
the per-Pod guidance. CHANGELOG entry under Unreleased / Fixed.
* docs(changelog): link the skills cache reset entry to #6495
* fix(skills): drain the reload tail and keep marker failures out of 404
Two review follow-ups on the shared skills cache reset (#6495).
POST /skills/reload awaited the local refresh and the marker publication
unshielded. CancelledError is a BaseException, so a caller that
disconnected between the two slipped past the `except Exception -> 500`
mapping, surfaced no error, and left every peer replica serving the old
skill set -- the one outcome the operator hook for external mount writes
exists to prevent. The refresh and the publication now run as one
`await_drained(_drain_skill_mutation("reload", ...))` tail, the same
discipline the install/edit/delete/rollback/toggle tails already use; the
exception -> 500 mapping and the scope response are unchanged.
Publication errors could surface as 404. resolve_shared_config_path()
only swallows the resolution-time FileNotFoundError; a config directory
that vanishes between resolution and the lock file / atomic replace still
raised FileNotFoundError from the marker write, and the install, edit,
delete and rollback handlers mapped that to HTTP 404 with the raw OS
error text for a skill that had in fact been applied. publish_skills_cache_reset()
now re-raises OSError (and a misconfigured project root's ValueError) as
SkillCacheResetPublishError, a RuntimeError that no handler maps to
404/400, so every handler reports it through its generic 500 branch; the
message names only the exception type, never a server path. The install
handler gains an explicit clause that pins that contract beside its 404
mapping.
Tests: cancelling reload after the refresh still publishes the marker;
a FileNotFoundError from the marker write yields 500 (not 404) for
install, edit, delete and rollback, 500 for the public toggle and the
generic reload error; the wrapper type is not an OSError.
* fix(middleware): guard OverflowError when parsing non-finite Retry-After hints
* fix(middleware): guard OverflowError when parsing non-finite Retry-After hints
* fix(retry): bound provider retry hints across model paths
Share a 24-hour ceiling for middleware numeric/date hints and Claude's
integer-second hints. Non-finite or oversized values use local backoff;
valid hints above the jitter cap remain unchanged and negatives clamp to zero.
Parameterize unusable hints at both middleware test levels, including nan.
Cover numeric/date boundaries and sync/async recovery through both retry paths.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(mcp): skip task servers whose discovery failed instead of dropping all tools
Per-server discovery replaced a failed or timed-out server with an empty
tool list. For a server with task_toolsets, _configure_task_tools_for_server
then reported submit/status/cancel as missing and raised
McpTaskConfigurationError, which get_mcp_tools re-raises, so the tools of
every healthy server were discarded and the cache was never published:
each agent build rediscovered every server.
load_server_tools now returns None for a failed discovery, distinct from a
server that answered with zero tools, and the wrapping loop skips such
servers before task-toolset validation. A server that answers without its
configured raw tools still raises as a configuration error.
* docs(changelog): reference #6481
* docs(mcp): name the cache-reset endpoint as a task-server recovery path
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(mcp): fall back when a token response carries an unusable expires_in
OAuthTokenManager._fetch_token() converted the token endpoint's
expires_in to a timedelta while guarding only TypeError/ValueError. Two
server-controlled inputs raised OverflowError, which escaped the parser and
failed the whole token fetch, so the MCP server never received its
Authorization header:
* int(float('inf')) raises OverflowError. json.loads accepts Infinity, so a
non-finite expires_in reached it uncaught (NaN happened to be caught).
* int(1e30) succeeds, but timedelta is capped at 999_999_999 days and rejects
the magnitude with OverflowError. A large JSON integer literal behaves the
same way.
Catch OverflowError alongside TypeError/ValueError and treat a lifetime beyond
timedelta's own range as unusable, so both fall back to the documented
one-hour default like every other malformed value.
Fixes#6472
* fix(mcp): handle both token expiry overflow stages
---------
Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(migration): read thread owners from the configured database
migrate_user_isolation.py looked for thread owners in
{base_dir}/deer-flow.db, a file DeerFlow never creates. The database is
{sqlite_dir}/deerflow.db or PostgreSQL, so the owner map was always empty
and every legacy thread was moved to users/default/.
Read threads_meta through the configured database's sync SQLAlchemy URL
(SQLite or PostgreSQL, honoring postgres_schema). When legacy threads
exist but owners cannot be read (no database file, a memory backend, a
missing driver, or a failed query), stop before migrating anything;
--allow-missing-thread-owners keeps the old assign-to-default behavior
for installs that never recorded owners.
* docs(changelog): reference #6450
* fix(migration): document recovery and report unusable database URLs
Installs that already ran the old script have every legacy thread under
users/default/threads/, and re-running finds no threads/ to migrate. The
provisioner README and AUTH_DESIGN now give the recovery: move the owned
threads back to threads/ and re-run; an existing owner directory is never
overwritten and the old copy goes to migration-conflicts/. A test pins
that re-run behavior.
create_engine errors other than ImportError (e.g. ArgumentError from an
empty postgres_url) now stop with the same "Nothing was migrated." message
instead of a traceback.
The memory AGENTS.md note said threads go to --user-id; they go to their
threads_meta owner.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(models): validate Claude extended thinking budgets
* fix(models): preserve Claude thinking budget semantics
* fix(models): isolate Claude auto thinking budgets
* fix(models): preserve interleaved Claude thinking budgets
Consolidate the additional thinking-budget boundary and type coverage from PR #6482 into #6060. Keep copied automatic budgets and preserve valid manual interleaved budgets with supported models, tools, and the effective beta header. Document the validation scope and cover native payload construction and SDK header precedence.
* docs(models): avoid upstream Claude setup overlap
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The extension notify loop told a spent shared budget apart from a
TimeoutError a contributor raised on its own by re-reading loop.time()
after the await and comparing it with the deadline. The event loop runs a
scheduled timeout up to one clock_resolution early (BaseEventLoop._run_once
compares each handle's _when against now + clock_resolution), so on a coarse
clock -- Windows' ~15.6 ms against budgets as small as 20 ms -- that read can
still land strictly before the deadline. Budget exhaustion was then logged as
a phantom hook failure (ERROR plus an asyncio-internal traceback), and the
same lag let a later contributor run on a budget that had already run out.
Classify from asyncio.timeout's own expiry (expired()) instead of a second
clock read, and latch budget_spent once the budget is gone so the skip of
later contributors is deterministic too.
Fixes#6453
Co-authored-by: sxh313 <sxh313@users.noreply.github.com>
* test(auth): make user-list ordering independent of clock resolution
* docs(test): clarify deterministic user-list ordering
Explain the lexical tie-break over stored canonical UUID strings and give
user-repository ordering its own guidance section, separate from router fixtures.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(skillscan): classify env command launchers
* fix(skillscan): bound and correct env launcher parsing
Resolve env split strings iteratively with bounded work, preserving critical
findings on limit exhaustion. Tokenize each simple command in place and keep
quoted argv words distinct from adjacent redirection operators.
Honor env assignments after option markers, GNU split-string escapes, absolute
env paths, command/exec/sudo wrappers, and the non-exiting --debug option. Scope
stdin overrides to the pipeline consumer. Add 60 parser and scan regressions,
including recursion, many-command, and deep-group deadline cases.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(voice-input): separate transcript segments for providers that trim final results
readSpeechRecognitionTranscript concatenated final and interim
transcripts with no separator, relying on providers padding final
results with trailing whitespace. WebKit's SpeechRecognition
implementation trims them, so the last finalized word glued onto the
interim hypothesis ("hello world" + "today is" -> "hello worldtoday
is"), both in the live composer and in the submitted message.
Join every adjacent segment pair with an explicit space unless either
side of the boundary is a no-space script or its punctuation (Han, kana,
CJK symbols, full-width forms) — those keep the plain concatenation
Chinese and Japanese dictation always had. Korean is treated as a
space-delimited script, matching its eojeol spacing.
* fix(voice-input): consolidate transcript boundary regressions
Port the remaining regression cases from #6449, retaining equivalent English and Hangul fixtures already present in #6432. Attach leading punctuation and percent signs without losing spacing after sentence punctuation, quoted phrases, or combining accents. Pin whitespace-only segments and document the join contract.
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(sandbox): keep output and exit code when an e2b command fails
* docs(changelog): link #6441
* fix(sandbox): keep list_dir output when an e2b listing exits nonzero
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The no-tool native stream applied _decode_escaped_newlines_outside_fences
to each streamed chunk independently, but a fenced code block almost never
opens and closes within one token-sized chunk, so the per-chunk fence regex
never saw a complete ``` pair and decoded every literal \n inside the fence.
Code like print("a\nb") reached the streamed answer and the persisted
message content with a real newline, while the non-streaming path and the
tool-mode simulated stream preserved it.
Decode the native stream with a fence-stateful decoder instead: it carries
whether the stream is inside a fence across chunks and holds back a
trailing partial ```/`\n` sequence, so the joined stream equals the one-shot
decode of the same reply. A fence left unclosed by a truncated reply keeps
its escapes, matching how an unterminated fence still renders as code.
Document native no-tool streaming in the MindIE decode contract.
* fix(goal): fall back to sync methods on sync-only checkpointers
The goal helpers called any async checkpointer method that existed. Every
BaseCheckpointSaver defines the async methods, and the sync SqliteSaver and
PostgresSaver used by the TUI and embedded DeerFlowClient raise
NotImplementedError from them, so get_goal/set_goal/clear_goal always failed
there and the TUI printed "Could not set goal.". Fall back to the sync method
only on NotImplementedError; other async errors still propagate and sync
writes stay drained across cancellation.
* docs(changelog): reference #6448
* fix(goal): log when a checkpointer call falls back to the sync method
The fallback was silent, so a sync-only saver wired into the Gateway would
work through the thread-offloaded path with no trace. Emit a debug record
naming the saver and method; debug rather than warning because the TUI and
embedded client take this path on every call.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(uploads): keep in-flight staging files out of the startup sweep
`cleanup_stale_upload_staging_files` removed every `.upload-*.part` it
found at Gateway startup. That assumed a single Gateway: nothing can be
uploading while it boots, so every staging file is a crash leftover.
With several replicas sharing one home volume (Helm RWX), a replica
starting during a rolling update deleted the staging file of an upload
another replica was still writing; that upload then failed at its
`os.link` commit and lost its bytes.
The sweep now skips staging files younger than `UPLOAD_STAGING_MIN_AGE`
(24 hours, the same guard as the project-document `.staging` orphan
sweep in `projects/trash.py`). Chunk writes refresh the mtime, so an
upload in flight stays younger than the guard; an unreadable mtime keeps
the file. The startup call site is unchanged and uses the default.
Tests: the existing sweep tests backdate their orphans; new cases pin
young-kept / old-removed, the configurable `min_age`, unreadable and
vanished mtimes, and a router test that runs the sweep while an upload
is blocked mid-write and asserts the upload still commits.
* docs(changelog): link #6445
* fix(uploads): reclaim published staging aliases at any age
Review follow-up (#6445): a Gateway that dies after the commit's
`os.link` but before the staged-name removal leaves the final file and
its `.upload-*.part` alias on one inode. With only the age guard, the
next startup kept that alias for 24 hours, and because the sweep runs
only at startup, `copy_upload_file_no_symlink` (embedded
`DeerFlowClient.upload_files`) kept rejecting the destination for its
multiple links.
The sweep now stats each staging name with `os.lstat` (a `DirEntry`
stat reports `st_nlink == 0` on Windows) and removes a name with
`st_nlink > 1` at any age: a staged part only ever gains its second
link through its own commit, so such a name is a published alias, never
an upload in flight. Lone parts keep the 24 hour mtime guard. The
destination hardlink check is unchanged.
Tests cover the alias next to a young lone part, the replacement upload
succeeding afterwards, and the crash reproduced through the real upload
path with the staged-name removal patched out.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): probe the Redis stream bridge and provisioner in /health/ready
GET /health/ready only probed the ORM engine and the LangGraph checkpointer.
The Redis stream bridge client connects lazily and nothing ever pinged it, so a
Gateway whose Redis was down started, answered 200 ready to the Kubernetes
readinessProbe and Compose healthcheck, and kept receiving traffic while every
run's first publish failed. Since #6328 the Redis bridge is mandatory for
multi-instance deployments, so such a replica is not serviceable at all.
- StreamBridge gains `async ping() -> bool`; the base returns True (no external
backend), RedisStreamBridge round-trips PING and answers False on RedisError.
- /health/ready adds a `stream_bridge` probe (not_configured for the memory
bridge; unreachable flips the endpoint to 503; a missing bridge fails closed)
and a report-only `provisioner` probe of `{sandbox.provisioner_url}/health`
that never changes the status code, since every replica shares one
provisioner. The provisioner URL is snapshotted on app.state at startup.
- A probe that overruns the endpoint deadline is now the only one marked
unreachable; finished verdicts are kept.
- The public probe gate is one lock per probe kind, so one request's probes
still run concurrently while a thundering herd opens at most one probe
connection per kind per process.
- Tests: bridge ping cases, stream-bridge/provisioner probe cases, verdict and
deadline cases, and the first route-level /health/ready tests via TestClient.
- Docs: gateway AGENTS.md, CONFIGURATION.md multi-instance notes, Helm README
and probe comment, Compose healthcheck comment.
* docs(changelog): note the readiness probe fix (#6447)
* fix(gateway): apply the sandbox proxy policy to the provisioner probe
Review follow-up for #6447.
- The provisioner readiness probe built its httpx client with the default
trust_env=True. httpx 0.28 treats CIDR entries in NO_PROXY as host patterns
rather than subnet bypasses, so with HTTP_PROXY set and NO_PROXY=10.0.0.0/8
a private provisioner that the requests-based sandbox calls reach directly
would have been probed through the proxy and reported unreachable on every
check. The probe now reuses `sandbox_http_trust_env`: loopback, private,
link-local and cluster-local addresses bypass the proxy, external hosts keep
the environment's proxy settings.
- Pin the liveness/readiness separation with a route test: `GET /health`
stays 200 while the stream bridge and database are unreachable and
`/health/ready` is 503, so a Redis blip can never become a restart loop.
* feat(wizard): add Opper as an LLM provider
Opper (https://opper.ai) is an EU-hosted AI gateway: 700+ models from
50+ providers behind one OpenAI-compatible API and one key. This adds it
to the setup wizard's LLM_PROVIDERS next to OpenRouter, with the same
shape: langchain_openai:ChatOpenAI pointed at
https://api.opper.ai/v3/compat, env var OPPER_API_KEY.
Models use Opper's bare model ids: claude-sonnet-4-6 (default), gpt-5.5,
gemini-3.8-flash and gpt-5.4-mini. Because the GPT ids are bare,
langchain-openai drops the 0.7 temperature for them the same way it does
for direct OpenAI gpt-5 models.
Checked: python -m py_compile scripts/wizard/providers.py, and each
model id against Opper's public model list
(GET https://api.opper.ai/v3/models). Not run locally: make lint and
make test.
Disclosure: Opper is my startup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs: add Opper to the config example and READMEs
Add an Opper model example next to the other OpenAI-compatible
gateways, as requested in review:
- config.example.yaml: commented opper-claude-sonnet-4-6 block after
the other gateway examples (ChatOpenAI + base_url, OPPER_API_KEY)
- README.md and the translated READMEs: the same entry in the manual
model configuration examples, after OpenRouter
- .env.example and backend/docs/CONFIGURATION.md: OPPER_API_KEY, in the
shape of the Novita lines
claude-sonnet-4-6 is a pooled id in Opper's public catalogue
(GET https://api.opper.ai/v3/models). config.example.yaml still parses
with yaml.safe_load; config_version is unchanged because the schema is.
Disclosure: Opper is my startup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- secret-cloud-token: add the github_pat_ and AIza families, matching the
canonical PII detector.
- _is_shell_path: treat .zsh by suffix, so a missing shebang or a rename can
no longer bypass every shell rule.
- Sync the same two patterns into _EGRESS_SECRET_PATTERNS so the IM egress
scrubber stays in lockstep with skillscan.
- Regression tests for all of the above.
Co-authored-by: yu1128-L <296164533+yu1128-L@users.noreply.github.com>
* docs: run configuration and migration commands in the backend environment
* docs: preserve runtime paths and extras in backend commands
---------
Co-authored-by: Zhangchenhao-111 <285911435+Zhangchenhao-111@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
test_json_output_uses_concise_review_record_schema asserts the JSON record
against str(source_file). The detector builds location.path through
relative_to_repo, which is as_posix() on both of its branches, so on POSIX the
two strings are equal and the assertion holds. On Windows str(path) yields
backslashes, so the test fails against output that is behaving as documented.
One word: str(source_file) becomes source_file.as_posix(). On POSIX the two are
the same string, so this is a no-op for CI; on Windows it makes the assertion
match what the detector emits. Every other path field in the repository is
written with as_posix() as well.
Before: 1 failed, 50 passed. After: 51 passed, on Windows with Python 3.12.
The three sibling detector suites still pass (35 passed), and ruff check and
ruff format --check pass on the file.
* fix(frontend): keep composer context when an attachment upload fails
sendMessage called onSent right after the in-flight guard, before any
attachment was prepared or uploaded. The composer's onSent drops its
one-time state there: quotes, conversation references, staged project
files and the stored draft. PromptInput keeps the text and files when the
upload rejects, so the user saw the same message and retried it, but the
retry went out without that context.
onSent now fires once the send is dispatched: after the attachments
upload, right before the run is submitted. A failed preparation or upload
rejects first, so every caller (the chat composer and the side chat)
keeps its state for a retry.
Because onSent can now arrive after the composer has moved to another
conversation, InputBox checks which conversation it shows. If it changed,
onSent leaves that conversation's live state alone and retires only the
sending conversation's stored draft and staged files, through a new
clearProjectAttachments helper that does not touch mounted state.
The chat E2E that held an attachment upload asserted the draft was gone
mid-upload, which encoded the old timing. It now asserts the draft stays
stored while the upload is pending and is gone, without being rewritten,
once the send is dispatched.
* docs(changelog): reference #6412
* docs(frontend): note onSent's guarantee ends at dispatch
* fix(frontend): retire only what a detached send carried
A send whose upload finishes after its composer unmounted still took the
live path, because only a conversation switch was detected, and could
delete a draft a new composer saved under the same reusable key. A late
send also cleared the thread's whole staged list, dropping a document
attached during the upload.
Track the composer's lifetime instead of its key, and in the detached
path clear the stored draft only while it still holds the sent text and
retire only the staged entries the send captured.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
retry_item unconditionally flipped batch.status to queued, and claim_items
only schedules queued or running batches. Retrying one failed item in a
paused batch therefore lifted the pause barrier and let the scheduler
take every unrelated pending item without an explicit resume.
Leave the batch row untouched when it is paused; the retried item is
still reset to pending and waits for resume_batch like its siblings.
Non paused batches keep the existing requeue behavior.
Fixes#6411
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
* fix(client): count threads, not checkpoints, in list_threads
checkpointer.list(None, limit=N) returns the newest N checkpoints across
every thread, so one long conversation filled the limit and hid the rest:
the TUI thread picker and --resume <title> only saw the latest one or two
threads. List threads through their single step -1 root checkpoint
instead, then read the latest checkpoint per returned thread.
Add sort_by="updated_at" so --continue keeps resuming the most recently
active thread rather than the most recently created one.
* test(client): pin list_threads scan cost and namespace filtering; docs
* docs(client): state list_threads cost as checkpoints loaded, not rows scanned
* docs(changelog): reference #6426
* fix(client): list threads by checkpoint write order, including branches
Discovering threads by their step -1 input checkpoint dropped Gateway
branches, which are seeded through update_state (step 0), and ordering
by checkpoint ts missed goal writes, which copy the previous ts under a
fresh checkpoint id.
Enumerate threads from the checkpoint primary-key index on SQLite and
Postgres (unwrapping the delta-history cache), falling back to one
list(None) walk for any other saver, and order by checkpoint_id, the
key LangGraph itself uses for a thread's latest checkpoint.
* feat(authz): admin role-assignment API for configured custom roles (#4063 gap 2)
The RBAC provider accepts arbitrary configured role names (guest, ...),
but nothing could assign them to real users: User.system_role was pinned
to Literal["admin", "user"] at the API layer while the DB column stayed
a plain String(16) for exactly this future, and zhfeng's #3462 audit
listed the missing assignment surface as gap 2 — custom roles were
declarable in YAML but unreachable for authenticated web users.
- User/UserResponse.system_role widens to a non-empty string; whether a
role is *assignable* is the admin API's contract, not the model's.
- RbacAuthorizationProvider exposes known_roles; the Gateway's new
assignable_role_names() unions the built-ins with the provider's
configured role names (custom providers: built-ins only).
- New admin-only router /api/v1/admin/users: GET lists users with their
role; PATCH assigns — role validated against assignable names (an
unknown role would silently fall through to default_role on every
decision), 404 for missing users, and the last remaining admin cannot
be demoted (bootstrap lockout protection). Enforcement is unchanged:
the assignment only changes which role the principal carries; the
@require_permission route guards and runtime provider decisions remain
the single enforcement points.
- SQLite repository gains list_users (ordered oldest-first).
Tests: router guard/422/404/409/no-op/happy-path, last-admin invariant
(both directions), assignable names union, principal carries the
configured role; the two old Literal-rejection contract tests updated to
the new contract (custom role representable; empty string still
rejected).
* fix(auth): widen the frontend user schema for custom roles (review-lessons catch)
The role-assignment API's review-lessons sweep found the one consumer
class the implementation had not enumerated: the FRONTEND parses /me
with a strict userSchema whose system_role was z.enum(["admin",
"user"]) — a user assigned a provider-configured custom role (guest)
would fail userSchema.parse in server.ts and break the auth flow before
rendering anything.
- types.ts: system_role widens to z.string().min(1), kept in lockstep
with the backend model (min_length=1). All existing consumers compare
=== "admin", so any other value is simply non-admin — no behavior
change for admin/user.
- server.test.ts: contract tests — custom role parses verbatim; empty
string still rejected.
- config.example.yaml: the authorization comment now says custom roles
become reachable via PATCH /api/v1/admin/users/{id} and that a
declared-but-unassigned role grants nothing.
* fix(auth): commit the admin_users router, drop the stale type-ignore, document the surface
Review catch on #6382: the router module was never staged — app.py and
the tests import it, so the pushed branch could not boot (ImportError in
create_app). The local tree kept it as an untracked file, which is why
every check passed before the push.
- Actually commit app/gateway/routers/admin_users.py.
- _row_to_user: drop the now-stale '# type: ignore[arg-type]' on
system_role — the model field is a plain string now (keep the comma).
- README: the authorization paragraph now mentions the admin
role-assignment surface (GET/PATCH /api/v1/admin/users, admin-only,
last-admin demotion protected, declared-but-unassigned grants nothing).
- server.test.ts import-order lint fix from the frontend sweep.
* fix(auth): field-scoped role writes close the demotion-undo race
Review round 2 on #6382 — three findings, one design:
- [P1] update_user copied system_role from the caller's account snapshot,
so a password change that read the user as admin could restore a role
concurrently revoked by the demotion route (stale-snapshot write-back).
Account writes are now field-scoped against each other: update_user is
the credential/profile writer (never touches the role, mirrors the
row's current role back), update_system_role is the only role writer
(single-column, never touches credentials or token_version).
- The last-admin lockout check moved inside the role write's serialized
transaction: count-then-write across two sessions let two concurrent
demotions of the only two admins both pass. Same idiom as
create_first_admin — BEGIN IMMEDIATE on SQLite, pg_advisory_xact_lock
on a distinct key on Postgres, raise on unknown dialects.
- UserNotFoundError from the racing-delete window maps to the route's
404 instead of a 500.
Tests 11->12: real-repo regression covering the P1 interleaving
(demotion lands -> stale credential snapshot writes -> role stays
revoked, credentials applied), field isolation both directions, and the
last-admin invariant raised from inside the write with the row intact.
Mutation-verified: restoring the role write-back in update_user fails
it.
* style(auth): format the server auth tests
* fix(tests): migrate the personal-MCP demote revocation to update_system_role
The role-write field-scoping (66e2cec23) made update_user a
credential/profile-only writer, so the demote branch of
test_revocation_blocks_discovery... silently no-oped and CI failed on
all four shards (assert 2 == 1 — the role never changed).
- The demote now goes through update_system_role, the sole role writer.
- The fixture gains a second admin: the revocation test demotes its sole
synthetic admin, which the new last-admin lockout guard correctly
blocks — a second admin keeps the scenario legal without weakening
either invariant.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): rebase scheduler shutdown bound onto latest main
* style(gateway): format scheduled task shutdown
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(frontend): keep the side-chat draft when a send fails
PromptInput clears the composer only when onSubmit resolves and keeps it
on rejection. The side chat's handleSubmit caught every failure, showed
the toast and resolved, so a failed side-chat creation or attachment
upload lost the typed text and attachments. The first message to a new
side chat was also cleared as soon as it was queued, before the queued
effect sent it, so a failure there could not be retried either.
handleSubmit now rethrows after its toast, and the queued first send
carries the submit's resolve/reject so the composer settles with that
send's outcome. The queued effect no longer toasts on its own, so a
failed first send shows one error instead of two.
* docs(changelog): reference #6407
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
