* fix(wizard): replace retired Claude Sonnet 4 default with claude-sonnet-5-5
The setup wizard's Anthropic default model is now `claude-sonnet-5-5` instead of the retired `claude-sonnet-4-20250514`. A new per-model `model_omitted_config` field makes the wizard leave out the extended-thinking (`budget_tokens`) and thinking-disabled templates for that model only, since Sonnet 5.5 rejects them; the other Claude models are unchanged. The Anthropic examples in `config.example.yaml` and `configuration.mdx` (which used `claude-3-5-sonnet-20241022`) now use `claude-sonnet-5-5` without those settings, and the change also adds a regression test and a CHANGELOG entry.
Notice anthropic-retired-models-2026-10-27 about api.anthropic.com's API describes the change. It was read from a file and is not signed by the provider.
* Update CHANGELOG with PR reference #7585
Added a reference to pull request #7585 in the CHANGELOG.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): keep sync_labels.py from crashing on non-UTF-8 stdio
The ✓ / ✗ status glyphs and the em dash in the Done line crashed the sync
with UnicodeEncodeError whenever stdout was redirected on a host whose
stdio encoding cannot represent them (Windows code pages, POSIX C locale,
PYTHONIOENCODING=ascii) — after `gh label create --force` had already run,
leaving a half-applied sync and a traceback. gh's UTF-8 diagnostics were
additionally decoded with the host locale, mangling failure messages.
Closes#7389.
- Reconfigure stdout/stderr in main(): UTF-8 with backslashreplace when
redirected, backslashreplace only on consoles.
- Decode the gh subprocess as UTF-8 with errors="replace", matching
scripts/support_bundle.py's _run_command.
- Add backend/tests/test_sync_labels_script.py: hermetic offline cases
(stub yaml, stub gh) running the real script under PYTHONIOENCODING=ascii
for the success path, a UTF-8 gh failure, and --dry-run. All three go
red on the unfixed script.
* style(tests): fix sync-labels regression lint
* docs(scripts): document label sync encoding policy
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): make doctor.py --help print usage without running diagnostics
scripts/doctor.py never parsed its arguments: `python scripts/doctor.py
--help` printed no usage and ran the whole diagnostic suite (subprocess
probes, config resolution, provider checks) and exited 0; any unrecognized
flag was silently ignored the same way. Closes#7386.
- Parse arguments with argparse.ArgumentParser in the __main__ entry point:
--help prints usage and exits 0, unknown flags exit 2.
- The no-argument invocation used by `make doctor` is unchanged, and main()
stays directly callable from the existing unit tests (they run in-process
under pytest, whose argv must not reach argparse).
- Add backend/tests/test_doctor_script.py: offline subprocess cases for
--help (usage, no diagnostics run), an unknown flag (exit 2), and the
no-argument run (report footer present, exit 0 or 1).
* test(scripts): isolate doctor CLI subprocess checks
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
The analyzer prints ensure_ascii=False JSON, but the parent pipes used
the ambient locale codec and the child inherited the ambient stdio
codec: non-ASCII skill facts crashed the review pass on cp1252
Windows and mojibake'd on cp936. Pin both sides to UTF-8.
Fixes#7387
The two-Gateway harness checklist in CONFIGURATION.md said a crashed
leader hands its providers over within lease_seconds + grace_seconds; a
peer only tries to acquire on its fixed lease_seconds // 3 tick, so the
worst case is one renewal interval more (50 s with the defaults), as the
leader-lease section of the same file already says. The grace_seconds
field description had the same shorter bound.
The channels.leader_lease comment in config.example.yaml still said web
UI credential changes reach the holder only when it next starts the
channel; since #6988 the holder applies them within one renewal interval.
The harness `check` footer still listed the IM channel gates as waiting
for leader election, which landed in #6843; it now points at the manual
channel item in CONFIGURATION.md, which also gains the runtime-config step.
configure.py did not parse its arguments, so `--help` printed no usage and
ran the bootstrap as a side effect (creating config.yaml, .env and
frontend/.env in a fresh checkout), and any unrecognized flag was silently
ignored the same way. Parse arguments with argparse before touching the
filesystem: --help now prints usage and exits 0, unknown flags exit 2, and
the no-argument invocation used by `make configure` is unchanged.
Extend the regression tests from #7102 with two offline subprocess cases
covering both new behaviors.
Fixes#7382
`render-config`, `secrets` and `nginx-conf` write `--out` through
`os.replace`, so an existing directory surfaced as a raw `PermissionError`
traceback, and a file ancestor as `OSError` from `mkdir`.
Validate the first existing ancestor once in `main()` before any writer runs,
and state the file contract in `--help` for all three commands.
* fix(ci): retarget label docs at triage.yml and guard label declarations
.github/labels.yml still described the auto-labeling setup as the three
workflows triage.yml replaced (pr-labeler.yml / pr-triage.yml /
issue-triage.yml), and its section headers pointed at the deleted
.github/labeler.yml and pr-triage.yml. A reader following those pointers
lands on 404s.
The same pair had an unguarded failure mode: triage.yml applies labels by
name, and GitHub rejects the whole addLabels batch with 422 when any name is
missing from the repository. Nothing checked that a label added to the
workflow also reached the declarative source of truth.
Point the comments at triage.yml and add scripts/check_label_declarations.py,
which fails when the workflow applies a label labels.yml does not declare.
* fix(ci): trace applied labels and cover the declaration guard
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(agent): expose read-only run status tool
* refactor(tools): name the subagent dispatch tool set
Hoist the hardcoded subagent dispatch tool-name literal into _SUBAGENT_DISPATCH_TOOL_NAMES so a future dispatch-tool rename has one place to update. No behavior change: the folded-subagent context_fill guard compares the same two names.
* test(tools): account for run status in collision assembly
* docs: keep script guidance within size budget
* fix(startup): detect extras for browser-configured IM channels
* docs(startup): clarify channel extra provider source
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): reject a directory passed to support_bundle --out
`--out` names the zip, but the path went straight to `zipfile.ZipFile`, so
pointing it at an existing directory ended in a raw `PermissionError` traceback
(Windows) instead of the CLI's usual `error: ...` + exit 2.
`create_support_bundle` now raises the `ValueError` the CLI already maps to a
clean message, and the regression test pins exit 2 with no traceback.
* docs(scripts): say "zip file path" for support_bundle --out
Review nit: the help text said only "Output zip path", so the
file-not-directory contract the new validation enforces was invisible until a
run failed.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): find Git Bash under a PortableGit mingw64 layout
`scripts/run-with-git-bash.cmd` derives `bash.exe` from `git.exe`'s parent,
which only holds for the Git for Windows `<root>/cmd` and `<root>/bin`
layouts. PortableGit and GitHub Desktop ship `git.exe` in
`<root>/mingw64/bin`, so the wrapper looked for `<root>/mingw64/bin/bash.exe`
and reported "Could not locate Git for Windows Bash" even though
`<root>/bin/bash.exe` exists. Every Makefile recipe that runs a repository
shell script failed on those hosts.
Resolve the shell under the Git installation root instead, and mirror the rule
in `backend/tests/support/shell.py` (documented as mirroring the wrapper) so
the shell-script suites stop skipping on the same hosts.
* test(backend): pin both Git layouts in one shell-discovery suite
Review follow-up for the PortableGit probe:
- `_git_roots` now probes `git.exe`'s parent and grandparent unconditionally —
exactly the two directories the Makefile wrapper checks — so the batch probe
and the Python mirror cannot drift; the docstring records that the first
`mingw64` candidate normally misses and is kept only to mirror it.
- fold the candidate-layout cases and the Windows-only wrapper case into
`tests/test_support_shell.py`, which already owns this module's unit tests,
instead of a second file with a duplicate `_touch` helper.
- report the wrapper's stdout in the failure message: its diagnostic goes to
stdout, so `stderr` was empty on the discovery-failure path.
* feat(deploy): select production Compose env files
Use an operator-selected dotenv file consistently for interpolation, container environment, secret probes and build extras. Preserve default selection and shutdown after a profile is removed.
Co-Authored-By: Codex <noreply@openai.com>
* fix(deploy): address production env selection review
Clarify Docker launcher entry points and render real Compose through the launcher to verify both service environments.
Co-Authored-By: Codex <noreply@openai.com>
* test(deploy): cover unreadable env selectors
Reject unreadable explicit selectors before Docker is invoked, using POSIX permission checks and restoring the fixture mode in finally.
Co-Authored-By: Codex <noreply@openai.com>
* test(deploy): scope dotenv assertions to selected secrets
---------
Co-authored-by: Codex <noreply@openai.com>
* fix(models): fall through an unusable credential source instead of ending the lookup
load_claude_code_credential's env branch returned None even when
_credential_from_direct_token rejected the value, so a valid
~/.claude/.credentials.json was never consulted for a blank token.
load_codex_cli_credential's 'or' chain stops at the first truthy layout,
so a blank or non-string top-level access_token hid a usable 'token' or
'tokens.access_token' in the same file -- and made the loader disagree
with scripts/doctor.py, which validates all three layouts.
Fixes#7139
* fix(models): try both env token sources and log the rejected one
A whitespace-only CLAUDE_CODE_OAUTH_TOKEN is rejected by _credential_from_direct_token, but os.getenv(A) or os.getenv(B) never reads ANTHROPIC_AUTH_TOKEN while A is truthy, so the blank first variable hid a usable second one. Test each source with the same non-blank rule, and log the rejected variable at debug (no token material) so the resolution outcome stays observable.
Also refresh the doctor.py mirror comment: the loader rejects whitespace-only tokens in every layout now, so the documented strictness divergence is gone.
* style(tests): format credential fallback log assertion
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(docker): make dev provisioner opt-in via compose profile
Gate the Kubernetes provisioner behind a Compose profile and have the
wrapper enable it only when config selects provisioner sandbox mode.
This lets the default Docker Compose stack start without kubeconfig.
Fixes#3232
* test(docker): render every compose profile in the env-file regression test
main added backend/tests/test_compose_default_bind_host.py, whose _render helper asserted that the dev provisioner still loads ../.env. `docker compose config` without a profile omits profile-gated services, so that assertion failed once the provisioner became opt-in.
Pass `--profile *` from the helper: the test is about an existing .env reaching every service, so it has to render the services a profile gates rather than only the ones `up` starts by default.
* docs(docker): restore script mode and localize direct Compose guidance
The dev provisioner is now opt-in, but the earlier edit also flipped scripts/docker.sh from 100755 to 100644 even though README.md still documents direct `./scripts/docker.sh` invocation. Restore the executable bit.
Also add the direct-Compose guidance to the translated READMEs so readers of README_zh.md, README_ja.md, and README_fr.md learn that `--profile provisioner` selects the Kubernetes mode and the default stack does not need Kubernetes.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): feed config-upgrade inline python on stdin, not argv
On Windows setups where python/uv resolve through shim layers that
reparse arguments with cmd.exe (e.g. pyenv-win), the multi-line
`python -c "..."` payloads in config-upgrade.sh get mangled mid-flight
(observed as `IndentationError: unexpected indent` on a stray
`|| goto :error` line) and `make dev` fails before the Gateway starts.
Feed both inline scripts on stdin instead, which is safe across every
spawn chain, and load the checkout .env explicitly inside the resolver
probe: the default load_dotenv() discovery walks up from cwd/__file__,
which a stdin-fed script (`python -`) cannot satisfy. Pin both with a
static regression test.
* test(config): isolate upgrade fixtures from inherited dotenv selectors
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): arbitrate IM transports with a per-provider leader lease
Every Gateway instance connected every enabled IM platform with the same
bot credentials, so channels could not run on more than one instance
(Telegram 409 storms, Discord answering twice, scattered per-process
state). channels.leader_lease was reserved by #6746; this implements it.
- channel_leases table (migration 0041): one row per provider. Acquire is
one dialect-native upsert decided against the database clock and bumps
a fencing epoch; renew extends only a live lease at the holder's epoch;
release sets the expiry to 0 so a peer takes over on its next tick.
- app/channels/leadership.py: the holder's local monotonic deadline
(send time of the last confirmed renewal + lease_seconds); holds() turns
false one renewal interval early, so the transport stops at least
grace_seconds before a peer may connect. DB calls are bounded.
- ChannelService: start fence in _start_channel (no DB call) behind every
start path; fixed-rate supervisor renews/acquires and runs transport
starts/stops as per-provider tasks; takeover reloads the provider's
saved config; failed starts back off; shutdown releases only leases
whose transport stopped. get_status reports leader/leader_here and the
holder's transport state on every instance.
- Notification worker leaves rows of providers another instance holds.
- Gate: the enabled=false refusal now recommends auto/true; an explicit
true with database.backend memory is refused when a channel is enabled.
- Docs, Helm, deployment guides, AGENTS.md and CHANGELOG updated.
* docs(changelog): link the channel leader lease entry to #6843
* fix(channels): see peer-held leases for providers absent from local config
Review on #6843: a provider connected from the web UI on a peer never
enters this instance's channel config, so the notification worker here
still claimed its rows (and parked them for 900s) and GET /api/channels/
reported no leader for it. Every supervision round, and startup, now also
reads the whole lease table once (bounded), and notification exclusions
and status use those observations: a provider whose live lease a peer
holds is left to it even without local config.
Running that read concurrently with renewals exposed that, through
SQLAlchemy's aiosqlite adapter, an UPDATE ... RETURNING racing a read on
another connection can hand back one of its own bind values instead of
the column (stored rows stay correct; plain sqlite3 and raw aiosqlite do
not show it). A lease must never take such a row for a confirmed
renewal, so acquire/renew/release now decide by affected-row count and
acquire reads its epoch back in the same transaction.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(channels): add channels.leader_lease and gate it under multi-process signals
IM channels other than GitHub connect outbound to their platform. With more
than one Gateway instance each one opens that connection with the same bot
credentials: Telegram rejects the second poller with 409, Discord answers every
message twice and per-process ChannelManager state diverges. The multi-process
startup gate said nothing about channels; the only guard was a comment in the
Helm values.
Add the `channels.leader_lease` section (`enabled: auto|true|false`,
`lease_seconds`, `grace_seconds`; startup-only, registered in the reload
boundary) that the per-provider leader lease will read, validated by a Pydantic
model from the non-schema `channels` mapping. The gate now refuses
`enabled: false` under a multi-process signal while a non-GitHub provider is
enabled under `channels.*` or connectable through `channel_connections`, naming
the providers and the rollback; a malformed block is refused on every topology.
`auto` follows the multi-process signal, so a single instance is unchanged.
ChannelService strips the block before its per-provider loops and exposes it
as `leader_lease`; `ensure_ready_channels` only visits registered providers.
The two-Gateway harness overlay leaves the block alone. config_version 60 in
the example, the Helm chart and its README; docs replace the comment-only
"keep 1 replica" guidance with the gate.
* docs(changelog): link #6746
* fix(channels): stop recommending the unimplemented leader lease as a remedy
Review on #6746: the refusal told operators to switch channels.leader_lease
from false to auto/true "so one instance owns each provider's connection", but
the transport arbitration ships separately, so that change only bypassed the
clause and every worker still started the same bot.
The refusal now offers only the remedies that work today (a single instance, or
disabling the channels) and states that auto/true are reserved for the lease
that ships separately. A regression helper in the gate tests asserts the
remedies start with the rollback and that auto/true are never offered. The
config example, Helm values/README, CONFIGURATION.md, README, the deployment
guide (en/zh), the model docstrings and both changelogs describe auto/true as
reserved settings rather than an ownership guarantee.
* fix(channels): warn on an enabled unknown channel key; reword the changelog entry
Review on #6746: the registry guard in ensure_ready_channels skipped a
misspelled provider key (channels.telegam) silently, where it previously
reached _start_channel and logged 'Unknown channel type'. An enabled dict
block that is neither a registered provider nor leader_lease now logs a
warning that lists the registry.
The changelog entry claimed the chart and docs drop the "keep 1 replica"
guidance; they keep it (auto/true do not make several replicas safe yet) and
only stop relying on the comment alone.
* fix(dev): disable UI channel providers in multi-instance overlay
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(extensions): add example guide and packaging checks
* fix(extensions): align packaging checks with installer
Reject Windows drive-qualified wheel paths before extraction and reuse the installer entry-point validator. Cover unsafe paths, dotted attributes, and valid dotted modules in regression tests.
* fix(extensions): skip unused wheel extraction
Only extract wheels when browser manifests need validation, with a backend-only regression test. Link the three remaining example READMEs in the developer guide.
* docs(frontend): trim guidance intro below size budget
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(community): add You.com web search provider
Adds You.com as an opt-in community `web_search` engine, mirroring the
existing community engines (tavily, brave, exa, firecrawl, groundroute).
You.com runs its own web index and returns extracted page snippets per
hit rather than a single meta description, so a research loop gets usable
context without a follow-up fetch.
The provider picks its endpoint by whether a key is configured, so it
works out of the box and scales up when the operator adds one:
- no key -> /v1/agents/search, keyless free tier
- YDC_API_KEY -> /v1/search with X-API-Key, full Search API
Endpoint and headers are always chosen together; the keyless endpoint
rejects an auth header, so a stale key can never be sent there. Both
return the same body shape, so one normalizer covers both.
httpx only, no new dependency and no pyproject extra, so uv.lock is
untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(community/youcom): review follow-ups — drop stray .pr-body.md, guard OverflowError, add 429 hint
* fix(doctor): make the You.com free-provider detail true with or without YDC_API_KEY
The doctor detail returned before any key check, so operators who had
set YDC_API_KEY still saw 'set YDC_API_KEY for the full Search API'.
Reworded to match the sibling free-provider entries and stay accurate in
both states.
* fix(search): sweep youcom max_results onto the shared owner
The single-owner gate that landed with #6563 flags this branch's private
_coerce_max_results as a newly hand-rolled copy, so backend-unit-tests
now fails after merging main. Follow the coordination note from review:
delegate to deerflow.community.search_max_results.coerce_max_results,
which owns the rejection profile (booleans, non-integral floats,
OverflowError), and keep only the 1-100 mirror of You.com's server-side
count clamp on top — clamping policy stays per-provider.
The .inf regression test now goes through the public tool path and also
pins the bool and non-integral-float shapes of the shared bar.
* docs(tools): use the bare you.com platform link in config and docs
Match every other provider entry: no other provider in config.example.yaml
or the en/zh tools.mdx uses campaign-tagged URLs, so drop the utm params
from the three you.com/platform links. Integration attribution stays in
the identifying User-Agent sent to You.com hosts.
* test(youcom): cover shared max_results ownership and bounds
Register You.com among the shared-owner providers and assert default handling and the 100-result cap through outbound request parameters.
---------
Co-authored-by: Brian Sparker <brainsparker@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: valueadd-lgtm <339269770+valueadd-lgtm@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): reject local BIND_HOST values nginx would misread
nginx-local-conf.sh checked only for address and hostname characters.
BIND_HOST=0 rendered `listen 0:2026;`, which nginx resolves to 0.0.0.0 and
binds on every IPv4 interface, and 1.2.3 bound 1.2.0.3. Values nginx rejects
outright ([[fd00::1]], -foo) passed the serve.sh preflight and failed only
after stop_all had torn down the running stack.
Accept exactly a dotted-quad IPv4 address, an IPv6 address (brackets
allowed around IPv6 only), or an RFC 1123 hostname. Returning to the
loopback default removes a temp/nginx.local.conf rendered for an earlier
value, and stop_all now says why it can quit a rendered-config nginx
through the tracked config; a test pins the shared pid directive.
* docs(changelog): reference #6635
* docs(changelog): match the zh reference spacing
* fix(scripts): check every BIND_HOST hostname character before splitting labels
read -r -a stops at the first newline, so is_hostname validated only the
first line of a value like $'localhost\nbad' and the rest reached the
rendered listen directive. Check the allowed characters across the whole
value first, and cover multi-line, CR and tab values for each validator.
* feat(dev): add a two-Gateway multi-instance harness script
There was no way to run the deployment.multi_instance topology locally:
scripts/serve.sh pins one Gateway to port 8001, the local nginx config has a
single upstream, and getting two processes past the multi-instance startup gate
needs Postgres, Redis, a shared DEER_FLOW_HOME, shared auth secrets and a
per-process DeerMem retrieval index by hand.
scripts/dev_multi_instance.sh (make dev-multi / dev-multi-check /
dev-multi-down) starts throwaway Postgres and Redis containers on loopback,
renders a config from the developer's config.yaml (or config.example.yaml),
and runs Gateway A on 8001 and Gateway B on 8011 with one shared home, one
copy of extensions_config.json and generated AUTH_JWT_SECRET,
DEER_FLOW_INTERNAL_AUTH_TOKEN and DEER_FLOW_CREDENTIALS_KEY values, plus an
optional round-robin nginx on 2027. B starts after A answers /health: Alembic
migrations are advisory-locked, but the LangGraph checkpointer/store setup()
that follows is not.
scripts/dev_multi_instance.py holds the testable parts: the config overlay
(neutralizing and reporting settings the gate refuses), the secrets file, the
nginx config, and `check`, which automates the scriptable cross-instance
acceptance checks: readiness with stream_bridge ok, A-issued sessions and the
internal token accepted by B, thread/upload/artifact visibility, a skill
toggle seen by B, nginx balancing, and SSE join on B with Last-Event-ID resume
on A. Tests pin that the rendered config passes the real
_enforce_postgres_for_multi_worker gate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): link the two-Gateway harness entries to #6613
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(dev): keep the multi-instance harness inside its own state
Address review findings on the two-Gateway harness (#6613):
- Containers carry a deerflow.harness.state-dir label; up/down remove only
containers of their own state dir and refuse those of another checkout or
DEERFLOW_MI_STATE_DIR instead of force-removing them.
- Containers are removed with `docker rm -f -v`, so the anonymous volumes
the postgres and redis images declare no longer leak on every up/down.
- `up` installs an EXIT trap once it starts containers, so any later failure
(including B's port being taken after preflight) tears down A and both
containers; start_gateway returns failures instead of exiting the script,
and the Gateway is disowned so stopping it prints no job notice.
- The overlay removes any checkpointer section (a postgres one would write
checkpoints and the Store into the developer's database), points an
explicit sandbox.ownership and a Redis checkpoint_cache at the harness
Redis (keeping ownership timing), drops DeerMem storage_path (both
spellings) and the blob_storage root so data lands in the harness home,
gives a local AIO sandbox a state-dir-specific container_prefix, and
reports a non-DeerMem memory backend it keeps sharing.
- nginx paths are rendered as quoted string literals ('$' and control
characters are refused), so a state dir with spaces works.
- `check` skips the run-based SSE check when model discovery did not
succeed, unless --with-llm, and enforces the SSE time budget per raw line
so heartbeat-only traffic cannot hold it open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(scripts): keep the harness guidance under the AGENTS.md soft limit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(security): add DEER_FLOW_CREDENTIALS_KEY for at-rest credential encryption
There was no at-rest encryption key. ChannelCredentialCipher had no
production caller: the channel service, the scheduled-task notification
outbox and the channel-connections router all built
ChannelConnectionRepository without a cipher, so store_credentials refused
(RuntimeError) and get_credentials always returned None.
- New harness module deerflow.config.credentials_key: env-only
DEER_FLOW_CREDENTIALS_KEY (comma-separated Fernet keys; first encrypts,
all decrypt via MultiFernet), CredentialsCipher with encrypt_text /
decrypt_text / rotate_text, a fernet:v2: prefix with legacy fernet:v1:
values still readable, and get_credentials_cipher() as the process-wide
entry point. Malformed keys are refused without echoing key material.
Without the variable a single instance generates
{base_dir}/.credentials_key. Never derived from AUTH_JWT_SECRET.
- New deerflow.config.secret_file.read_or_create_secret_file: O_EXCL
create (0600) plus read-back with a bounded settle window, shared by
.credentials_key, .jwt_secret (fixes two replicas cold-starting on a
shared volume keeping different JWT secrets) and the managed-model key.
- ChannelCredentialCipher now builds on CredentialsCipher (same
constructors); the Gateway loads the cipher off the event loop at
startup when channel_connections is enabled and passes it to every
ChannelConnectionRepository. Undecryptable values stay "missing + warn".
- Startup gate: a malformed key stops startup in any topology; a declared
multi-instance deployment with channel_connections enabled and no key
is refused. Plain multi-worker shares the generated file and is allowed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(deploy): generate and inject DEER_FLOW_CREDENTIALS_KEY in Helm, compose and deploy.sh
- Helm: the app Secret generates a Fernet key (randBytes 32 translated to
the urlsafe alphabet) and preserves it across upgrades via lookup like
AUTH_JWT_SECRET; the gateway reads it through an optional secretKeyRef
so a user-managed existingAppSecret without it still boots (the Gateway
gate refuses only multi-instance deployments that store credentials).
README, NOTES and values document backup, rotation and existingAppSecret.
- scripts/deploy.sh resolves the key shell -> .env -> persisted file ->
generated, persisting it as $DEER_FLOW_HOME/.credentials_key (0600,
noclobber/O_EXCL with read-back): the same file the Gateway generates in
its runtime home, so either side's key is the one both keep.
- Both compose files pass DEER_FLOW_CREDENTIALS_KEY=${...:-} to the gateway.
- .env.example entry with a generation hint; README and CHANGELOG notes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(security): make abandoned secret-file replacement single-winner
A rename is last-writer-wins, so replicas that restart together and all
find the same abandoned empty secret file could each replace it and keep
a different value. The process that exclusively creates
<name>.replacing now owns the replacement, re-checks the file is still
empty, and publishes by renaming the claim over it; peers wait for that
value, bounded so a crashed replacer cannot block startup forever.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): link DEER_FLOW_CREDENTIALS_KEY entries to #6611
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(security): name the setting that turns off a credentials-key consumer
The multi-instance refusal ended with "or disable
channel_connections.enabled=true"; it now says "or set
channel_connections.enabled=false".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(security): never expose a secret file before its value is written
O_EXCL published the name before its content, so a creator that stalled
mid-write left an empty file a peer replaced after the settle window; the
resumed creator then returned a value nobody persisted, or deleted the
peer's valid file when its own write failed. Creators now write and sync
a private temporary file and hard-link it to the name, which fails if
the name exists; a failed creator only removes its own temporary file.
Filesystems without hard links publish through the same single-winner
<name>.replacing claim that replaces abandoned empty files, so creation
and replacement are serialized there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(models): recover an empty managed-model key when no catalog exists
An interrupted first save could leave an empty key file; every later save
then passed empty bytes to Fernet and the endpoint returned 503 until the
file was removed by hand. An empty key with no catalog encrypted nothing,
so it now goes through the secret-file recovery path; next to an existing
catalog it is still refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(docker): pass the dotenv DEER_FLOW_CREDENTIALS_KEY through make docker-start
scripts/docker.sh runs Compose from docker/ without --env-file, so the dev
gateway's DEER_FLOW_CREDENTIALS_KEY=${DEER_FLOW_CREDENTIALS_KEY:-} entry
interpolated to empty and overrode the key the checkout .env supplied via
env_file. Export it from the dotenv like AUTH_TRUSTED_PROXIES.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Fail closed on delegated fetch backends to close SSRF gap
* fix(community): resolve delegated-fetch credentials from one config snapshot
Address review feedback on the fail-closed SSRF guard:
- firecrawl/fastcrw client construction no longer re-reads get_app_config(),
so the API key and endpoint come from the same revision the endpoint screen
validated (prevents cross-revision credential misrouting).
- fix the browserless test double to accept the new two-argument
_get_browserless_client signature.
- document the check-only (DNS rebinding) caveat on
validate_delegated_backend_url.
- add a CHANGELOG upgrade note for the default self-hosted backend break.
* fix(doctor): consolidate delegated fetch backend checks and deployment guidance
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(scripts): bind local make dev and make start to loopback
serve.sh and backend/Makefile started the Gateway with --host 0.0.0.0,
nginx.local.conf listened on every interface, and Next.js kept its
all-interfaces default outside Windows, so a local stack on a LAN or VPN
exposed ports 2026, 8001 and 3000, including /setup before the first
admin existed. The Docker stack and the README already document a
loopback-only default.
Bind the Gateway and frontend to 127.0.0.1 and keep nginx on loopback
unless BIND_HOST is set, the variable the Docker stack already honors.
nginx-local-conf.sh renders a copy listening on BIND_HOST, and serve.sh
resolves it before stopping anything so an invalid value cannot tear
down a running stack.
* docs(changelog): reference #6587
* docs(changelog): separate the loopback entry from the next bullet
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(release): reject a stale backend/uv.lock in the version gate
verify_versions.sh compared only Chart.yaml, pyproject.toml and
package.json, so bumping those three by hand passed the v* gate while
backend/uv.lock still recorded the previous root package version. The
backend image installs with uv sync --locked, so the chart and the other
images published while the backend image failed to build.
Run uv lock --check in backend/ as part of the gate, letting uv own the
PEP 440 normalization, and fail when uv is missing. The reusable
verify-versions workflow installs the uv version the backend image pins.
* docs(changelog): reference #6588
* fix(release): explain an unparsed lock version and stop blaming the lock for every uv failure
Print a fallback when the lock's root entry is not found, so the job log
does not show an empty field, and word the uv lock --check annotation so a
network error or uv crash is not reported as a stale lock.
* fix(auth): key the compose login lockout by client, not by nginx
Failed logins are counted per client IP, and the Gateway honors X-Real-IP
only from a peer listed in AUTH_TRUSTED_PROXIES. The compose files never
set it, and every browser request reaches the Gateway from the nginx
container, so all logins shared nginx's address: five wrong passwords
from anyone locked every user out for five minutes.
AUTH_TRUSTED_PROXIES now also accepts hostnames, resolved off the event
loop and cached for 10 seconds (failed lookups included, and only when
X-Real-IP is present), and both compose files default it to the bundled
nginx service, whose address Docker assigns. nginx overwrites X-Real-IP
with $remote_addr on every Gateway route; a test pins that for the three
maintained configs.
* docs(changelog): reference #6519
* fix(auth): share one trusted-proxy lookup across concurrent cache misses
A burst of setup-status / login requests on an expired hostname entry
started one getaddrinfo per request while DNS was slow, occupying the
default executor that password verification also uses. The cache check
and an in-flight task registration now happen without a yield between
them; later callers join that task through asyncio.shield, so a cancelled
request cannot cancel the lookup the others joined, and the task caches
its own result.
The compose default keeps `:-` on purpose: every browser request arrives
through nginx, so an empty AUTH_TRUSTED_PROXIES would only bring back the
shared lockout. The compose comment and .env.example say so, and the
rendered-compose test now covers the explicitly empty value.
* fix(docker): keep a .env AUTH_TRUSTED_PROXIES under make docker-start
docker.sh start runs Compose from docker/ without --env-file, so the dev
compose file's ${AUTH_TRUSTED_PROXIES:-nginx} interpolated from the shell
only, and the resolved environment entry outranked the operator's value
loaded through env_file. Export it from .env alongside the proxy variables
(a shell export still wins), and strip a CRLF before the quotes.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(deploy): fail fast when DEER_FLOW_HOME is not writable
`make docker-start` bind-mounts the host `backend/` directory into the gateway
container, whose process runs as root, so any state it creates under
`backend/.deer-flow` ends up owned by root on the host. A later `make up` runs
`scripts/deploy.sh` as the invoking user and only fails deep into the run with a
bare "Permission denied" while persisting generated secrets (deploy.sh:234).
Check writability right after `mkdir -p "$DEER_FLOW_HOME"` and, when it fails,
print the exact `chown` recovery command instead of letting the run get halfway
through. `make down` is excluded so stopping the stack is never blocked.
Fixes#6462
* test(deploy): cover runtime home writability preflight
* fix(deploy): diagnose persisted secret permissions and preserve teardown
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(wizard): add Opper as an LLM provider
Opper (https://opper.ai) is an EU-hosted AI gateway: 700+ models from
50+ providers behind one OpenAI-compatible API and one key. This adds it
to the setup wizard's LLM_PROVIDERS next to OpenRouter, with the same
shape: langchain_openai:ChatOpenAI pointed at
https://api.opper.ai/v3/compat, env var OPPER_API_KEY.
Models use Opper's bare model ids: claude-sonnet-4-6 (default), gpt-5.5,
gemini-3.8-flash and gpt-5.4-mini. Because the GPT ids are bare,
langchain-openai drops the 0.7 temperature for them the same way it does
for direct OpenAI gpt-5 models.
Checked: python -m py_compile scripts/wizard/providers.py, and each
model id against Opper's public model list
(GET https://api.opper.ai/v3/models). Not run locally: make lint and
make test.
Disclosure: Opper is my startup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs: add Opper to the config example and READMEs
Add an Opper model example next to the other OpenAI-compatible
gateways, as requested in review:
- config.example.yaml: commented opper-claude-sonnet-4-6 block after
the other gateway examples (ChatOpenAI + base_url, OPPER_API_KEY)
- README.md and the translated READMEs: the same entry in the manual
model configuration examples, after OpenRouter
- .env.example and backend/docs/CONFIGURATION.md: OPPER_API_KEY, in the
shape of the Novita lines
claude-sonnet-4-6 is a pooled id in Opper's public catalogue
(GET https://api.opper.ai/v3/models). config.example.yaml still parses
with yaml.safe_load; config_version is unchanged because the schema is.
Disclosure: Opper is my startup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(deploy): let make up start on a checkout without .env files
.env and frontend/.env are gitignored and make up does not create them,
but docker/docker-compose.yaml listed both as short-form env_file entries,
which Compose treats as required. On a fresh clone make up, make down and
make prod-logs all stopped with "env file .../.env not found" (exit 14)
before building or touching anything.
Mark the three entries optional with the long-form path/required syntax
the development compose file already uses. Compose loads them when they
exist and skips them otherwise; an unreadable or non-file .env still
fails. This needs Docker Compose 2.24+, the floor the README already
documents, and deploy.sh's startup-failure hint now names it for older
clients.
deploy.sh deliberately does not seed the files from the examples the way
docker.sh start does: .env.example carries placeholder API keys the
production Gateway would receive, and make config skips files that exist.
* docs(changelog): reference #6370
* test(deploy): gate Compose clients and cover make up
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(make): name what make clean deletes and refuse under a live gateway
`make help` described `make clean` as cleaning up "temporary files", but it
deletes backend/.deer-flow: the local database, users, threads, uploads,
memory, and secrets. Both Docker stacks mount that directory into the
deer-flow-gateway container, and `make stop` only stops local services, so
the data could be deleted under a running Gateway.
Say what is deleted in the help text and before deleting, and refuse while
the deer-flow-gateway container is running. The check runs before `make
stop`, which would otherwise stop the running stack's sandboxes first; a
missing or unreachable Docker passes.
* docs(changelog): reference #6351 for the make clean guard
* fix(make): share make clean's data list and pin stop before the delete
The help line listed "database, threads, uploads, secrets" while the
deletion notice also named users and memory. Define the list once in
RUNTIME_DATA_CONTENTS and use it in both.
Running sandbox containers left behind by `make down` are already stopped
by `make stop` (serve.sh --stop reaps deer-flow-sandbox*), which runs after
the guard and before the delete. Note that in the guard script, and pin
the ordering: the stub stop now records that the data still exists.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(gateway): read uvicorn's WEB_CONCURRENCY in the remaining worker-count readers
* fix(gateway): name the worker spelling an operator can act on
The browser refusal and the dedupe warning both took the count but printed
GATEWAY_WORKERS, so on the launches that pass uvicorn no --workers the advice
silences the gate while the extra processes keep starting. Resolve which
variable set the count and print that, and let an unparsable GATEWAY_WORKERS
fall through to WEB_CONCURRENCY instead of reporting a single worker.
Also stop the comments from claiming the readers are equivalent: they share
the two spellings and the blank-means-unset rule, but channel_connections.py
turns an unparsable value into 0 and refuses while these stay inert.
* docs: state the single-worker requirement under both spellings
The browser tool group and the inbound dedupe warning now refuse or warn on
WEB_CONCURRENCY as well, so guidance that names only GATEWAY_WORKERS=1 is
incomplete for the shipped Dockerfile and serve.sh launches, where uvicorn
takes the process count from WEB_CONCURRENCY.
* test(features): isolate the browser capability surface from the shell's worker count
browser_capability() asks for the Gateway worker count, which this change makes
readable from WEB_CONCURRENCY, so an exported count in the invoking shell turned
test_features_reports_browser_control_enabled_when_configured_and_runtime_available
into a failure that has nothing to do with the router.
