mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-27 13:40:37 +00:00
* fix(security): add DEER_FLOW_CREDENTIALS_KEY for at-rest credential encryption
There was no at-rest encryption key. ChannelCredentialCipher had no
production caller: the channel service, the scheduled-task notification
outbox and the channel-connections router all built
ChannelConnectionRepository without a cipher, so store_credentials refused
(RuntimeError) and get_credentials always returned None.
- New harness module deerflow.config.credentials_key: env-only
DEER_FLOW_CREDENTIALS_KEY (comma-separated Fernet keys; first encrypts,
all decrypt via MultiFernet), CredentialsCipher with encrypt_text /
decrypt_text / rotate_text, a fernet:v2: prefix with legacy fernet:v1:
values still readable, and get_credentials_cipher() as the process-wide
entry point. Malformed keys are refused without echoing key material.
Without the variable a single instance generates
{base_dir}/.credentials_key. Never derived from AUTH_JWT_SECRET.
- New deerflow.config.secret_file.read_or_create_secret_file: O_EXCL
create (0600) plus read-back with a bounded settle window, shared by
.credentials_key, .jwt_secret (fixes two replicas cold-starting on a
shared volume keeping different JWT secrets) and the managed-model key.
- ChannelCredentialCipher now builds on CredentialsCipher (same
constructors); the Gateway loads the cipher off the event loop at
startup when channel_connections is enabled and passes it to every
ChannelConnectionRepository. Undecryptable values stay "missing + warn".
- Startup gate: a malformed key stops startup in any topology; a declared
multi-instance deployment with channel_connections enabled and no key
is refused. Plain multi-worker shares the generated file and is allowed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(deploy): generate and inject DEER_FLOW_CREDENTIALS_KEY in Helm, compose and deploy.sh
- Helm: the app Secret generates a Fernet key (randBytes 32 translated to
the urlsafe alphabet) and preserves it across upgrades via lookup like
AUTH_JWT_SECRET; the gateway reads it through an optional secretKeyRef
so a user-managed existingAppSecret without it still boots (the Gateway
gate refuses only multi-instance deployments that store credentials).
README, NOTES and values document backup, rotation and existingAppSecret.
- scripts/deploy.sh resolves the key shell -> .env -> persisted file ->
generated, persisting it as $DEER_FLOW_HOME/.credentials_key (0600,
noclobber/O_EXCL with read-back): the same file the Gateway generates in
its runtime home, so either side's key is the one both keep.
- Both compose files pass DEER_FLOW_CREDENTIALS_KEY=${...:-} to the gateway.
- .env.example entry with a generation hint; README and CHANGELOG notes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(security): make abandoned secret-file replacement single-winner
A rename is last-writer-wins, so replicas that restart together and all
find the same abandoned empty secret file could each replace it and keep
a different value. The process that exclusively creates
<name>.replacing now owns the replacement, re-checks the file is still
empty, and publishes by renaming the claim over it; peers wait for that
value, bounded so a crashed replacer cannot block startup forever.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): link DEER_FLOW_CREDENTIALS_KEY entries to #6611
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(security): name the setting that turns off a credentials-key consumer
The multi-instance refusal ended with "or disable
channel_connections.enabled=true"; it now says "or set
channel_connections.enabled=false".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(security): never expose a secret file before its value is written
O_EXCL published the name before its content, so a creator that stalled
mid-write left an empty file a peer replaced after the settle window; the
resumed creator then returned a value nobody persisted, or deleted the
peer's valid file when its own write failed. Creators now write and sync
a private temporary file and hard-link it to the name, which fails if
the name exists; a failed creator only removes its own temporary file.
Filesystems without hard links publish through the same single-winner
<name>.replacing claim that replaces abandoned empty files, so creation
and replacement are serialized there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(models): recover an empty managed-model key when no catalog exists
An interrupted first save could leave an empty key file; every later save
then passed empty bytes to Fernet and the endpoint returned 503 until the
file was removed by hand. An empty key with no catalog encrypted nothing,
so it now goes through the secret-file recovery path; next to an existing
catalog it is still refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(docker): pass the dotenv DEER_FLOW_CREDENTIALS_KEY through make docker-start
scripts/docker.sh runs Compose from docker/ without --env-file, so the dev
gateway's DEER_FLOW_CREDENTIALS_KEY=${DEER_FLOW_CREDENTIALS_KEY:-} entry
interpolated to empty and overrode the key the checkout .env supplied via
env_file. Export it from the dotenv like AUTH_TRUSTED_PROXIES.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
83 lines
3.0 KiB
Python
83 lines
3.0 KiB
Python
"""Authentication configuration for DeerFlow."""
|
|
|
|
import logging
|
|
import os
|
|
import secrets
|
|
|
|
from pydantic import BaseModel, Field
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
_SECRET_FILE = ".jwt_secret"
|
|
|
|
|
|
class AuthConfig(BaseModel):
|
|
"""JWT and auth-related configuration. Parsed once at startup.
|
|
|
|
Note: the ``users`` table now lives in the shared persistence
|
|
database managed by ``deerflow.persistence.engine``. The old
|
|
``users_db_path`` config key has been removed — user storage is
|
|
configured through ``config.database`` like every other table.
|
|
"""
|
|
|
|
jwt_secret: str = Field(
|
|
...,
|
|
description="Secret key for JWT signing. MUST be set via AUTH_JWT_SECRET.",
|
|
)
|
|
token_expiry_days: int = Field(default=7, ge=1, le=30)
|
|
oauth_github_client_id: str | None = Field(default=None)
|
|
oauth_github_client_secret: str | None = Field(default=None)
|
|
|
|
|
|
_auth_config: AuthConfig | None = None
|
|
|
|
|
|
def _load_or_create_secret() -> str:
|
|
"""Load persisted JWT secret from ``{base_dir}/.jwt_secret``, or generate and persist a new one.
|
|
|
|
The file is created exclusively and read back, so processes that share the
|
|
runtime home and cold-start together (uvicorn workers, replicas on a shared
|
|
volume) converge on one secret instead of each keeping its own.
|
|
"""
|
|
from deerflow.config.paths import get_paths
|
|
from deerflow.config.secret_file import read_or_create_secret_file
|
|
|
|
paths = get_paths()
|
|
secret_file = paths.base_dir / _SECRET_FILE
|
|
|
|
try:
|
|
return read_or_create_secret_file(secret_file, lambda: secrets.token_urlsafe(32))
|
|
except OSError as exc:
|
|
raise RuntimeError(f"Failed to read or persist the JWT secret at {secret_file}. Set AUTH_JWT_SECRET explicitly or fix DEER_FLOW_HOME/base directory permissions so DeerFlow can keep a stable auth secret.") from exc
|
|
|
|
|
|
def get_auth_config() -> AuthConfig:
|
|
"""Get the global AuthConfig instance. Parses from env on first call."""
|
|
global _auth_config
|
|
if _auth_config is None:
|
|
from dotenv import load_dotenv
|
|
|
|
from deerflow.env import load_selected_env_file
|
|
|
|
if not load_selected_env_file():
|
|
load_dotenv()
|
|
jwt_secret = os.environ.get("AUTH_JWT_SECRET")
|
|
if not jwt_secret:
|
|
jwt_secret = _load_or_create_secret()
|
|
os.environ["AUTH_JWT_SECRET"] = jwt_secret
|
|
logger.warning(
|
|
"⚠ AUTH_JWT_SECRET is not set — using an auto-generated secret "
|
|
"persisted to .jwt_secret. Sessions will survive restarts. "
|
|
"For production, add AUTH_JWT_SECRET to your dotenv file "
|
|
"(the DEER_FLOW_ENV_FILE target, or .env by default): "
|
|
'python -c "import secrets; print(secrets.token_urlsafe(32))"'
|
|
)
|
|
_auth_config = AuthConfig(jwt_secret=jwt_secret)
|
|
return _auth_config
|
|
|
|
|
|
def set_auth_config(config: AuthConfig) -> None:
|
|
"""Set the global AuthConfig instance (for testing)."""
|
|
global _auth_config
|
|
_auth_config = config
|