* feat(lark): sidecar credential broker for sandbox lark-cli (Pattern B)
Removes the plaintext Lark credential mounts (appSecret + OAuth tokens)
from the sandbox container. A long-running broker sidecar owns lark-cli
and the per-user config/data dirs and serves the command surface over
Pod loopback; the sandbox gets only a forwarding shim on PATH, so the
raw credential files never exist in the sandbox filesystem.
- lark_broker.py: stdlib-only loopback broker (argv passthrough with
shell=False, server-injected credential env, bounded I/O) + shim
script constant + install-shim mode.
- docker/lark-cli-broker: init(install-shim) + serve image.
- provisioner: LARK_CLI_BROKER_IMAGE + provision_lark_cli_broker →
shim init container + lark-cli-broker sidecar (config/data mounted
sidecar-only); credentials dropped from the sandbox container;
/api/capabilities reports lark_cli_broker_image. Broker supersedes
the Pattern A init-container binary when both are configured.
- gateway: lark_cli_env_overlay(broker=True) omits config/data env;
sandbox_lark_broker_active() TTL-cached mode resolver; broker added
to sandbox_runtime_mode / readiness and the settings UI.
Opt-in and off by default (empty LARK_CLI_BROKER_IMAGE ⇒ no change).
Closes#4338
* fix(lark): address Pattern B broker review findings (#4501)
Follow-up to the sidecar credential broker addressing the PR #4501 review:
- shim: split the on-PATH lark-cli into a /bin/sh launcher + Python shim body
so broker mode fails loudly (exit 127, actionable message) instead of ENOEXEC
when the sandbox image ships no python3; interpreter pinnable via
DEERFLOW_LARK_BROKER_PYTHON. Launcher bakes in the shim's absolute path since
$0 is the bare command name when run off PATH.
- broker: drop the dead cwd payload field (broker can't see the sandbox FS) and
document the command-surface-only / no-file-IO limitation.
- broker: return a structured 500 JSON on unexpected exec errors so the shim
gets a meaningful message, not an opaque transport failure; set a handler
socket timeout to bound slow/stuck connections.
- broker: add an opt-in DEERFLOW_LARK_BROKER_DENY_SUBCOMMANDS denylist that
refuses secret-dumping subcommands before spawning the binary, forwarded from
the provisioner sidecar.
- gateway: tighten the per-bash-call broker probe timeout (1.5s) and cache
negatives longer (300s) so non-broker remote-provisioner users don't pay a
latency hit; guard the mode cache with a lock; drop the dead
_probe_provisioner_lark_cli_init_image wrapper.
- docs: remove the broken design-doc link from the broker README.
Adds tests for launcher python resolution, cwd omission, denylist enforcement,
500-on-error, hot-path probe timeout + negative caching, and provisioner
denylist-env wiring.
* feat: add lark cli integration
* fix: polish lark integration actions
* feat: support lark incremental permissions
* fix: detect lark authorization completion
* fix: harden lark integration install
* feat: expand lark auth scopes and reuse host auth in sandbox
Default lark auth to least-privilege (recommend=false, base sign-in only)
and expose the full set of lark-cli --domain business domains as native
--domain grants instead of a 4-domain read-only mapping. Resolve the
skill pack from the latest larksuite/cli GitHub release at install time
with content-hash integrity, and surface version/runtime drift in status.
Share the per-user lark-cli config/data profile between the Gateway
Settings auth flow and agent conversations by mounting the integration
dirs into the AIO sandbox and injecting the matching env for lark-cli
commands, with an allowlisted extra_mounts path in the provisioner/K8s
backend and traversal guards on integration paths.
* style: fix lint issues from ruff and prettier
Sort imports in the provisioner PVC test and re-wrap two long i18n
description strings to satisfy backend ruff and frontend prettier CI.
* fix(lark): address managed integration review feedback
* fix(frontend): stabilize integrations settings e2e
* test(sandbox): isolate remote backend legacy visibility check
* test: fix backend unit failures after merge
* Harden Lark integration review fixes
* Format Lark integration E2E test
* fix(lark): harden sandbox credential exposure and status disclosure
Address willem_bd's security review on PR #3971:
- Mount the per-user lark-cli config dir (long-lived appSecret) read-only
into the AIO sandbox; only the refreshable-token data dir stays writable.
- Redact host filesystem paths (install_path, cli.path) from
GET /lark/status and the config/auth complete responses for non-admin
callers, fail-closed on any auth error.
- Document the npm postinstall trade-off (--ignore-scripts is not viable
because @larksuite/cli fetches its platform binary in postinstall).
- Document the sandbox credential trust boundary in AGENTS.md and README,
pointing at the sidecar-broker follow-up (#4338).
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: V-001 security vulnerability
Automated security fix generated by OrbisAI Security
* fix(sandbox): wire provisioner API key through backend client and config
RemoteSandboxBackend now accepts an api_key parameter and sends it as
X-API-Key on all five provisioner HTTP calls (list, create, destroy,
is_alive, discover). AioSandboxProvider reads provisioner_api_key from
SandboxConfig and forwards it at construction time. SandboxConfig
formally declares the field; config.example.yaml documents it under
Option 4; docker-compose-dev.yaml threads PROVISIONER_API_KEY into both
the provisioner and gateway containers so a single .env entry covers
both sides.
Tests: monkeypatch PROVISIONER_API_KEY and send X-API-Key headers in the
five parametrized threading tests (previously 401-failing); new
test_auth_middleware asserts /health is open, /api/* rejects no-header
and wrong-key with 401, and accepts the correct key.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(provisioner): correct fail-closed auth docs and fix test mock signatures
The first round of PR #4116 fixes left four blocking issues:
- Mock signatures in test_remote_sandbox_backend.py (17) and
test_aio_sandbox_provider.py (2) didn't accept the new headers= kwarg,
causing TypeError on every provisioner HTTP call test.
- sandbox_config.py and config.example.yaml described the auth as
optional ("leave unset to disable") but the middleware is fail-closed:
an unset PROVISIONER_API_KEY causes 401 on every /api/* request.
- .env.example had no PROVISIONER_API_KEY entry, leaving users with an
empty value and silent 401s.
- No test covered the PROVISIONER_API_KEY="" fail-closed path.
Fix all four: add headers=None to all mock signatures, correct the
field description and example comment to state that both sides must
have the same key set, add PROVISIONER_API_KEY to .env.example with
generation guidance, add test_auth_middleware_unset_key, and add a
logger.warning on auth rejection for observability.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(provisioner): gate legacy skills mount by user visibility
* fix(test): aio sandbox provider
* fix: use shared legacy skill visibility helper for sandbox mounts
* fix(provisioner): keep k8s calls off event loop
* test(provisioner): scan blocking IO by module name
* test(provisioner): exercise create path so BlockBuster has real work
* feat(provisioner): make sandbox container port configurable
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* feat(provisioner): add optional PVC support for sandbox volumes (#1978)
Add SKILLS_PVC_NAME and USERDATA_PVC_NAME env vars to allow sandbox
Pods to use PersistentVolumeClaims instead of hostPath volumes. This
prevents data loss in production when pods are rescheduled across nodes.
When USERDATA_PVC_NAME is set, a subPath of threads/{thread_id}/user-data
is used so a single PVC can serve multiple threads. Falls back to hostPath
when the new env vars are not set, preserving backward compatibility.
* add unit test for provisioner pvc volumes
* refactor: extract shared provisioner_module fixture to conftest.py
Agent-Logs-Url: https://github.com/bytedance/deer-flow/sessions/e7ccf708-c6ba-40e4-844a-b526bdb249dd
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: JeffJiang <for-eleven@hotmail.com>
* Adds Kubernetes sandbox provisioner support
* Improves Docker dev setup by standardizing host paths
Replaces hardcoded host paths with a configurable root directory,
making the development environment more portable and easier to use
across different machines. Automatically sets the root path if not
already defined, reducing manual setup steps.
