Zeren Wang d700ad9edf
feat(memory): add Jev pre-screening and signal classification over a shared TypeSafe client (#5906)
* feat(memory): add Jev pre-screening and signal classification over a shared TypeSafe client

- refactor(typesafe): extract a shared Jev (TypeSafe) client and unify its
  configuration: one connection-precedence rule, credential fingerprint, wire-size
  counting and error taxonomy, shared by the guardrail gate and the memory hooks.
- feat(memory): wire `memory.prescreen` and `memory.signal_classification` into
  DeerMem through one injected judge hook (both off by default, `enforce` gated by
  the shadow evaluation).
- config: add the top-level `typesafe:` block and the two judging slots to
  `config.example.yaml` and the Helm values, bump `config_version` to 50, and sync
  the Helm README example with it.
- test(memory): add the shadow-evaluation script (`eval_memory_prescreen.py`) with
  per-stratum gates, censoring and the no-network baseline.

Review follow-ups folded in:
- `mutations_accepted` counts only facts that survive capacity enforcement and the
  upload-event scrub that precedes persistence, only removals/reinforcements of ids
  that existed, and persisted near-duplicate merges (matched by id, or by identity
  for legacy id-less facts); it is published only after persistence succeeds.
- The cached manager hot-reloads its judge when a memory slot or the shared
  `typesafe` block changes (including a credential rotation); publication is
  serialized and revalidates the config generation, so a build started before an
  `enforce` -> `off` rollback cannot land after it.
- Cache provenance is per answer: the `cached` flag and the reported model follow
  the answers a side actually consumed, across partial merges and answerless retries.
- An oversized noul integer stays a per-question error instead of raising.
- Incremental call savings use the shared scored denominator.
- The sensitive-stratum gate requires evidence for each of identity, preference
  and correction.

* fix(typesafe): resolve one credential per layer and refuse unusable connection values

Review follow-ups on the shared TypeSafe client (#5906):

- A consumer's `api_key_env` is no longer overridden by a literal `api_key` in the
  shared `typesafe:` block: the first layer that sets either credential setting
  decides it, and inside one layer the literal key wins (the standalone
  semantics). A consumer whose endpoint does not accept the block's key no longer
  sends it on every call, and the missing-credential message keeps naming the
  variable that layer configured.
- `base_url` is validated where it is configured: `http(s)` only, with no query,
  fragment, embedded credentials, whitespace or control characters, and a
  malformed port fails at construction instead of on every request (httpx raises
  `UnsupportedProtocol` / `InvalidURL` per call, and the query would have been
  swallowed into the appended endpoint path).
- A credential that cannot travel as a header value — surrounding whitespace from
  a secret mounted from a file, a non-printable character — is refused when the
  connection resolves. It used to construct fine and then put the whole
  `Bearer <key>` value into h11's `LocalProtocolError` message, which the
  guardrail middleware's `logger.exception` prints.
- The client raises transport failures without chaining the original exception,
  so that message can no longer carry the credential through the cause chain; the
  exception type stays in the message for diagnosis.
- `TypeSafeConfig` validates in strict mode, agreeing with the consumer `config`
  path: `max_attempts: true` was silently read as `1`. An integer stays a usable
  float, so `timeout: 5` keeps working.

Tests: 20 cases across `tests/test_typesafe_config.py` and
`tests/test_typesafe_client.py`, each red on the previous revision and green now.
367 focused typesafe/memory/config tests, `make test-blocking-io` (167) and the
eval-script suites (50) pass; `ruff check` and `ruff format --check` are clean.
Docs synced: `backend/docs/GUARDRAILS.md`, `typesafe/AGENTS.md`,
`config.example.yaml`.

* fix(typesafe,memory): bound the reply, record request failures, gate enforce on the evidence

Second review round on #5906: the remaining items from the review thread and from
the follow-up pass.

Typesafe client (the reply is treated as hostile input too):

- Send `Accept-Encoding: identity`, refuse a body that carries a non-identity
  `Content-Encoding`, and read through a bounded `iter_bytes` that stops past
  `MAX_RESPONSE_BYTES` (64 KiB). A gzip body that expands to 128 MiB used to be
  decoded in full and, on the gate path, allowed the tool call unexamined; it is
  now `invalid_response` (a denial with `fail_closed: true`).
- Map `RecursionError` from `json.loads` to `invalid_response`: deep nesting
  escaped the module's taxonomy, so the memory consumers lost the round's record
  instead of recording a fallback.

Memory judge:

- A request-level failure is recorded as `request_failed`, never `no_verdict`
  (`REASON_REQUEST_FAILED` was defined and never emitted, so an unreachable
  endpoint was indistinguishable from a provider with no usable answer). The
  adapters' `decide()` now propagates `TypeSafeError` for the coordinator to record
  and log; `no_verdict` stays for a question-level failure, and an ineligible side
  keeps its own reason even when the round's request failed.

Shadow evaluation (the enablement gate reads the evidence, not one number):

- The miss-rate gate requires the exact one-sided Clopper-Pearson upper bound to
  meet the 1% target; a sample that only meets the point estimate is
  `INSUFFICIENT` (0/250 was reported as a pass with a 1.19% bound).
- New `baseline_incremental_savings` gate: a baseline that saves as many calls as
  the model — the shipped all-trivial sample — fails instead of enabling.
- Negative counts (`mutations_accepted`, message / batch / token counts) are
  treated as absent rather than scored: `-3` read as "nothing lost", adding a
  phantom saving to the numerator and the denominator.

Docs: the design documents cited but absent from the repository are replaced with
in-repo pointers (`MEMORY_IMPROVEMENTS.md`, `agents/memory/AGENTS.md`,
`GUARDRAILS.md`), and the gate and limit wording now matches what the code
enforces.

Tests: 31 new or updated cases, each red on the previous revision (compressed body
refused even when it decodes to a valid envelope, body over the cap refused, body
at the cap answered, deep nesting mapped, `request_failed` on each request-level
failure, the bound and baseline gates, the negative counter). 454 focused tests,
`make test-blocking-io` (167) and the eval suites pass; `ruff check` and
`ruff format --check` are clean.

* fix(memory-eval): pair the baseline comparison on the same records

Review follow-up on the incremental-savings gate added in ed930478 (P2): both
sides of the comparison are now the records the no-network baseline can score
(`trivial_only` plus a readable signals list).

Subtracting that subset's heuristic savings from model savings over *all* scorable
skips credited the model with batches the baseline never saw: 300 reviewed skips
with a single annotated record computed 300-1=299 "incremental", passed the gate
and set `enable_recommended`, while the paired comparison was model=1 against
heuristic=1 (incremental 0).

- `Evidence` carries the paired numbers (`baseline_paired_skips`,
  `baseline_model_saved_calls`, `baseline_heuristic_saved_calls`) instead of one
  heuristic count, filled by `build_report` from the baseline summary.
- The gate reports `INSUFFICIENT` when no scored skip carries the annotation and
  when the annotation covers only a subset of the scored skips (the message carries
  the paired counts and the `paired of scored` coverage), `FAIL` on a non-positive
  paired gain, and `PASS` only on a positive gain over the full paired set.
- Docs state the pairing/coverage requirement in
  `backend/docs/MEMORY_IMPROVEMENTS.md` and in the script docstring.

Tests: the reviewer's counterexample as a report-level regression (it failed with
`assert 'PASS' == 'INSUFFICIENT'` on ed930478 while `enable_recommended` was true),
a unit case for the paired-subset `INSUFFICIENT`, and the renamed paired fields.
344 focused tests, `make test-blocking-io` (167) and the eval suites pass;
`ruff check` / `ruff format --check` clean.

* fix(typesafe): bound the sync body read by the deadline

`deadline_seconds` only covered the sync path's headers, the completed body and
the parse, so an endpoint that drips bytes just under the read timeout held the
caller for one read per piece — the whole of `MAX_RESPONSE_BYTES` past the budget.
The read loop now checks the budget before every transport read, which bounds the
overshoot to one read. The pieces are the transport's own reads, not re-chunked
output: `iter_bytes(chunk_size)` pulls many reads inside a single yielded chunk,
so a check at that granularity would never see a drip.

`tests/test_typesafe_client.py` pins it with a valid envelope delivered as 24
pieces of 50 ms against a 0.2 s deadline: red before (24 pieces pulled), green now
(~5), on both the sync and async paths. Docstring and `typesafe/AGENTS.md` track
the new granularity.

* fix(memory): read the YAML off spelling, and veto a skip over the whole batch

Two review findings on the memory judging path.

`memory.prescreen.mode` / `memory.signal_classification.mode` rejected the
`mode: off` spelling `config.example.yaml` ships and a rollback types: YAML 1.1
parses the unquoted value as a boolean, so both fields saw `False`. The failed
reload left the previous judge running, so both fields now normalize `False` to
`off` before the `Literal` runs. `True` stays a validation error (no mode it
could mean). The examples quote the value, and the docs record why.

The pre-screen's L3 veto read `detect_signals`' 6-message window while the batch
it answers for is the whole post-watermark feed, so an explicit correction in an
earlier turn of an eight-message batch did not stop the skip that then advanced
the watermark past it. `detect_signals` gains a `window` parameter (default
unchanged) and the updater scans the full feed for the veto; the extraction hint
and the reinforcement gate keep the window.

Both regressions are red on the previous head: the config test reproduced the
`input_value=False` rejection through `AppConfig.from_file`, and the batch test
reproduced the prescreen judging (and skipping) a batch whose first turn carries
the correction.
2026-09-27 20:28:05 +08:00
..

DeerFlow Helm Chart

Deploys the full DeerFlow stack to Kubernetes: gateway (backend + embedded LangGraph runtime), frontend (Next.js), nginx (internal reverse proxy preserving the compose routing), and the provisioner (K8s-native sandbox that spawns code-execution Pods on demand).

This chart translates the production docker/docker-compose.yaml into native Kubernetes resources. No existing repo files are modified.

Prerequisites

  • A Kubernetes cluster (Docker Desktop K8s, OrbStack, kind, k3d, or a real cluster).
  • kubectl + helm 3.8+ installed (OCI registry support stabilized in 3.8; earlier 3.x needs HELM_EXPERIMENTAL_OCI=1).
  • The three DeerFlow images — either the published ones (see "Install the published chart" below) or built locally (see step 1).
  • An Ingress controller (e.g. ingress-nginx) if you enable ingress.

Install the published chart (GHCR)

The chart and all three images are published to GHCR on every v* release tag (see .github/workflows/container.yaml and chart.yaml). Skip the build step and install directly:

helm install deer-flow oci://ghcr.io/<owner>/charts/deer-flow \
  --version <version> \
  -n deer-flow --create-namespace \
  -f my-values.yaml

where <owner> is the GitHub owner the chart is published from and <version> matches the release tag without the leading v (tag v0.1.0 → --version 0.1.0).

Note: the helm chart is new in 2.1.0 - no chart was published before it. It publishes to oci://ghcr.io/<owner>/charts/deer-flow (the charts/ prefix keeps it distinct from the deer-flow-{backend,frontend,provisioner} image packages).

Point the chart at the published images:

image:
  registry: ghcr.io/<owner>     # owner prefix; images are <owner>/deer-flow-<name>
  tag: "<version>"              # match the release tag (sans leading `v`)
  pullSecrets:
    - { name: regcred }         # only if the GHCR package is private

The chart's gatewayImage / frontendImage / provisionerImage defaults already match the published image names (deer-flow-backend, deer-flow-frontend, deer-flow-provisioner), so only registry and tag are required. New GHCR packages default to private — flip the package to public in its GHCR settings page for unauthenticated pulls, otherwise create a pull secret (step 1) and reference it via image.pullSecrets.

The OCI chart and the images are versioned independently of the chart's appVersion; always set image.tag to the release that matches your chart --version unless you have a reason to pin differently.

1. Build & push images (custom builds only)

Skip this section if you're using the published chart above. To build the images yourself from the existing Dockerfiles:

REGISTRY=ghcr.io/yourorg
TAG=latest

# backend - build with the `postgres` extra so multi-replica deploys can use
# shared Postgres (matches the published image)
docker build -t $REGISTRY/deer-flow-backend:$TAG --build-arg UV_EXTRAS=postgres -f backend/Dockerfile .
# frontend
docker build -t $REGISTRY/deer-flow-frontend:$TAG -f frontend/Dockerfile .
# provisioner
docker build -t $REGISTRY/deer-flow-provisioner:$TAG -f docker/provisioner/Dockerfile docker/provisioner

docker push $REGISTRY/deer-flow-backend:$TAG
docker push $REGISTRY/deer-flow-frontend:$TAG
docker push $REGISTRY/deer-flow-provisioner:$TAG

These names match the chart's gatewayImage / frontendImage / provisionerImage defaults, so only image.registry and image.tag need to point at them.

If your registry needs auth, create a pull secret:

kubectl create secret docker-registry regcred \
  --docker-server=ghcr.io \
  --docker-username=youruser \
  --docker-password=yourtoken \
  -n deer-flow

2. Configure values

Copy and edit values.yaml → my-values.yaml. At minimum set:

image:
  registry: ghcr.io/yourorg
  tag: latest
  pullSecrets:
    - { name: regcred }

ingress:
  enabled: true
  className: nginx
  host: deer-flow.example.com
  tls:
    enabled: true
    secretName: deer-flow-tls

secrets:
  OPENAI_API_KEY: sk-...
  # add channel tokens, search keys, etc. as needed

The default ingress annotations permit a 100 MiB local .skill archive plus multipart framing, stream request bodies without ingress buffering, and allow up to 600 seconds for a response, which the API requests that wait on a model call or a whole run all need — skill install and custom-skill edits (each file is scanned by an LLM), /api/threads/{id}/compact, /api/input-polish, and /api/runs/wait. If you replace ingress.annotations, preserve equivalent size, streaming, and response-timeout settings for your ingress controller, or local skill uploads may fail before DeerFlow completes the installation and those requests may time out while Gateway is still working — for /api/runs/wait the disconnect also cancels the run.

Provide your model config under config (keep secrets as $VAR references — they resolve from the secrets map):

config: |
  config_version: 50
  models:
    - name: gpt-4
      use: langchain_openai:ChatOpenAI
      model: gpt-4
      api_key: $OPENAI_API_KEY
      request_timeout: 600.0
  sandbox:
    use: deerflow.community.aio_sandbox:AioSandboxProvider
    provisioner_url: http://provisioner:8002
  database:
    backend: postgres
    postgres_url: $DATABASE_URL
    pool_recycle: 300
    command_timeout: 30
  checkpointer:
    type: postgres
    connection_string: $DATABASE_URL
  stream_bridge:
    type: redis   # cross-pod SSE; URL from DEER_FLOW_STREAM_BRIDGE_REDIS_URL
  knowledge_base:
    enabled: true
    scope_selection_enabled: false
    # Provider connection/retrieval settings belong on the knowledge_search tool.
  # Tools MUST be listed explicitly - the agent gets none otherwise
  # (BUILTIN_TOOLS only adds present_file + ask_clarification). The chart
  # default in values.yaml enables the sandbox tools + web tools (web_search,
  # web_fetch, image_search - no API key); when you override `config:`, copy
  # them in. Full list in values.yaml / config.example.yaml. The web tools need
  # outbound egress from the gateway pod.
  tool_groups:
    - name: web
    - name: file:read
    - name: file:write
    - name: bash
    - name: knowledge
  tools:
    - name: web_search
      group: web
      use: deerflow.community.ddg_search.tools:web_search_tool
      max_results: 5
    - name: web_fetch
      group: web
      use: deerflow.community.jina_ai.tools:web_fetch_tool
      timeout: 10
    - name: image_search
      group: web
      use: deerflow.community.image_search.tools:image_search_tool
      max_results: 5
    - name: knowledge_search
      group: knowledge
      use: deerflow.community.ragflow.tools:knowledge_search_tool
      base_url: http://ragflow:9380
      api_key: $RAGFLOW_API_KEY
    - name: list_knowledge_bases
      group: knowledge
      use: deerflow.community.ragflow.tools:list_knowledge_bases_tool
    - name: bash
      group: bash
      use: deerflow.sandbox.tools:bash_tool
    # also: ls, read_file, glob, grep, write_file, str_replace (see values.yaml)

$DATABASE_URL is injected from the postgres Secret (see below). The checkpointer: section is required for multi-replica operation — the LangGraph Store (cross-thread memory + thread list) reads it and does not fall back to database:. stream_bridge.type: redis is the default and routes live SSE events through the bundled redis StatefulSet (or redis.external). Because config: is a single override blob, a partial config: replaces the chart default entirely - keep the tools:/tool_groups: block (or the agent will have no tools) and the sandbox:/database:/checkpointer:/stream_bridge: sections shown above.

extensionsConfig is an initial seed, not a live read-only mount. An init container copies it into /app/backend/.deer-flow/extensions-config/extensions_config.json, where the Gateway can persist MCP and skill-state API updates. With persistence.home.enabled: true, the runtime file is kept on the home PVC and is not overwritten by later Helm upgrades; delete that runtime file before a pod restart only when you intentionally want a changed extensionsConfig seed to replace it. With persistence disabled, the writable copy uses emptyDir and is reseeded whenever the Pod is replaced.

3. Install (from a local chart checkout)

For a custom build or local development, install from the chart directory:

helm install deer-flow deploy/helm/deer-flow \
  -n deer-flow --create-namespace \
  -f my-values.yaml

4. Verify

kubectl -n deer-flow get pods
kubectl -n deer-flow port-forward svc/nginx 2026:2026
curl http://localhost:2026/health          # gateway health via nginx

Hit the Ingress host (map it in /etc/hosts for local clusters) to load the UI.

Provisioner sanity check:

kubectl -n deer-flow exec deploy/deer-flow-provisioner -- curl -s localhost:8002/health

Architecture notes

  • PostgreSQL is the default database. A bundled single-instance postgres StatefulSet (postgresql.enabled: true) runs in the namespace and the gateway connects via the in-cluster Service. The DSN is auto-generated into a Secret (key database-url) and injected as DATABASE_URL; config.yaml references it as $DATABASE_URL in database.postgres_url. Schema is bootstrapped automatically on gateway startup (alembic create_all + stamp head). For real HA, disable the bundled instance and point at a managed DB:
    postgresql:
      enabled: false
      external:
        host: mydb.example.com   # or set databaseUrl / existingSecret
        port: 5432
        database: deerflow
        username: deerflow
        password: changeme
    
  • Graceful shutdown & memory drain. The gateway pod sets terminationGracePeriodSeconds (default 45s, overridable via gateway.terminationGracePeriodSeconds) plus an optional preStop sleep (gateway.preStopSleepSeconds, default 5s). The grace period MUST exceed the Gateway's graceful-shutdown work — channel stop (~5s) plus the memory-queue drain (memory.shutdown_flush_timeout_seconds, default 30s) plus a buffer — because the drain runs on a daemon thread and K8s SIGKILLs anything still running at the end of the grace window. K8s defaults to 30s, which SIGKILLs the drain mid-flight and silently re-introduces the memory loss the drain is fixing. When you raise memory.shutdown_flush_timeout_seconds, raise gateway.terminationGracePeriodSeconds to match (channel stop + drain + buffer).
  • Gateway replicas. Postgres + the Redis stream bridge together make the gateway's persisted state (checkpointer + run/thread metadata) and live stream path cross-pod-safe. The default is still 1 replica: do not raise gateway.replicas past 1 yet. Run control — create_or_reject dedup, cancel, and orphan reconciliation — is still worker-local (in-process asyncio.Lock + in-memory record.task), tracked by issue #3948. With >1 replica a double-submit can create two runs on one thread (checkpoint corruption), a cancel can land on a non-owner pod (409), and a crashed pod's runs stay pending/running forever. Stay on 1 replica until that work lands.
  • Scheduled task recovery. If a deployment explicitly enables scheduler.multi_instance: true, it must use shared Postgres, run_ownership.heartbeat_enabled: true, and run_events.backend: db. Scheduler startup then preserves live scheduled runs owned by another Pod, atomically takes over only expired leases, and fences stale post-launch bookkeeping. max_concurrent_runs is a shared global cap across Pods, including pre-launch dispatch reservations. Restart all Gateway Pods after changing these startup-only settings. This does not remove the broader Gateway replica limitations described above.
  • Redis stream bridge. A bundled single-instance redis StatefulSet (redis.enabled: true, redis:7-alpine) runs in the namespace and the gateway connects via the in-cluster Service. Per-run SSE events are stored in Redis Streams (PR #3191) so a client connected to any gateway pod receives live events and reconnect resumes from Last-Event-ID. The URL is auto-generated into a Secret (key redis-url) and injected as DEER_FLOW_STREAM_BRIDGE_REDIS_URL; config.yaml sets stream_bridge.type: redis by default. No-auth by default (ClusterIP isolation, matching compose); set redis.auth.password to enable AUTH. For a managed Redis, disable the bundled instance and point at it via redis.external.
  • Persistence. A PVC (<release>-home) backs /app/backend/.deer-flow (sqlite DB, memory, custom agents, per-thread user-data). The gateway mounts it with subPath: deer-flow so the layout matches the provisioner's PVC user-data mode. Default ReadWriteOnce; use ReadWriteMany (NFS) on multi-node clusters so sandbox Pods on other nodes can mount it.
  • Provisioner RBAC. The provisioner gets a ServiceAccount with a namespaced Role (get/list/watch/create/delete on pods + services) and a narrow ClusterRole (namespace get/create). It uses in-cluster service-account creds — no kubeconfig mount. The unused update/patch/pods-exec/events verbs were dropped (audited against docker/provisioner/app.py).
  • Skills. Disabled by default (emptyDir at /app/skills). Populate via skills.existingClaim or skills.configMap, or bake skills into a custom gateway image.

Security

Enforced posture

All workloads run as non-root with all Linux capabilities dropped. No container escalates privileges or runs as uid 0.

workload runAsUser fsGroup writable-path handling
gateway 1000 1000 .deer-flow PVC group-writable via fsGroup; PYTHONDONTWRITEBYTECODE=1 suppresses .pyc writes; UV_CACHE_DIR=/tmp
frontend 1000 (node) 1000 emptyDir at /app/frontend/.next/cache (root-owned in the image)
nginx 101 (nginx) 101 command writes the rendered config to /tmp/nginx.conf and loads nginx -c /tmp/nginx.conf (since /etc/nginx is root-owned); emptyDir at /var/cache/nginx
provisioner 1000 — no PVC; PYTHONDONTWRITEBYTECODE=1
postgres 999 (postgres) 999 official postgres:16 entrypoint detects non-root and skips the chown/gosu dance; data PVC group-writable via fsGroup
redis 999 (redis) 999 official redis:7-alpine entrypoint detects non-root and skips the gosu dance; data PVC group-writable via fsGroup

Every container sets:

  • runAsNonRoot: true
  • allowPrivilegeEscalation: false
  • capabilities.drop: ["ALL"]
  • seccompProfile: { type: RuntimeDefault }

All listening ports are >1024 (8001 / 3000 / 2026 / 8002 / 5432), so no NET_BIND_SERVICE capability is required.

ConfigMap rollout. ConfigMaps mount via subPath, which does not receive in-place updates — a helm upgrade that changes only a ConfigMap would leave pods on stale config. Each pod template carries a checksum/* annotation (SHA256 of the rendered ConfigMap): checksum/config + checksum/extensions on the gateway, checksum/nginx on nginx. Any content change alters the pod spec and triggers a rolling restart.

Resource defaults. Every workload ships with modest requests+limits in values.yaml; override per workload (gateway.resources, frontend.resources, nginx.resources, provisioner.resources, postgresql.primary.resources, redis.primary.resources).

Not yet enforced (deferred hardening)

These are intentionally not set in this chart revision. Each can be added per-workload with testing:

  • readOnlyRootFilesystem: true — makes the container's root filesystem immutable so a compromised process can't persist changes to the image. Not enabled because it requires auditing every runtime write path and mounting an emptyDir over each. Known paths:
    • gateway / frontend / nginx / provisioner: /tmp (uv cache, python tempfiles, the nginx config + pid, node temp) — one emptyDir at /tmp each.
    • postgres: /tmp and /var/run/postgresql (the Unix-socket dir). The first four are mechanical. postgres is the hard case — the official image writes its socket to /var/run/postgresql and isn't designed for a read-only root, so it may need socket-path redirection (PGHOST/unix_socket_directories). Optionally, add USER directives to the backend/Dockerfile, frontend/Dockerfile, and docker/provisioner/Dockerfile so the images are non-root by default (defense in depth — the chart already forces the uid via securityContext, so this is not required). A cluster enforcing the restricted Pod Security Admission standard would require this setting.
  • Provisioner RBAC narrowing. The Role grants get/list/watch/create/delete on pods and services in the namespace (update/patch/pods-exec/events were dropped as unused). These verbs still apply to all Pods in the namespace, not just sandbox Pods — RBAC can't scope by label, so the remaining options are a dedicated sandbox namespace or admission control (OPA/Kyverno).
  • startupProbe. Workloads have readiness + liveness probes but no startup probe. The gateway's livenessProbe.initialDelaySeconds: 30 covers slow starts today; a startupProbe would let it take arbitrarily long to initialize without risking a liveness kill during a cold start (e.g. slow model config load).

None of these affect correctness of the current deployment.

Migrating an existing volume to non-root

fsGroup does not apply to subPath mounts, and it changes group ownership but not file mode — so a PVC written by an earlier root run (e.g. a cluster that ran the gateway as root before enabling this hardening, or a backup restore of root-owned files) will keep files like .jwt_secret at 0600 root:root. The non-root gateway (uid 1000) then can't read them and crashes on the first auth request with RuntimeError: Failed to read JWT secret from .../​.jwt_secret.

Fresh installs are unaffected — uid 1000 creates every file as 1000:1000.

To fix an existing root-written PVC, run a one-shot root pod that chowns the volume to the gateway uid (1000), then restart the gateway:

cat <<'EOF' | kubectl apply -n deer-flow -f -
apiVersion: v1
kind: Pod
metadata: { name: fix-home-perms, namespace: deer-flow }
spec:
  restartPolicy: Never
  containers:
    - name: chown
      image: busybox:1.36
      command: ["sh", "-c"]
      args: ["chown -R 1000:1000 /home-pvc/deer-flow && chmod -R g+rwX /home-pvc/deer-flow"]
      volumeMounts:
        - { name: home, mountPath: /home-pvc }
  volumes:
    - name: home
      persistentVolumeClaim: { claimName: deer-flow-deer-flow-home }
EOF
kubectl -n deer-flow wait --for=condition=Ready pod/fix-home-perms --timeout=30s
kubectl -n deer-flow delete pod fix-home-perms
kubectl -n deer-flow rollout restart deploy/deer-flow-deer-flow-gateway

(On a single-node cluster the fix pod can mount the RWO PVC concurrently with the gateway; on multi-node, scale the gateway to 0 first.) A durable alternative — an opt-in root volumePermissions initContainer that chowns on every start (the Bitnami pattern) — is not yet wired into this chart; it would introduce a root container, so it's left as an operator decision for now.

Sandbox Service type

The provisioner exposes each sandbox Pod behind a per-sandbox Service whose type is controlled by provisioner.sandboxServiceType (default ClusterIP).

ClusterIP (default, recommended). The provisioner returns a cluster-DNS URL - http://sandbox-<id>-svc.<namespace>.svc.cluster.local:8080 - so the gateway reaches its sandbox entirely inside the cluster network. No node IP, no high port, and the code-execution surface is not bound on every node's interfaces. This is correct for the chart, where the gateway always runs in-cluster.

NodePort (Docker-Compose/hybrid escape hatch). Set provisioner.sandboxServiceType: NodePort only when the gateway is not in K8s (e.g. the compose dev path, where the gateway is a container reaching sandbox Pods on the host's Docker Desktop K8s). The provisioner then returns http://{NODE_HOST}:{NodePort}. NODE_HOST defaults to the provisioner pod's node IP via the downward API (status.hostIP); because a NodePort is exposed on every node, the gateway can reach <node-IP>:<NodePort> on most clusters without configuration. Override provisioner.nodeHost only if your CNI or network policy blocks pod->node-IP traffic:

kubectl get nodes -o wide    # use INTERNAL-IP or EXTERNAL-IP
provisioner:
  sandboxServiceType: NodePort
  nodeHost: 192.168.x.x

On multi-node clusters, also switch persistence.home.accessMode to ReadWriteMany (this is orthogonal to the Service type - it governs whether a sandbox Pod can be scheduled on a node other than the gateway's).

Lint / dry-run

helm lint deploy/helm/deer-flow
helm template deer-flow deploy/helm/deer-flow -n deer-flow -f my-values.yaml | \
  kubectl apply --dry-run=client -f -

Uninstall

helm uninstall deer-flow -n deer-flow
# the PVC is NOT deleted by default — remove it manually if desired:
kubectl -n deer-flow delete pvc -l app.kubernetes.io/instance=deer-flow