fix(gateway): make recursion limit configurable (#5390)

* fix(gateway): make recursion limit configurable

* docs: keep backend guidance within inherited size budget

* fix(gateway): address recursion limit review feedback

* docs(gateway): clarify recursion default scope

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

README.md

@@ -291,6 +291,13 @@ single-label cluster hosts, and Docker/Podman internal hostnames do not inherit
 honor environment proxy settings.
 
 Backend processes automatically pick up `config.yaml` changes on the next config access, so model metadata updates do not require a manual restart during development.
+
+Gateway runs use the top-level `recursion_limit` in `config.yaml` when an API
+request does not provide one. The default is `100`; valid per-request values
+take precedence, and `max_recursion_limit` (default `1000`) caps both. Changes
+apply to the next run without restarting the Gateway. This top-level setting
+applies to Gateway API runs; IM channel and embedded `DeerFlowClient` runs
+retain their own defaults and per-call override paths.
 The checkpoint storage settings `database.checkpoint_channel_mode` and
 `database.checkpoint_delta.snapshot_frequency` (default `10`) are exceptions:
 both are frozen when the process first builds an agent (including through

backend/AGENTS.md

@@ -2,7 +2,7 @@
 
 ## Project Overview
 
-DeerFlow is a LangGraph-based AI super agent system with a full-stack architecture. The backend provides a "super agent" with sandbox execution, persistent memory, subagent delegation, and extensible tool integration - all operating in per-thread isolated environments.
+The backend runs a LangGraph-based super agent with sandbox execution, persistent memory, subagent delegation, and extensible tools in isolated per-thread environments.
 
 **Architecture**:
 - **Gateway API** (port 8001): REST API plus embedded LangGraph-compatible agent runtime

backend/app/gateway/AGENTS.md

@@ -2,6 +2,11 @@
 
 FastAPI listens on port 8001; health: `GET /health` (liveness) and `GET /health/ready` (readiness; concurrently probes the ORM engine behind `database:` plus the effective LangGraph checkpointer/Store backend - the legacy `checkpointer:` section, otherwise derived from `database:`, resolved from the startup config snapshot recorded on `app.state` - beneath a single bounded deadline, with connection-opening probes serialized behind a strict per-process gate, 503 while either is unreachable or the startup backend cannot be resolved, `not_configured` for process-local backends such as `backend=memory`). Set `GATEWAY_ENABLE_DOCS=false` to disable the default `/docs`, `/redoc`, and `/openapi.json` endpoints.
 
+`build_run_config()` resolves the default LangGraph super-step budget from the
+hot-reloaded top-level `recursion_limit` setting. A valid request-level value
+takes precedence; invalid values fall back to that default, and
+`max_recursion_limit` caps both sources.
+
 Durable MCP notifications use internal Agent runs. Keep their trusted delivery instruction outside the user-input boundary, and frame serialized remote events as untrusted before model invocation. Strict thread existence/ownership admission dead-letters events whose task outlives its deleted chat instead of recreating the thread.
 
 CORS is same-origin by default when requests enter through nginx on port 2026. Split-origin or port-forwarded browser clients must opt in with `GATEWAY_CORS_ORIGINS` (exact origins); Gateway `CORSMiddleware` and `CSRFMiddleware` both read that variable so browser CORS and auth-origin checks stay aligned. Those clients also need `CORS_EXPOSED_HEADERS` (`csrf_middleware.py`): run-creating routes return the run's id in `Content-Location`, which is not CORS-safelisted, so JS cannot read it unless it is exposed — and the LangGraph SDK resolves run metadata from that header alone, so withholding it breaks `useStream`'s `onCreated` and thread-gated actions.

backend/app/gateway/services.py

@@ -661,12 +661,36 @@ def resolve_agent_factory(assistant_id: str | None):
 # client-supplied ``recursion_limit`` verbatim: an arbitrarily large value lets
 # a single run execute unbounded LangGraph super-steps (each at least one LLM
 # call), enabling runaway API cost / DoS. ``_DEFAULT_RECURSION_LIMIT`` is the
-# server default when the client sends nothing; the hard ceiling any client
-# value is clamped to is configurable via ``AppConfig.max_recursion_limit``.
+# fallback when app config cannot be loaded; the normal server default and hard
+# ceiling are configurable via ``AppConfig.recursion_limit`` and
+# ``AppConfig.max_recursion_limit``.
 _DEFAULT_RECURSION_LIMIT = 100
 _DEFAULT_MAX_RECURSION_LIMIT = 1000
 
 
+def _resolve_gateway_recursion_limits() -> tuple[int, int]:
+    """Resolve the run default and ceiling from one hot-reloaded snapshot."""
+    try:
+        app_config = get_app_config()
+        raw = app_config.recursion_limit
+        max_limit = app_config.max_recursion_limit
+        if raw > max_limit:
+            logger.warning(
+                "recursion_limit %d exceeds max_recursion_limit %d; clamped to %d for Gateway runs",
+                raw,
+                max_limit,
+                max_limit,
+            )
+        return min(raw, max_limit), max_limit
+    except Exception:
+        logger.warning(
+            "failed to load app config; falling back to recursion_limit=%d and max_recursion_limit=%d for Gateway runs",
+            _DEFAULT_RECURSION_LIMIT,
+            _DEFAULT_MAX_RECURSION_LIMIT,
+        )
+        return _DEFAULT_RECURSION_LIMIT, _DEFAULT_MAX_RECURSION_LIMIT
+
+
 def _resolve_max_recursion_limit() -> int:
     """Resolve the clamp ceiling from ``AppConfig.max_recursion_limit``.
 
@@ -717,15 +741,15 @@ def _resolve_scheduler_recursion_limit() -> int:
         return _DEFAULT_RECURSION_LIMIT
 
 
-def _clamp_recursion_limit(value: Any, max_limit: int) -> int:
+def _clamp_recursion_limit(value: Any, max_limit: int, default_limit: int) -> int:
     """Clamp a client-supplied ``recursion_limit`` into a safe server range.
 
     Non-integer values (including ``bool``, an ``int`` subclass) and non-positive
-    values fall back to ``_DEFAULT_RECURSION_LIMIT``; valid positive integers are
+    values fall back to the configured default; valid positive integers are
     capped at ``max_limit`` (from ``AppConfig.max_recursion_limit``).
     """
     if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
-        return _DEFAULT_RECURSION_LIMIT
+        return default_limit
     return min(value, max_limit)
 
 
@@ -749,16 +773,17 @@ def build_run_config(
     load the matching ``agents/<name>/SOUL.md`` and per-agent config —
     without it the agent silently runs as the default lead agent.
 
-    This mirrors the channel manager's ``_resolve_run_params`` logic so that
-    the LangGraph Platform-compatible HTTP API and the IM channel path behave
-    identically.
+    This mirrors the channel manager's ``_resolve_run_params`` logic except for
+    the recursion default: Gateway API runs use the configured top-level
+    ``recursion_limit``, while IM channel runs retain their own default.
     """
     # Lead-agent recursion budget (LangGraph super-steps for the lead graph
     # only). Independent of subagent depth: a `task()` dispatch runs the whole
     # subagent inside ONE lead tools-node step, and subagents enforce their own
-    # limit via `subagents.max_turns`. Do not conflate this 100 with the
+    # limit via `subagents.max_turns`. Do not conflate this budget with the
     # general-purpose subagent's max_turns.
-    config: dict[str, Any] = {"recursion_limit": _DEFAULT_RECURSION_LIMIT}
+    default_recursion_limit, max_recursion_limit = _resolve_gateway_recursion_limits()
+    config: dict[str, Any] = {"recursion_limit": default_recursion_limit}
     if request_config:
         # LangGraph >= 0.6.0 introduced ``context`` as the preferred way to
         # pass thread-level data and rejects requests that include both
@@ -807,14 +832,13 @@ def build_run_config(
         # super-steps (runaway LLM cost / DoS). Applied after the passthrough so
         # it overrides whatever the client sent.
         if "recursion_limit" in request_config:
-            max_limit = _resolve_max_recursion_limit()
-            clamped = _clamp_recursion_limit(request_config["recursion_limit"], max_limit)
+            clamped = _clamp_recursion_limit(request_config["recursion_limit"], max_recursion_limit, default_recursion_limit)
             if clamped != request_config["recursion_limit"]:
                 logger.warning(
                     "build_run_config: clamped client recursion_limit %r -> %d (max %d). thread_id=%s",
                     request_config["recursion_limit"],
                     clamped,
-                    max_limit,
+                    max_recursion_limit,
                     thread_id,
                 )
             config["recursion_limit"] = clamped

backend/docs/API.md

@@ -265,16 +265,19 @@ for runs without changed outputs keep their existing shape.
 **Recursion Limit:**
 
 `config.recursion_limit` caps the number of graph steps LangGraph will execute
-in a single run. The unified Gateway path defaults to `100` in
-`build_run_config` (see `backend/app/gateway/services.py`), which is a safer
-starting point for plan-mode or subagent-heavy runs. Clients can still set
-`recursion_limit` explicitly in the request body; increase it if you run deeply
-nested subagent graphs. Scheduled-task launches do not take a client body: they
+in a single run. The unified Gateway path uses the top-level `recursion_limit`
+from `config.yaml` (default `100`) when a request does not provide one. Clients
+can still set `recursion_limit` explicitly in the request body, and a valid
+request value takes precedence. Scheduled-task launches do not take a client body: they
 use `scheduler.recursion_limit` from `config.yaml` (default `1000`, matching
 the web UI). For safety, the Gateway clamps any supplied
-value to a configurable server ceiling (`max_recursion_limit` in `config.yaml`,
+or configured value to a server ceiling (`max_recursion_limit` in `config.yaml`,
 default `1000`) so a single run cannot execute unbounded graph steps (runaway
-LLM cost / DoS); invalid or non-positive values fall back to the `100` default.
+LLM cost / DoS); invalid or non-positive request values fall back to the
+configured default. Both top-level fields are read per run, so edits apply to
+the next request without restarting the Gateway. This top-level setting applies
+to Gateway API runs only; IM channel and embedded `DeerFlowClient` runs retain
+their own defaults and override paths.
 
 **Configurable Options:**
 - `model_name` (string): Override the default model

backend/docs/CONFIGURATION.md

@@ -25,6 +25,20 @@ preference hints for requests that should prefer a specific MCP server or tool.
 See [MCP Server Configuration](MCP_SERVER.md#routing-hints) for the schema,
 example, and soft-vs-hard routing boundary.
 
+### Recursion Limits
+
+Gateway runs use the top-level `recursion_limit` as their LangGraph super-step
+budget when the request does not include an explicit value. It defaults to
+`100`; raise it for deployments whose normal tasks need longer agent loops.
+Valid request values take precedence, while invalid values fall back to the
+configured default. `max_recursion_limit` (default `1000`) caps both sources to
+limit runaway LLM cost. Both settings are read per run, so changes apply to the
+next request without a Gateway restart.
+
+These settings apply to Gateway API runs. IM channel runs and embedded
+`DeerFlowClient` runs retain their own defaults and can be overridden through
+their channel/client-specific configuration or per-call options.
+
 ### Models
 
 Configure the LLM models available to the agent:

backend/packages/harness/deerflow/config/AGENTS.md

@@ -24,6 +24,8 @@ Setup: Copy `config.example.yaml` to `config.yaml` in the **project root** direc
 
 **Config Versioning**: `config.example.yaml` has a `config_version` field. On startup, `AppConfig.from_file()` compares user version vs example version and emits a warning if outdated. Missing `config_version` = version 0. Run `make config-upgrade` to auto-merge missing fields. When changing the config schema, bump `config_version` in `config.example.yaml`.
 
+Top-level `recursion_limit` and `max_recursion_limit` are hot-reloaded per Gateway run. The former supplies the default when a request omits or provides an invalid value; the latter caps both configured and client-provided budgets.
+
 **Config Caching**: `get_app_config()` caches the parsed config, but automatically reloads it when the resolved config path or file content signature changes. The signature includes file metadata and a content digest, so Gateway and LangGraph reads stay aligned with `config.yaml` edits even on object-store or network mounts where mtime can remain stale.
 
 **Config Hot-Reload Boundary**: Gateway dependencies route through `get_app_config()` on every request, so per-run fields like `models[*].max_tokens`, `summarization.*`, `title.*`, `memory.*`, `subagents.*`, `verification.*`, `tools[*]`, and the agent system prompt pick up `config.yaml` edits on the next message. `AppConfig` is intentionally **not** cached on `app.state` — `lifespan()` keeps a local `startup_config` variable for one-shot bootstrap work and passes it to `langgraph_runtime(app, startup_config)`.

backend/packages/harness/deerflow/config/app_config.py

@@ -215,10 +215,15 @@ class AppConfig(BaseModel):
             ),
         ),
     )
+    recursion_limit: int = Field(
+        default=100,
+        ge=1,
+        description="Default LangGraph recursion_limit for Gateway runs when the client does not provide one. Applied per run and capped by max_recursion_limit.",
+    )
     max_recursion_limit: int = Field(
         default=1000,
         ge=1,
-        description="Hard server-side ceiling for a client-supplied run recursion_limit. Client values above this are clamped; prevents runaway LangGraph super-steps (LLM cost / DoS).",
+        description="Hard server-side ceiling for configured defaults and client-supplied run recursion_limit values. Values above this are clamped; prevents runaway LangGraph super-steps (LLM cost / DoS).",
     )
     models: list[ModelConfig] = Field(default_factory=list, description="Available models")
     sandbox: SandboxConfig = Field(

backend/tests/test_gateway_services.py

@@ -623,6 +623,26 @@ def test_build_run_config_basic():
     assert config["recursion_limit"] == 100
 
 
+def test_build_run_config_uses_configured_default_recursion_limit(_stub_app_config):
+    """Runs without a request override use the operator-configured default."""
+    from app.gateway.services import build_run_config
+    from deerflow.config.app_config import AppConfig, reset_app_config, set_app_config
+
+    set_app_config(
+        AppConfig.model_validate(
+            {
+                "sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"},
+                "recursion_limit": 700,
+            }
+        )
+    )
+    try:
+        config = build_run_config("thread-1", None, None)
+        assert config["recursion_limit"] == 700
+    finally:
+        reset_app_config()
+
+
 def test_build_run_config_with_overrides():
     from app.gateway.services import build_run_config
 
@@ -721,13 +741,102 @@ def test_build_run_config_preserves_reasonable_recursion_limit(_stub_app_config)
     assert config["recursion_limit"] == 250
 
 
-def test_build_run_config_rejects_invalid_recursion_limit(_stub_app_config):
-    """Non-positive / non-int / bool values fall back to the server default."""
-    from app.gateway.services import _DEFAULT_RECURSION_LIMIT, build_run_config
+def test_build_run_config_client_recursion_limit_overrides_configured_default(_stub_app_config):
+    """An explicit valid client value takes precedence over the server default."""
+    from app.gateway.services import build_run_config
+    from deerflow.config.app_config import AppConfig, reset_app_config, set_app_config
 
-    for bad in (0, -5, "1000", 3.5, True, None):
-        config = build_run_config("thread-1", {"recursion_limit": bad}, None)
-        assert config["recursion_limit"] == _DEFAULT_RECURSION_LIMIT, bad
+    set_app_config(
+        AppConfig.model_validate(
+            {
+                "sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"},
+                "recursion_limit": 700,
+            }
+        )
+    )
+    try:
+        config = build_run_config("thread-1", {"recursion_limit": 250}, None)
+        assert config["recursion_limit"] == 250
+    finally:
+        reset_app_config()
+
+
+def test_build_run_config_rejects_invalid_recursion_limit(_stub_app_config):
+    """Non-positive / non-int / bool values fall back to the configured default."""
+    from app.gateway.services import build_run_config
+    from deerflow.config.app_config import AppConfig, reset_app_config, set_app_config
+
+    set_app_config(
+        AppConfig.model_validate(
+            {
+                "sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"},
+                "recursion_limit": 700,
+            }
+        )
+    )
+
+    try:
+        for bad in (0, -5, "1000", 3.5, True, None):
+            config = build_run_config("thread-1", {"recursion_limit": bad}, None)
+            assert config["recursion_limit"] == 700, bad
+    finally:
+        reset_app_config()
+
+
+def test_build_run_config_logs_and_uses_fallback_when_app_config_unavailable(monkeypatch, caplog):
+    """A config-load failure falls back visibly instead of silently."""
+    from app.gateway import services
+
+    monkeypatch.setattr(services, "get_app_config", lambda: (_ for _ in ()).throw(RuntimeError("broken config")))
+    caplog.set_level(logging.WARNING, logger="app.gateway.services")
+
+    config = services.build_run_config("thread-1", {"recursion_limit": 0}, None)
+
+    assert config["recursion_limit"] == services._DEFAULT_RECURSION_LIMIT
+    assert any("failed to load app config; falling back to recursion_limit=100" in record.message for record in caplog.records)
+
+
+def test_build_run_config_invalid_client_recursion_limit_uses_configured_default(_stub_app_config):
+    """An invalid client value cannot erase the operator-configured default."""
+    from app.gateway.services import build_run_config
+    from deerflow.config.app_config import AppConfig, reset_app_config, set_app_config
+
+    set_app_config(
+        AppConfig.model_validate(
+            {
+                "sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"},
+                "recursion_limit": 700,
+            }
+        )
+    )
+    try:
+        config = build_run_config("thread-1", {"recursion_limit": 0}, None)
+        assert config["recursion_limit"] == 700
+    finally:
+        reset_app_config()
+
+
+def test_build_run_config_clamps_configured_default_to_ceiling(_stub_app_config, caplog):
+    """The operator default remains bounded by max_recursion_limit."""
+    from app.gateway.services import build_run_config
+    from deerflow.config.app_config import AppConfig, reset_app_config, set_app_config
+
+    set_app_config(
+        AppConfig.model_validate(
+            {
+                "sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"},
+                "recursion_limit": 700,
+                "max_recursion_limit": 500,
+            }
+        )
+    )
+    try:
+        caplog.set_level(logging.WARNING, logger="app.gateway.services")
+        config = build_run_config("thread-1", None, None)
+        assert config["recursion_limit"] == 500
+        assert any("recursion_limit 700 exceeds max_recursion_limit 500" in record.message for record in caplog.records)
+    finally:
+        reset_app_config()
 
 
 def test_build_run_config_clamps_recursion_limit_with_context(_stub_app_config):

config.example.yaml

@@ -20,7 +20,7 @@
 # ============================================================================
 # Bump this number when the config schema changes.
 # Run `make config-upgrade` to merge new fields into your local config.yaml.
-config_version: 42
+config_version: 43
 
 # ============================================================================
 # Logging
@@ -94,14 +94,16 @@ token_budget:
   hard_stop_threshold: 1.0      # Force stop at 100% of the budget
 
 # ============================================================================
-# Recursion Limit — Hard ceiling for a client-supplied run recursion_limit
+# Recursion Limits
 # ============================================================================
 # A run's recursion_limit caps the number of LangGraph super-steps (each is at
-# least one LLM call). The Gateway never trusts a client-supplied value
-# verbatim: any value above this ceiling is clamped down to it, preventing
-# runaway API cost / DoS. Invalid or non-positive client values fall back to
-# the server default of 100. Raise this only if you legitimately run very
-# deeply nested subagent graphs.
+# least one LLM call). Clients may override the default per request; invalid or
+# non-positive client values fall back to this configured default.
+recursion_limit: 100
+
+# The Gateway never trusts a configured or client-supplied value above this
+# ceiling, preventing runaway API cost / DoS. Raise it only if you legitimately
+# run very deeply nested subagent graphs.
 max_recursion_limit: 1000
 
 

deploy/helm/deer-flow/README.md

@@ -131,7 +131,7 @@ they resolve from the `secrets` map):
 
 ```yaml
 config: |
-  config_version: 42
+  config_version: 43
   models:
     - name: gpt-4
       use: langchain_openai:ChatOpenAI

deploy/helm/deer-flow/values.yaml

@@ -249,8 +249,10 @@ ingress:
 # -- DeerFlow config.yaml content. Secrets MUST stay as $VAR references — never
 #    inline literal secret values here. The default enables provisioner sandbox.
 config: |
-  config_version: 42
+  config_version: 43
   log_level: info
+  recursion_limit: 100
+  max_recursion_limit: 1000
 
   models: []
   # Example (uncomment & set the matching secret in `secrets`):