deer-flow/docs/superpowers/plans/2026-07-24-issue-4416-run-metadata-secrets.md

Issue 4416 Run Metadata Secrets Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Reject the legacy top-level metadata.auth_token field before a run can persist anything, preserve request-scoped MCP credentials in config.context.secrets, and hide the legacy key from historical API responses.

Architecture: deerflow.runtime.secret_context remains the single owner of secret-carrier and legacy-key policy. app.gateway.services.start_run() is the sole new-run admission boundary; API serializers call the same non-mutating metadata redactor only to hide historical records, without changing stores or callback implementations.

Tech Stack: Python 3.12, FastAPI, Pydantic v2, LangGraph/LangChain runnable context, pytest/anyio, ruff.

Global Constraints

• Reject only the exact top-level metadata key auth_token; do not add heuristic matching for names such as token, api_key, or nested arbitrary metadata.
• Return HTTP 422 with migration guidance to config.context.secrets.
• Validate before RunManager.create_or_reject(), thread upsert/status changes, build_run_config(), callback creation, RunJournal event emission, and background task creation.
• Reuse backend/packages/harness/deerflow/runtime/secret_context.py; do not maintain duplicate secret-key lists in stores, callbacks, routers, or response models.
• Historical API hiding must not mutate the stored run, thread, checkpoint, or event object.
• Independent POST /api/threads and PATCH /api/threads/{thread_id} metadata contracts are outside #4416; they do not admit a run and must not silently adopt a second policy behavior.
• Existing nested values in config.context.secrets must remain available to the live MCP interceptor while remaining absent from persisted run config.
• Features and bug fixes ship with tests; run backend formatting and lint checks before completion.
• Update README.md, backend/docs/MCP_SERVER.md, and backend/AGENTS.md in the same change set.

---

Task 1: Central legacy metadata policy

Files:

• Create: backend/tests/test_run_metadata_secret_safety.py
• Modify: backend/packages/harness/deerflow/runtime/secret_context.py

Interfaces:

• Consumes: existing redact_config_secrets(config: Any) -> Any.

• Produces: LEGACY_AUTH_TOKEN_METADATA_KEY: str, LegacyRunMetadataSecretError(ValueError), validate_run_metadata_secrets(metadata: Any) -> None, and redact_metadata_secrets(metadata: Any) -> Any.

• Step 1: Write failing unit tests for exact-key admission and non-mutating redaction

import pytest

from deerflow.runtime.secret_context import (
    LegacyRunMetadataSecretError,
    redact_metadata_secrets,
    validate_run_metadata_secrets,
)


@pytest.mark.parametrize("value", ["secret", "", None, {"nested": True}])
def test_validate_run_metadata_rejects_auth_token_key_by_presence(value):
    with pytest.raises(LegacyRunMetadataSecretError, match=r"config\.context\.secrets"):
        validate_run_metadata_secrets({"auth_token": value, "token_usage": 7})


@pytest.mark.parametrize(
    "metadata",
    [None, "not-a-mapping", {"token": "keep", "nested": {"auth_token": "keep"}}],
)
def test_validate_run_metadata_accepts_non_legacy_shapes(metadata):
    validate_run_metadata_secrets(metadata)


def test_redact_metadata_secrets_removes_exact_key_without_mutating_source():
    source = {
        "auth_token": "legacy-secret",
        "token_usage": 7,
        "nested": {"auth_token": "ordinary-nested-metadata"},
    }

    redacted = redact_metadata_secrets(source)

    assert redacted == {
        "token_usage": 7,
        "nested": {"auth_token": "ordinary-nested-metadata"},
    }
    assert source["auth_token"] == "legacy-secret"
    assert redacted is not source

• Step 2: Run the new tests and observe RED

Run:

cd backend
.venv/bin/python -m pytest tests/test_run_metadata_secret_safety.py -q

Expected: collection fails because the four new symbols do not yet exist.

• Step 3: Add the minimal centralized policy

Add to secret_context.py:

LEGACY_AUTH_TOKEN_METADATA_KEY = "auth_token"


class LegacyRunMetadataSecretError(ValueError):
    """Raised when a run puts a request credential in persisted metadata."""


def validate_run_metadata_secrets(metadata: Any) -> None:
    """Reject the legacy credential field at run admission."""
    if isinstance(metadata, dict) and LEGACY_AUTH_TOKEN_METADATA_KEY in metadata:
        raise LegacyRunMetadataSecretError(
            "Run metadata key 'auth_token' is not allowed; "
            "pass request-scoped credentials via config.context.secrets instead."
        )


def redact_metadata_secrets(metadata: Any) -> Any:
    """Return API-safe metadata without mutating historical storage objects."""
    if not isinstance(metadata, dict):
        return metadata
    return {
        key: value
        for key, value in metadata.items()
        if key != LEGACY_AUTH_TOKEN_METADATA_KEY
    }

• Step 4: Run the policy tests and observe GREEN

Run:

cd backend
.venv/bin/python -m pytest tests/test_run_metadata_secret_safety.py -q

Expected: all Task 1 tests pass.

• Step 5: Commit the policy unit

git add backend/packages/harness/deerflow/runtime/secret_context.py backend/tests/test_run_metadata_secret_safety.py
git commit -m "fix(security): centralize legacy run metadata policy"

Task 2: Unified run admission before persistence

Files:

• Modify: backend/tests/test_gateway_services.py
• Modify: backend/app/gateway/services.py

Interfaces:

• Consumes: validate_run_metadata_secrets(metadata: Any) -> None and LegacyRunMetadataSecretError.

• Produces: start_run() rejection with HTTPException(status_code=422) before any run/thread persistence; scheduled launches inherit the same boundary.

• Step 1: Add a failing real-store regression test

Add this reusable test setup beside _capture_start_run_graph_input:

def _make_start_run_persistence_context():
    from types import SimpleNamespace

    from langgraph.checkpoint.memory import InMemorySaver
    from langgraph.store.memory import InMemoryStore

    from deerflow.persistence.thread_meta.memory import MemoryThreadMetaStore
    from deerflow.runtime import RunManager
    from deerflow.runtime.runs.store.memory import MemoryRunStore

    run_store = MemoryRunStore()
    thread_store = MemoryThreadMetaStore(InMemoryStore())
    state = SimpleNamespace(
        stream_bridge=SimpleNamespace(),
        run_manager=RunManager(store=run_store),
        checkpointer=InMemorySaver(),
        store=InMemoryStore(),
        run_event_store=SimpleNamespace(),
        run_events_config=None,
        thread_store=thread_store,
        checkpoint_channel_mode="full",
        scheduled_task_service=None,
    )
    request = SimpleNamespace(
        headers={},
        state=SimpleNamespace(),
        app=SimpleNamespace(state=state),
    )
    return request, run_store, thread_store

Use it in a synchronous pytest test with an asyncio.run() scenario, matching the existing test_gateway_services.py style:

def test_start_run_rejects_legacy_auth_token_before_persistence():
    async def _scenario():
        request, run_store, thread_store = _make_start_run_persistence_context()
        body = RunCreateRequest(
            assistant_id="lead_agent",
            input={"messages": [{"role": "user", "content": "hi"}]},
            metadata={"auth_token": "legacy-secret", "token_usage": 7},
        )

        with patch(
            "app.gateway.services.run_agent", new_callable=AsyncMock
        ) as run_agent:
            with pytest.raises(HTTPException) as exc_info:
                await start_run(body, "thread-secret-admission", request)

        assert exc_info.value.status_code == 422
        assert "config.context.secrets" in str(exc_info.value.detail)
        assert await run_store.list_by_thread("thread-secret-admission") == []
        assert await thread_store.get("thread-secret-admission") is None
        run_agent.assert_not_called()

    asyncio.run(_scenario())

• Step 2: Add a failing ordinary-metadata preservation test

Start a run with:

metadata = {"token_usage": 7, "source": "regression"}

The test function must accept the existing _stub_app_config fixture because an admitted run calls get_run_context() and resolves the live AppConfig. Use _make_start_run_persistence_context(), patch resolve_agent_factory, and capture the live config in an async fake_run_agent:

def test_start_run_preserves_ordinary_metadata(_stub_app_config):
    async def _scenario():
        thread_id = "thread-ordinary-metadata"
        metadata = {"token_usage": 7, "source": "regression"}
        request, _run_store, thread_store = (
            _make_start_run_persistence_context()
        )
        captured: dict[str, Any] = {}

        async def fake_run_agent(*args, **kwargs):
            captured["config"] = kwargs["config"]

        with (
            patch(
                "app.gateway.services.resolve_agent_factory",
                return_value=object(),
            ),
            patch(
                "app.gateway.services.run_agent",
                side_effect=fake_run_agent,
            ),
        ):
            record = await start_run(
                RunCreateRequest(
                    assistant_id="lead_agent",
                    input={
                        "messages": [
                            {"role": "user", "content": "hi"}
                        ]
                    },
                    metadata=metadata,
                ),
                thread_id,
                request,
            )
            await record.task

        assert record.metadata == metadata
        assert (await thread_store.get(thread_id))["metadata"] == metadata
        assert captured["config"]["metadata"] == metadata

    asyncio.run(_scenario())

This prevents the exact-key rule from becoming broad metadata stripping.

• Step 3: Run both admission tests and observe RED

Run:

cd backend
.venv/bin/python -m pytest \
  tests/test_gateway_services.py::test_start_run_rejects_legacy_auth_token_before_persistence \
  tests/test_gateway_services.py::test_start_run_preserves_ordinary_metadata \
  -q

Expected: the legacy request is currently accepted and persisted.

• Step 4: Install validation at the first line of start_run()

Import the shared symbols and add before get_stream_bridge(request):

try:
    validate_run_metadata_secrets(getattr(body, "metadata", None))
except LegacyRunMetadataSecretError as exc:
    raise HTTPException(status_code=422, detail=str(exc)) from exc

Do not add checks to RunStore, ThreadMetaStore, RunJournal, build_run_config, or the router handlers.

• Step 5: Run both admission tests and observe GREEN

Run the exact Task 2 Step 3 command.

Expected: both tests pass; the rejected path creates no row or task.

• Step 6: Add and run the scheduled-launch regression

Call the real scheduled launcher with a deliberately minimal app. Because validation must be the first operation in start_run(), the request must receive the policy error before the empty app state is accessed:

def test_launch_scheduled_thread_run_rejects_legacy_auth_token():
    async def _scenario():
        with pytest.raises(HTTPException) as exc_info:
            await launch_scheduled_thread_run(
                thread_id="thread-scheduled",
                assistant_id="lead_agent",
                prompt="Run in background",
                app=SimpleNamespace(state=SimpleNamespace()),
                metadata={"auth_token": "legacy-secret"},
            )

        assert exc_info.value.status_code == 422
        assert "config.context.secrets" in str(exc_info.value.detail)

    asyncio.run(_scenario())

Run:

cd backend
.venv/bin/python -m pytest \
  tests/test_gateway_services.py::test_launch_scheduled_thread_run_rejects_legacy_auth_token \
  -q

Expected: PASS, proving scheduled metadata reaches the same admission policy rather than bypassing it.

• Step 7: Commit the admission boundary

git add backend/app/gateway/services.py backend/tests/test_gateway_services.py
git commit -m "fix(security): reject secrets at run admission"

Task 3: Non-mutating historical API hiding

Files:

• Modify: backend/tests/test_run_metadata_secret_safety.py
• Modify: backend/tests/test_run_events_endpoint.py
• Modify: backend/app/gateway/routers/thread_runs.py
• Modify: backend/app/gateway/routers/threads.py

Interfaces:

• Consumes: redact_metadata_secrets(metadata: Any) -> Any.

• Produces: redacted RunResponse, ThreadResponse, ThreadStateResponse, HistoryEntry, and run-event rows while leaving their input records unchanged.

• Step 1: Add failing response-model tests

Construct a historical RunRecord containing:

legacy_metadata = {"auth_token": "legacy-secret", "token_usage": 7}
record = RunRecord(
    run_id="legacy-run",
    thread_id="legacy-thread",
    assistant_id="lead_agent",
    status=RunStatus.success,
    on_disconnect=DisconnectMode.cancel,
    metadata=legacy_metadata,
)

response = _record_to_response(record)

assert response.metadata == {"token_usage": 7}
assert record.metadata["auth_token"] == "legacy-secret"

Import RunRecord from deerflow.runtime.runs.manager and DisconnectMode / RunStatus from deerflow.runtime.runs.schemas.

For each thread response model:

@pytest.mark.parametrize(
    ("response_class", "required_fields"),
    [
        (ThreadResponse, {"thread_id": "legacy-thread"}),
        (ThreadStateResponse, {}),
        (HistoryEntry, {"checkpoint_id": "legacy-checkpoint"}),
    ],
)
def test_thread_metadata_response_models_hide_historical_auth_token(
    response_class, required_fields
):
    source = {"auth_token": "legacy-secret", "token_usage": 7}
    response = response_class(**required_fields, metadata=source)

    assert response.metadata == {"token_usage": 7}
    assert source["auth_token"] == "legacy-secret"

Cover all three classes explicitly: ThreadResponse, ThreadStateResponse, and HistoryEntry.

• Step 2: Run the response tests and observe RED

Run:

cd backend
.venv/bin/python -m pytest tests/test_run_metadata_secret_safety.py -q

Expected: historical response objects still contain auth_token.

• Step 3: Apply the shared redactor to response construction

In _record_to_response() use:

metadata=redact_metadata_secrets(record.metadata),

In threads.py, create one local response base class:

class _MetadataRedactingResponse(BaseModel):
    @field_validator("metadata", mode="before", check_fields=False)
    @classmethod
    def _redact_legacy_metadata_secret(cls, value: Any) -> Any:
        return redact_metadata_secrets(value)

Make ThreadResponse, ThreadStateResponse, and HistoryEntry inherit from this base. Do not alter ThreadCreateRequest, ThreadPatchRequest, _SERVER_RESERVED_METADATA_KEYS, or stored rows.

Keep check_fields=False: the base class intentionally declares a validator for a field that exists only on its subclasses. In the worktree's installed Pydantic 2.13.3, field_validator still accepts check_fields, and removing it raises PydanticUserError while defining the base class.

Scope note: record.kwargs["config"]["metadata"] is the caller's raw LangChain RunnableConfig.metadata, not RunCreateRequest.metadata, and is outside the exact documented legacy request field addressed by #4416. Do not claim the admission validator cleans this separate mapping. Existing historical values there remain subject to the operator rotation and retained data cleanup guidance in Task 4; changing the raw RunnableConfig contract requires separate policy review.

• Step 4: Run the response tests and observe GREEN

Run the exact Task 3 Step 2 command.

Expected: all helper and response tests pass.

• Step 5: Add a failing historical run.start event test

Extend test_run_events_endpoint.py with a fake store returning one row:

stored_row = {
    "seq": 1,
    "event_type": "run.start",
    "metadata": {
        "caller": "lead_agent",
        "auth_token": "legacy-secret",
        "token_usage": 7,
    },
}

Call list_run_events() and assert:

assert events[0]["metadata"] == {
    "caller": "lead_agent",
    "token_usage": 7,
}
assert stored_row["metadata"]["auth_token"] == "legacy-secret"
assert events[0] is not stored_row

• Step 6: Run the event test and observe RED

Run:

cd backend
.venv/bin/python -m pytest \
  tests/test_run_events_endpoint.py::test_list_run_events_redacts_historical_run_start_metadata \
  -q

Expected: returned event metadata still exposes auth_token.

• Step 7: Redact event rows without mutating the store result

Replace the direct return in list_run_events() with:

events = await event_store.list_events(
    thread_id,
    run_id,
    event_types=types,
    task_id=task_id,
    limit=limit,
    after_seq=after_seq,
)
return [
    {
        **event,
        "metadata": redact_metadata_secrets(event.get("metadata")),
    }
    if isinstance(event, dict) and "metadata" in event
    else event
    for event in events
]

• Step 8: Run historical output tests and observe GREEN

Run:

cd backend
.venv/bin/python -m pytest \
  tests/test_run_metadata_secret_safety.py \
  tests/test_run_events_endpoint.py \
  -q

Expected: all tests pass, including existing event forwarding behavior.

• Step 9: Commit historical API hiding

git add \
  backend/app/gateway/routers/thread_runs.py \
  backend/app/gateway/routers/threads.py \
  backend/tests/test_run_metadata_secret_safety.py \
  backend/tests/test_run_events_endpoint.py
git commit -m "fix(security): hide legacy secrets from history APIs"

Task 4: Supported MCP carrier and operator documentation

Files:

• Modify: backend/tests/test_mcp_session_pool.py
• Modify: backend/tests/test_skill_request_scoped_secrets.py
• Modify: backend/docs/MCP_SERVER.md
• Modify: README.md
• Modify: backend/AGENTS.md

Interfaces:

• Consumes: LangGraph get_config()["context"]["secrets"] and existing redact_config_secrets.

• Produces: tested MCP header injection from the supported carrier and migration/rotation/cleanup guidance.

• Step 1: Add a failing MCP interceptor carrier regression

Extend the existing pooled-tool header test pattern with an interceptor that reads the live LangGraph config:

async def secret_header_interceptor(request, handler):
    from langgraph.config import get_config

    secrets = (get_config().get("context") or {}).get("secrets") or {}
    return await handler(
        request.override(headers={"Authorization": f"Bearer {secrets['MCP_AUTH_TOKEN']}"})
    )

Patch langgraph.config.get_config to return:

{"context": {"secrets": {"MCP_AUTH_TOKEN": "nested-secret"}}}

Invoke the wrapped tool and assert:

mock_session.call_tool.assert_awaited_once_with(
    "act",
    {"x": 1},
    meta={"headers": {"Authorization": "Bearer nested-secret"}},
)

• Step 2: Run the MCP regression

Run:

cd backend
.venv/bin/python -m pytest \
  tests/test_mcp_session_pool.py::test_session_pool_interceptor_reads_request_scoped_secret \
  -q

Expected before any production change: PASS if the documented supported carrier is already wired correctly. A failure is a real compatibility gap and must be fixed in the narrowest runtime layer before proceeding.

• Step 3: Strengthen persisted-config coverage for nested values

Change the existing config-redaction fixture to:

"secrets": {
    "ERP_TOKEN": _SECRET,
    "nested": {"secondary": _SECRET},
},

Keep assertions that the live source remains intact and the redacted copy contains no secret values.

• Step 4: Run the supported-carrier regression group

Run:

cd backend
.venv/bin/python -m pytest \
  tests/test_mcp_session_pool.py::test_session_pool_interceptor_reads_request_scoped_secret \
  tests/test_skill_request_scoped_secrets.py::TestLeakSurfaces::test_redact_config_secrets_strips_from_persisted_config \
  -q

Expected: both tests pass.

• Step 5: Replace the unsafe MCP documentation example

In backend/docs/MCP_SERVER.md, replace get_config()["metadata"]["auth_token"] with:

from langgraph.config import get_config


def build_auth_interceptor():
    async def interceptor(request, handler):
        config = get_config()
        secrets = (config.get("context") or {}).get("secrets") or {}
        token = secrets.get("MCP_AUTH_TOKEN")
        if token:
            request = request.override(
                headers={**(request.headers or {}), "Authorization": f"Bearer {token}"}
            )
        return await handler(request)

    return interceptor

Document the request shape:

{
  "metadata": {"source": "my-client"},
  "config": {
    "context": {
      "secrets": {"MCP_AUTH_TOKEN": "<request-scoped credential>"}
    }
  }
}

State that metadata.auth_token is rejected with 422 and is never the supported interceptor path.

• Step 6: Add repository and operator guidance

Add a concise security note to README.md linking to backend/docs/MCP_SERVER.md.

Update backend/AGENTS.md to state:

• start_run() validates the exact legacy key before run/thread persistence;

• secret_context.py owns admission and output-redaction policy;

• historical API hiding does not delete old database/event/log/snapshot/backup material;

• deployments that previously used metadata.auth_token must rotate the credential and clean all retained copies;

• restarting or upgrading DeerFlow does not perform that cleanup.

• Step 7: Commit supported flow and docs

git add \
  backend/tests/test_mcp_session_pool.py \
  backend/tests/test_skill_request_scoped_secrets.py \
  backend/docs/MCP_SERVER.md \
  README.md \
  backend/AGENTS.md
git commit -m "docs(security): migrate MCP credentials to secret context"

Task 5: Full verification and handoff

Files:

• Verify all modified files.

Interfaces:

• Consumes: all preceding commits.

• Produces: formatting-clean, lint-clean, regression-tested branch ready for review.

• Step 1: Run the focused security regression suite

cd backend
.venv/bin/python -m pytest \
  tests/test_run_metadata_secret_safety.py \
  tests/test_gateway_services.py \
  tests/test_run_events_endpoint.py \
  tests/test_mcp_session_pool.py \
  tests/test_skill_request_scoped_secrets.py \
  -q

Expected: all selected tests pass.

• Step 2: Format and verify formatting

cd backend
.venv/bin/ruff format \
  packages/harness/deerflow/runtime/secret_context.py \
  app/gateway/services.py \
  app/gateway/routers/thread_runs.py \
  app/gateway/routers/threads.py \
  tests/test_run_metadata_secret_safety.py \
  tests/test_gateway_services.py \
  tests/test_run_events_endpoint.py \
  tests/test_mcp_session_pool.py \
  tests/test_skill_request_scoped_secrets.py
.venv/bin/ruff format --check \
  packages/harness/deerflow/runtime/secret_context.py \
  app/gateway/services.py \
  app/gateway/routers/thread_runs.py \
  app/gateway/routers/threads.py \
  tests/test_run_metadata_secret_safety.py \
  tests/test_gateway_services.py \
  tests/test_run_events_endpoint.py \
  tests/test_mcp_session_pool.py \
  tests/test_skill_request_scoped_secrets.py

Expected: no files require further formatting.

• Step 3: Run lint

cd backend
.venv/bin/ruff check \
  packages/harness/deerflow/runtime/secret_context.py \
  app/gateway/services.py \
  app/gateway/routers/thread_runs.py \
  app/gateway/routers/threads.py \
  tests/test_run_metadata_secret_safety.py \
  tests/test_gateway_services.py \
  tests/test_run_events_endpoint.py \
  tests/test_mcp_session_pool.py \
  tests/test_skill_request_scoped_secrets.py

Expected: All checks passed!

• Step 4: Run the full backend suite

cd backend
.venv/bin/python -m pytest -q

Expected: the full backend suite passes. If an unrelated environment-dependent test fails, record the exact command, output, and why it is unrelated; do not claim a clean full suite.

• Step 5: Inspect repository hygiene

git diff --check upstream/main...HEAD
git status --short
git log --oneline --decorate upstream/main..HEAD

Expected: no whitespace errors; only intentional files are changed; the branch contains the design, plan, implementation, tests, and documentation commits.

• Step 6: Commit any formatter-only changes

If Step 2 changed files:

git add \
  backend/packages/harness/deerflow/runtime/secret_context.py \
  backend/app/gateway/services.py \
  backend/app/gateway/routers/thread_runs.py \
  backend/app/gateway/routers/threads.py \
  backend/tests/test_run_metadata_secret_safety.py \
  backend/tests/test_gateway_services.py \
  backend/tests/test_run_events_endpoint.py \
  backend/tests/test_mcp_session_pool.py \
  backend/tests/test_skill_request_scoped_secrets.py
git commit -m "style: format issue 4416 security fix"

If Step 2 changed nothing, skip this commit.

• Step 7: Summarize evidence for review

Report:

• the admission boundary and exact 422 migration message;
• the five historical response surfaces;
• proof that rejected requests create neither run nor thread records;
• proof that nested config.context.secrets reaches an MCP interceptor but not persisted run config;
• focused and full-suite test counts;
• any remaining operational requirement to rotate and clean historical credentials.