mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-14 16:08:41 +00:00
* fix(tools): run tool assembly off-loop at async entry points get_available_tools() may block on MCP cache initialization while it is called on async agent-assembly paths (task_tool, durable batch execution), stalling the calling event loop for the full discovery duration. Dispatch the (unchanged, synchronous) assembly call to a worker thread via asyncio.to_thread at the two async entry points so the loop keeps processing requests, SSE frames, cancellations, and timers. Fixes #5172 * fix(tools): offload lead-agent assembly off-loop and pin with blocking-io anchors Review follow-up for #5224: - run_agent now dispatches agent_factory(...) through asyncio.to_thread, so lead-agent assembly (including both get_available_tools call sites in _assemble_lead_agent) runs off the event loop — the Gateway headline scenario from issue #5172. - _ensure_sync_invocable_tool takes a double-checked threading.Lock, making the in-place tool.func wrap on the shared tool singletons explicitly single-shot now that assembly can run concurrently on worker threads. - Add backend/tests/blocking_io/test_tool_assembly_offloop.py: blocking-probe anchors for task_tool and SubagentBatchService._execute_item under the strict Blockbuster gate, plus a meta-check proving the gate trips on the exact syscall class (ExtensionsConfig.from_file on the loop). Verified the anchor goes red when the offload is flattened back to a plain call. * fix(gateway): build checkpoint state accessor off-loop; anchor run_agent offload Review follow-up for #5224: - Add abuild_checkpoint_state_accessor (asyncio.to_thread around the unchanged sync builder) and switch every async call site to it: the stateless_wait route, thread_runs, both threads call sites, and the build_thread_checkpoint_state_accessor boundary. The agent-factory assembly re-enters get_available_tools() and may block on MCP cache initialization; repeat calls hit _state_accessor_graph_cache and only pay the thread hop. - Add a third blocking-io anchor driving the real run_agent with minimal RunManager/bridge stubs; the factory performs a real production blocking read (ExtensionsConfig.from_file()) and the test asserts assembly never runs on the main thread. Verified the anchor goes red when the run_agent offload is flattened back to a plain call. - Adapt the test_threads_router checkpoint-builder patch sites to the new async name. * refactor(tools): carry assembly offloads on a dedicated bounded pool Review follow-up for #5224: - Add utils/assembly_io.py: a dedicated ThreadPoolExecutor (default 8 workers, DEER_FLOW_ASSEMBLY_WORKERS-overridable, mirroring utils/file_io.py and tools/sync.py) with run_assembly(), which copies contextvars explicitly. A hung stdio MCP server parks its worker for the full MCP timeout; carrying assembly hops on the loop's default executor would let a few parked assemblies queue every other to_thread/run_in_executor(None, ...) caller behind them. - Switch all four offloads (run_agent, task_tool, batch _execute_item, abuild_checkpoint_state_accessor) to run_assembly(). - State the cold-path behavior in the accessor docstring: the graph cache validates factory identity, so non-identity-stable factories may duplicate lead-agent assembly across concurrent readers (MCP discovery stays process-wide single-flight); the pool bounds the duplicates. - Add a fourth blocking-io anchor driving build_thread_checkpoint_state_ accessor with a per-resolution fresh factory (always a cache miss) and the real production blocking read; enumerate all four offloads in the gate's module docstring. Verified the anchor goes red when abuild_checkpoint_state_accessor is flattened back to a plain call. * fix(subagents): revalidate batch item before launch; make assembly pool observable Review follow-up for #5224: - _execute_item() revalidates the durable state right after assembly and before executor.execute_async(): renew_item_lease() returns valid=False when cancel_batch() terminalized the item or the lease was lost while assembly was parked, and the launch is skipped (the canceller already finalized the item). Previously the launch was unconditional and the poll loop's cancellation checks only started after execution began. - Regression test driving the real SQLite repository: a blocking assembly probe parks _execute_item, cancel_batch() lands, and the launch is skipped with the item staying cancelled. Verified the test goes red when the revalidation is removed. - run_assembly() tracks pending assemblies and logs a throttled WARNING once the pending count exceeds the worker count, so assembly starvation (workers parked on a hung MCP server) is distinguishable from idle. - The run_agent blocking-io anchor now binds a sentinel extension snapshot via ctx.extensions and asserts the factory observed it through get_agent_build_extensions(), pinning run_assembly()'s ContextVar propagation. Verified red when ctx.run is dropped. - Document the assembly pool in backend/AGENTS.md. * fix(utils): decrement the assembly pending count on the pool thread The pending-assembly counter behind the starvation warning decremented from the asyncio future's done callback, which never fires once the submitting loop is closed while its worker is still running: the count ratcheted up permanently and eventually fired the starvation warning with no starvation behind it (reproduced at 97dc9bec by review). Decrement instead from the dispatched work item: run_assembly() wraps func so a finally drops the count under the pending lock on the pool thread, and the done callback is gone. Pin the counter with tests/test_assembly_io.py: a healthy call returns the count to zero, and an abandoned loop (stopped while the worker is parked) does not wedge it — the abandoned case goes red against the old done-callback decrement. * docs(utils): fix the pending-counter comment after the decrement move The comment still described the removed done-callback decrement, contradicting _work()'s own comment; state the actual mechanism (increment on the loop before dispatch, decrement from the dispatched work item's finally on a pool thread). * test(gateway): retarget checkpoint-accessor stubs to the services seam thread_runs and runs now call abuild_checkpoint_state_accessor, so the upstream wait-reader, regenerate-prepare, and idempotency tests must stub the sync builder where abuild resolves it (app.gateway.services); stubbing the removed router re-exports fails with AttributeError at setup. The async seam semantics are unchanged: run_assembly invokes the stubbed sync builder off-loop and propagates its return values and exceptions. Move the agent/tool assembly off-load note from backend/AGENTS.md to deerflow/utils/AGENTS.md (next to assembly_io.py) so the effective instruction chain for agents/middlewares no longer grows past the AG002 hard limit. * fix(runtime): serialize same-key accessor assembly and release queued-cancel slots Address the three review follow-ups on the assembly off-load: - assembly_io: a job cancelled while still queued never runs its work item, so the dispatched finally never fired and _pending_assemblies stayed elevated until a false starvation warning. Exactly-once cleanup now rides the concurrent future's cancelled() state — cancel() only succeeds before the executor starts the item, so cancelled() is true precisely when the finally will never run — plus a submit-failure release; the one-worker queued-cancellation case is pinned red/green. - services: overlapping cold readers sharing one cache key could both run full agent assembly. _state_accessor_graph now serializes per key through a thread-side KeyedLockTable (pool threads, no running loop) and re-validates factory/app-config identity under the lock, so the factory runs exactly once while identity changes still rebuild. Cache dict access is lock-guarded now that construction runs off-loop. - guidance inventory: register deerflow/utils/AGENTS.md in EXPECTED_GUIDANCE_PATHS so test_repository_has_the_approved_scoped_ guidance_shape matches the relocated assembly note (CI shard 4). * test(keyed-lock): pin KeyedLockTable reclamation and waiter bypass directly Thread-side counterparts of the async table's own tests: overlapping hold() calls serialize (a late arrival joins the live entry instead of creating a second lock that bypasses a queued waiter), the last check-in pops the entry, and many unique keys leave the registry empty. Both regressions verified red — popping unconditionally trips the late-arrival test, never reclaiming trips the many-keys test. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
1851 lines
84 KiB
Python
1851 lines
84 KiB
Python
"""Thread CRUD, state, and history endpoints.
|
|
|
|
Combines the existing thread-local filesystem cleanup with LangGraph
|
|
Platform-compatible thread management backed by the checkpointer.
|
|
|
|
Channel values returned in state responses are serialized through
|
|
:func:`deerflow.runtime.serialization.serialize_channel_values` to
|
|
ensure LangChain message objects are converted to JSON-safe dicts
|
|
matching the LangGraph Platform wire format expected by the
|
|
``useStream`` React hook.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import shutil
|
|
import uuid
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
|
|
from langgraph.checkpoint.base import empty_checkpoint
|
|
from langgraph.types import Overwrite
|
|
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
|
from sqlalchemy.exc import IntegrityError
|
|
|
|
from app.gateway.authz import require_permission
|
|
from app.gateway.checkpoint_lineage import (
|
|
CheckpointLineageError,
|
|
CheckpointParentMissingError,
|
|
find_checkpoint_before_message,
|
|
find_checkpoint_before_message_chronologically,
|
|
is_duration_only_checkpoint,
|
|
)
|
|
from app.gateway.deps import get_checkpointer, get_run_event_store, get_run_manager
|
|
from app.gateway.internal_auth import get_trusted_internal_owner_user_id
|
|
from app.gateway.services import (
|
|
abuild_checkpoint_state_accessor,
|
|
build_checkpoint_state_mutation_accessor,
|
|
build_thread_checkpoint_state_accessor,
|
|
build_thread_checkpoint_state_mutation_accessor,
|
|
reserve_checkpoint_write,
|
|
strip_server_owned_state_metadata,
|
|
)
|
|
from app.gateway.utils import sanitize_log_param
|
|
from deerflow.agents.thread_state import THREAD_STATE_REDUCER_FIELDS
|
|
from deerflow.config.paths import Paths, get_paths
|
|
from deerflow.config.summarization_config import ContextSize
|
|
from deerflow.persistence.thread_meta import PROJECT_FILTER_UNSET, THREAD_ARCHIVED_METADATA_KEY, THREAD_PINNED_METADATA_KEY, THREAD_PROJECT_METADATA_KEY, ThreadOwnershipConflictError
|
|
from deerflow.runtime import ThreadOperationKind, serialize_channel_values_for_api
|
|
from deerflow.runtime.checkpoint_mode import CheckpointModeMismatchError, CheckpointModeReconfigurationError
|
|
from deerflow.runtime.checkpoint_state import graph_reducer_channels, graph_state_schema, graph_writable_channels
|
|
from deerflow.runtime.context_compaction import (
|
|
ContextCompactionDisabled,
|
|
ContextCompactionFailed,
|
|
ThreadCompactionResult,
|
|
compact_thread_context,
|
|
)
|
|
from deerflow.runtime.events.message_seq import stamp_messages_with_seq
|
|
from deerflow.runtime.goal import (
|
|
DEFAULT_MAX_GOAL_CONTINUATIONS,
|
|
build_goal_state,
|
|
ensure_thread_checkpoint,
|
|
goal_thread_lock,
|
|
read_thread_goal,
|
|
write_thread_goal,
|
|
)
|
|
from deerflow.runtime.journal import build_branch_history_seed_events
|
|
from deerflow.runtime.runs.manager import ConflictError
|
|
from deerflow.runtime.runs.worker import RUN_MESSAGE_IDS_METADATA_KEY, valid_duration_entry, valid_run_message_id_entry
|
|
from deerflow.runtime.secret_context import redact_metadata_secrets
|
|
from deerflow.runtime.user_context import get_effective_user_id
|
|
from deerflow.utils.file_io import run_file_io
|
|
from deerflow.utils.thread_id import ThreadId, resolve_thread_id, validate_thread_id
|
|
from deerflow.utils.time import coerce_iso, now_iso
|
|
|
|
logger = logging.getLogger(__name__)
|
|
router = APIRouter(prefix="/api/threads", tags=["threads"])
|
|
|
|
_CHECKPOINT_MODE_ERRORS = (CheckpointModeMismatchError, CheckpointModeReconfigurationError)
|
|
|
|
|
|
def _optional_run_event_store(request: Request) -> Any:
|
|
"""Return the run event store, or ``None`` when the app has none wired.
|
|
|
|
Reads must not start depending on the feed: seq is placement metadata, and a
|
|
response without it degrades to the client's own ordering rule rather than
|
|
failing. ``get_run_event_store`` raises instead, which is right for the
|
|
endpoints that cannot work without a feed.
|
|
"""
|
|
return getattr(request.app.state, "run_event_store", None)
|
|
|
|
|
|
def _checkpoint_mode_http_error(exc: Exception, thread_id: str) -> HTTPException:
|
|
"""Map checkpoint-mode guard failures to precise HTTP statuses.
|
|
|
|
A mismatch means the thread's persisted checkpoints conflict with the
|
|
process's frozen mode (operator-actionable, 409); a reconfiguration means
|
|
the process itself is mid mode-flip (transient, 503). Both must surface
|
|
their message — a generic 500 would force operators to grep logs to
|
|
discover the root cause after a mode flip.
|
|
"""
|
|
if isinstance(exc, CheckpointModeMismatchError):
|
|
return HTTPException(status_code=409, detail=f"Thread {thread_id}: {exc}")
|
|
return HTTPException(status_code=503, detail=str(exc))
|
|
|
|
|
|
# Metadata keys that the server controls; clients are not allowed to set
|
|
# them. Pydantic ``@field_validator("metadata")`` strips them on every
|
|
# inbound model below so a malicious client cannot reflect a forged
|
|
# owner identity through the API surface. Defense-in-depth — the
|
|
# row-level invariant is still ``threads_meta.user_id`` populated from
|
|
# the auth contextvar; this list closes the metadata-blob echo gap.
|
|
_SERVER_RESERVED_METADATA_KEYS: frozenset[str] = frozenset({"owner_id", "user_id", THREAD_PROJECT_METADATA_KEY})
|
|
_SIDECAR_METADATA_KEY = "deerflow_sidecar"
|
|
_BRANCH_METADATA_KEY = "deerflow_branch"
|
|
_BRANCH_TITLE_SEQUENCE_METADATA_KEY = "branch_title_sequence"
|
|
# Thread-scoped runtime channels a branch must NOT inherit from its parent:
|
|
# ``sandbox.sandbox_id`` binds path mappings and the release lifecycle to the
|
|
# *parent* thread, so copying it would make the branch read/write the parent's
|
|
# workspace (bypassing the per-branch user-data clone) and release the
|
|
# parent's sandbox after its first run; the branch lazily acquires its own
|
|
# sandbox keyed by its own thread_id instead. ``thread_data`` is recomputed
|
|
# from the branch's thread_id by ThreadDataMiddleware on every run.
|
|
_BRANCH_EXCLUDED_CHANNELS = frozenset({"sandbox", "thread_data"})
|
|
_BRANCH_HISTORY_SCAN_LIMIT = 200
|
|
_BRANCH_HISTORY_RAW_SCAN_LIMIT = _BRANCH_HISTORY_SCAN_LIMIT * 2
|
|
_BRANCH_TITLE_MAX_LENGTH = 256
|
|
_BRANCH_TITLE_SEQUENCE_MAX = 9_007_199_254_740_991
|
|
_BRANCH_SIBLING_PAGE_SIZE = 100
|
|
|
|
|
|
def _strip_reserved_metadata(metadata: dict[str, Any] | None) -> dict[str, Any]:
|
|
"""Return ``metadata`` with server-controlled keys removed."""
|
|
if not metadata:
|
|
return metadata or {}
|
|
return {k: v for k, v in metadata.items() if k not in _SERVER_RESERVED_METADATA_KEYS}
|
|
|
|
|
|
def _is_organization_metadata_patch(metadata: dict[str, Any]) -> bool:
|
|
"""Recognize list-organization writes that must preserve activity time."""
|
|
return bool(metadata) and set(metadata) <= {THREAD_PINNED_METADATA_KEY, THREAD_ARCHIVED_METADATA_KEY} and all(isinstance(value, bool) for value in metadata.values())
|
|
|
|
|
|
def _message_id(message: Any) -> str | None:
|
|
if isinstance(message, dict):
|
|
raw = message.get("id")
|
|
else:
|
|
raw = getattr(message, "id", None)
|
|
return raw if isinstance(raw, str) and raw else None
|
|
|
|
|
|
def _message_type(message: Any) -> str | None:
|
|
if isinstance(message, dict):
|
|
raw = message.get("type")
|
|
else:
|
|
raw = getattr(message, "type", None)
|
|
return raw if isinstance(raw, str) and raw else None
|
|
|
|
|
|
def _message_additional_kwargs(message: Any) -> dict[str, Any]:
|
|
if isinstance(message, dict):
|
|
raw = message.get("additional_kwargs")
|
|
else:
|
|
raw = getattr(message, "additional_kwargs", None)
|
|
return raw if isinstance(raw, dict) else {}
|
|
|
|
|
|
def _is_branch_visible_message(message: Any) -> bool:
|
|
if _message_additional_kwargs(message).get("hide_from_ui") is True:
|
|
return False
|
|
return _message_type(message) in {"human", "ai"}
|
|
|
|
|
|
def _is_branch_assistant_message(message: Any) -> bool:
|
|
return _message_type(message) == "ai"
|
|
|
|
|
|
def _checkpoint_messages(snapshot: Any) -> list[Any]:
|
|
values = getattr(snapshot, "values", None) or {}
|
|
messages = values.get("messages") if isinstance(values, dict) else None
|
|
return list(messages) if isinstance(messages, list) else []
|
|
|
|
|
|
def _checkpoint_id(snapshot: Any) -> str | None:
|
|
config = getattr(snapshot, "config", {}) or {}
|
|
raw = config.get("configurable", {}).get("checkpoint_id")
|
|
return raw if isinstance(raw, str) and raw else None
|
|
|
|
|
|
def _matches_branch_target(messages: list[Any], target_message_ids: set[str]) -> bool:
|
|
if not target_message_ids:
|
|
return False
|
|
|
|
index_by_id = {_message_id(message): index for index, message in enumerate(messages) if _message_id(message)}
|
|
if not target_message_ids.issubset(index_by_id.keys()):
|
|
return False
|
|
if any(not _is_branch_assistant_message(messages[index_by_id[message_id]]) for message_id in target_message_ids):
|
|
return False
|
|
|
|
target_end_index = max(index_by_id[message_id] for message_id in target_message_ids)
|
|
return not any(_is_branch_visible_message(message) for message in messages[target_end_index + 1 :])
|
|
|
|
|
|
def _branch_target_human_message(messages: list[Any], target_message_ids: set[str]) -> Any | None:
|
|
index_by_id = {_message_id(message): index for index, message in enumerate(messages) if _message_id(message)}
|
|
if not target_message_ids.issubset(index_by_id.keys()):
|
|
return None
|
|
target_start_index = min(index_by_id[message_id] for message_id in target_message_ids)
|
|
return next(
|
|
(message for message in reversed(messages[:target_start_index]) if _message_type(message) == "human" and _is_branch_visible_message(message)),
|
|
None,
|
|
)
|
|
|
|
|
|
async def _find_branch_checkpoint(
|
|
accessor: Any,
|
|
config: dict[str, Any],
|
|
target_message_ids: set[str],
|
|
) -> Any:
|
|
try:
|
|
for snapshot in await accessor.ahistory(config, limit=_BRANCH_HISTORY_RAW_SCAN_LIMIT):
|
|
if is_duration_only_checkpoint(snapshot):
|
|
continue
|
|
if _matches_branch_target(_checkpoint_messages(snapshot), target_message_ids):
|
|
return snapshot
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, config.get("configurable", {}).get("thread_id", "")) from exc
|
|
except Exception:
|
|
thread_id = config.get("configurable", {}).get("thread_id", "")
|
|
logger.exception("Failed to scan branch checkpoint history for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to find branch checkpoint")
|
|
raise HTTPException(status_code=409, detail="This turn can no longer be branched from.")
|
|
|
|
|
|
async def _branch_targets_latest_turn(
|
|
accessor: Any,
|
|
config: dict[str, Any],
|
|
target_message_ids: set[str],
|
|
) -> bool:
|
|
"""Return whether the target turn is the final visible turn."""
|
|
try:
|
|
for snapshot in await accessor.ahistory(config, limit=_BRANCH_HISTORY_RAW_SCAN_LIMIT):
|
|
if is_duration_only_checkpoint(snapshot):
|
|
continue
|
|
messages = _checkpoint_messages(snapshot)
|
|
if not messages:
|
|
continue
|
|
return _matches_branch_target(messages, target_message_ids)
|
|
except Exception:
|
|
thread_id = config.get("configurable", {}).get("thread_id", "")
|
|
logger.warning(
|
|
"Failed to resolve latest turn for thread %s; treating branch as historical",
|
|
sanitize_log_param(thread_id),
|
|
exc_info=True,
|
|
)
|
|
return False
|
|
|
|
|
|
async def _find_branch_replay_base(
|
|
accessor: Any,
|
|
config: dict[str, Any],
|
|
snapshot: Any,
|
|
target_human_id: str,
|
|
) -> Any | None:
|
|
"""Resolve a replay base while preserving unlinked legacy histories."""
|
|
|
|
try:
|
|
return await find_checkpoint_before_message(
|
|
accessor,
|
|
snapshot,
|
|
target_human_id,
|
|
max_depth=_BRANCH_HISTORY_RAW_SCAN_LIMIT,
|
|
)
|
|
except CheckpointParentMissingError:
|
|
thread_id = config.get("configurable", {}).get("thread_id", "")
|
|
logger.debug(
|
|
"Could not resolve parent lineage for branch thread %s; falling back to history scan",
|
|
sanitize_log_param(thread_id),
|
|
exc_info=True,
|
|
)
|
|
except CheckpointLineageError as exc:
|
|
thread_id = config.get("configurable", {}).get("thread_id", "")
|
|
logger.warning(
|
|
"Rejected unsafe checkpoint lineage for branch thread %s",
|
|
sanitize_log_param(thread_id),
|
|
exc_info=True,
|
|
)
|
|
raise HTTPException(status_code=409, detail="This turn can no longer be branched from.") from exc
|
|
|
|
try:
|
|
history = await accessor.ahistory(config, limit=_BRANCH_HISTORY_RAW_SCAN_LIMIT)
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
thread_id = config.get("configurable", {}).get("thread_id", "")
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
except Exception as exc:
|
|
thread_id = config.get("configurable", {}).get("thread_id", "")
|
|
logger.exception("Failed to scan replay checkpoint history for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to inspect checkpoint history") from exc
|
|
|
|
replay_base, target_found = find_checkpoint_before_message_chronologically(history, target_human_id)
|
|
if not target_found:
|
|
logger.warning(
|
|
"Could not locate branch user message %s in chronological history for thread %s",
|
|
sanitize_log_param(target_human_id),
|
|
sanitize_log_param(config.get("configurable", {}).get("thread_id", "")),
|
|
)
|
|
return replay_base
|
|
|
|
|
|
def _ignore_branch_user_data(directory: str, names: list[str]) -> set[str]:
|
|
ignored: set[str] = set()
|
|
base = Path(directory)
|
|
for name in names:
|
|
path = base / name
|
|
if name.startswith(".upload-") and name.endswith(".part"):
|
|
ignored.add(name)
|
|
elif path.is_symlink():
|
|
ignored.add(name)
|
|
return ignored
|
|
|
|
|
|
def _copy_branch_user_data_sync(paths: Paths, source_thread_id: str, target_thread_id: str, *, user_id: str) -> str:
|
|
source = paths.sandbox_user_data_dir(source_thread_id, user_id=user_id)
|
|
target = paths.sandbox_user_data_dir(target_thread_id, user_id=user_id)
|
|
if not source.exists():
|
|
return "not_found"
|
|
|
|
shutil.copytree(source, target, ignore=_ignore_branch_user_data, dirs_exist_ok=True)
|
|
return "current_thread_best_effort"
|
|
|
|
|
|
async def _copy_branch_user_data(source_thread_id: str, target_thread_id: str) -> str:
|
|
paths = get_paths()
|
|
user_id = get_effective_user_id()
|
|
try:
|
|
return await run_file_io(_copy_branch_user_data_sync, paths, source_thread_id, target_thread_id, user_id=user_id)
|
|
except Exception:
|
|
logger.warning(
|
|
"Failed to copy user-data for branch %s -> %s",
|
|
sanitize_log_param(source_thread_id),
|
|
sanitize_log_param(target_thread_id),
|
|
exc_info=True,
|
|
)
|
|
return "failed"
|
|
|
|
|
|
def _next_branch_title_sequence(source_sequence: Any, *, source_is_branch: bool) -> int:
|
|
if source_is_branch and isinstance(source_sequence, int) and not isinstance(source_sequence, bool) and 2 <= source_sequence < _BRANCH_TITLE_SEQUENCE_MAX:
|
|
return source_sequence + 1
|
|
return 2
|
|
|
|
|
|
def _format_branch_display_name(base: str, sequence: int) -> str | None:
|
|
suffix = f" ({sequence})"
|
|
truncated_base = base[: _BRANCH_TITLE_MAX_LENGTH - len(suffix)].rstrip()
|
|
return f"{truncated_base}{suffix}" if truncated_base else None
|
|
|
|
|
|
def _default_branch_title(
|
|
source_title: Any,
|
|
*,
|
|
source_is_branch: bool = False,
|
|
source_sequence: Any = None,
|
|
sibling_records: list[dict[str, Any]] | None = None,
|
|
) -> tuple[str | None, int | None]:
|
|
if not isinstance(source_title, str):
|
|
return None, None
|
|
|
|
display_name = source_title.strip()
|
|
if source_is_branch:
|
|
while display_name.lower().startswith("branch:"):
|
|
display_name = display_name[len("branch:") :].strip()
|
|
|
|
if not display_name:
|
|
return None, None
|
|
|
|
sequence = _next_branch_title_sequence(source_sequence, source_is_branch=source_is_branch)
|
|
base = display_name
|
|
if sequence > 2:
|
|
source_suffix = f" ({sequence - 1})"
|
|
if display_name.endswith(source_suffix):
|
|
base = display_name[: -len(source_suffix)].rstrip()
|
|
|
|
occupied_titles = {sibling.get("display_name") for sibling in sibling_records or [] if isinstance(sibling.get("display_name"), str)}
|
|
display_name = _format_branch_display_name(base, sequence)
|
|
while display_name in occupied_titles:
|
|
if sequence >= _BRANCH_TITLE_SEQUENCE_MAX:
|
|
return None, None
|
|
sequence += 1
|
|
display_name = _format_branch_display_name(base, sequence)
|
|
return display_name, sequence if display_name is not None else None
|
|
|
|
|
|
async def _branch_sibling_records(thread_store: Any, parent_thread_id: str) -> list[dict[str, Any]]:
|
|
siblings: list[dict[str, Any]] = []
|
|
offset = 0
|
|
while True:
|
|
page = await thread_store.search(
|
|
metadata={"branch_parent_thread_id": parent_thread_id},
|
|
limit=_BRANCH_SIBLING_PAGE_SIZE,
|
|
offset=offset,
|
|
)
|
|
siblings.extend(page)
|
|
if len(page) < _BRANCH_SIBLING_PAGE_SIZE:
|
|
return siblings
|
|
offset += len(page)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Response / request models
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class ThreadDeleteResponse(BaseModel):
|
|
"""Response model for thread cleanup."""
|
|
|
|
success: bool
|
|
message: str
|
|
|
|
|
|
class _MetadataRedactingResponse(BaseModel):
|
|
@field_validator("metadata", mode="before", check_fields=False)
|
|
@classmethod
|
|
def _redact_legacy_metadata_secret(cls, value: Any) -> Any:
|
|
return redact_metadata_secrets(value)
|
|
|
|
|
|
class ThreadResponse(_MetadataRedactingResponse):
|
|
"""Response model for a single thread."""
|
|
|
|
# ThreadMetaStore records include internal lifecycle fields such as
|
|
# ``incarnation``. Keep the HTTP response as an explicit public projection.
|
|
model_config = ConfigDict(extra="ignore")
|
|
|
|
thread_id: str = Field(description="Unique thread identifier")
|
|
status: str = Field(default="idle", description="Thread status: idle, busy, interrupted, error")
|
|
created_at: str = Field(default="", description="ISO timestamp")
|
|
updated_at: str = Field(default="", description="ISO timestamp")
|
|
metadata: dict[str, Any] = Field(default_factory=dict, description="Thread metadata")
|
|
values: dict[str, Any] = Field(default_factory=dict, description="Current state channel values")
|
|
interrupts: dict[str, Any] = Field(default_factory=dict, description="Pending interrupts")
|
|
|
|
|
|
class ThreadCreateRequest(BaseModel):
|
|
"""Request body for creating a thread."""
|
|
|
|
thread_id: ThreadId | None = Field(default=None, description="Optional thread ID (auto-generated if omitted)")
|
|
assistant_id: str | None = Field(default=None, description="Associate thread with an assistant")
|
|
metadata: dict[str, Any] = Field(default_factory=dict, description="Initial metadata")
|
|
project_id: str | None = Field(default=None, description="Assign the new thread to this project (validated server-side)")
|
|
|
|
_strip_reserved = field_validator("metadata")(classmethod(lambda cls, v: _strip_reserved_metadata(v)))
|
|
|
|
|
|
class ThreadSearchRequest(BaseModel):
|
|
"""Request body for searching threads."""
|
|
|
|
archived: bool | None = Field(default=None, strict=True, description="Archive filter; omitted includes all, false includes legacy unarchived threads")
|
|
metadata: dict[str, Any] = Field(default_factory=dict, description="Metadata filter (exact match)")
|
|
project_id: str | None = Field(default=None, description="Filter by project; explicit null = unassigned threads; omit key for all")
|
|
limit: int = Field(default=100, ge=1, le=1000, description="Maximum results")
|
|
offset: int = Field(default=0, ge=0, description="Pagination offset")
|
|
status: str | None = Field(default=None, description="Filter by thread status")
|
|
|
|
@field_validator("metadata")
|
|
@classmethod
|
|
def _validate_metadata_filters(cls, v: dict[str, Any]) -> dict[str, Any]:
|
|
"""Reject filter entries the SQL backend cannot compile.
|
|
|
|
Enforces consistent behaviour across SQL and memory backends.
|
|
See ``deerflow.persistence.json_compat`` for the shared validators.
|
|
"""
|
|
if not v:
|
|
return v
|
|
from deerflow.persistence.json_compat import validate_metadata_filter_key, validate_metadata_filter_value
|
|
|
|
bad_entries: list[str] = []
|
|
for key, value in v.items():
|
|
if not validate_metadata_filter_key(key):
|
|
bad_entries.append(f"{key!r} (unsafe key)")
|
|
elif not validate_metadata_filter_value(value):
|
|
bad_entries.append(f"{key!r} (unsupported value type {type(value).__name__})")
|
|
if bad_entries:
|
|
raise ValueError(f"Invalid metadata filter entries: {', '.join(bad_entries)}")
|
|
return v
|
|
|
|
|
|
class ThreadStateResponse(_MetadataRedactingResponse):
|
|
"""Response model for thread state."""
|
|
|
|
values: dict[str, Any] = Field(default_factory=dict, description="Current channel values")
|
|
next: list[str] = Field(default_factory=list, description="Next tasks to execute")
|
|
metadata: dict[str, Any] = Field(default_factory=dict, description="Checkpoint metadata")
|
|
checkpoint: dict[str, Any] = Field(default_factory=dict, description="Checkpoint info")
|
|
checkpoint_id: str | None = Field(default=None, description="Current checkpoint ID")
|
|
parent_checkpoint_id: str | None = Field(default=None, description="Parent checkpoint ID")
|
|
created_at: str | None = Field(default=None, description="Checkpoint timestamp")
|
|
tasks: list[dict[str, Any]] = Field(default_factory=list, description="Interrupted task details")
|
|
|
|
|
|
class ThreadPatchRequest(BaseModel):
|
|
"""Request body for patching thread metadata."""
|
|
|
|
metadata: dict[str, Any] = Field(default_factory=dict, description="Metadata to merge")
|
|
|
|
_strip_reserved = field_validator("metadata")(classmethod(lambda cls, v: _strip_reserved_metadata(v)))
|
|
|
|
@field_validator("metadata")
|
|
@classmethod
|
|
def validate_archive_flag(cls, value: dict[str, Any]) -> dict[str, Any]:
|
|
if THREAD_ARCHIVED_METADATA_KEY in value and not isinstance(value[THREAD_ARCHIVED_METADATA_KEY], bool):
|
|
raise ValueError("deerflow_archived must be a boolean")
|
|
return value
|
|
|
|
|
|
class ThreadStateUpdateRequest(BaseModel):
|
|
"""Request body for updating thread state (human-in-the-loop resume)."""
|
|
|
|
values: dict[str, Any] | None = Field(default=None, description="Channel values to merge")
|
|
checkpoint_id: str | None = Field(default=None, description="Checkpoint to branch from")
|
|
checkpoint: dict[str, Any] | None = Field(default=None, description="Full checkpoint object")
|
|
as_node: str | None = Field(default=None, description="Node identity for the update")
|
|
|
|
|
|
class ThreadGoalRequest(BaseModel):
|
|
"""Request body for setting a thread-scoped goal."""
|
|
|
|
objective: str = Field(..., min_length=1, max_length=4000, description="Completion condition for the agent to keep pursuing")
|
|
max_continuations: int = Field(
|
|
default=DEFAULT_MAX_GOAL_CONTINUATIONS,
|
|
ge=0,
|
|
le=DEFAULT_MAX_GOAL_CONTINUATIONS,
|
|
description="Maximum automatic hidden continuation turns before stopping",
|
|
)
|
|
|
|
|
|
class ThreadGoalResponse(BaseModel):
|
|
"""Response model for a thread goal."""
|
|
|
|
goal: dict[str, Any] | None = Field(default=None, description="Current goal state, or null when no goal is active")
|
|
|
|
|
|
class ThreadCompactRequest(BaseModel):
|
|
"""Request body for manually compacting a thread's active context."""
|
|
|
|
force: bool = Field(default=True, description="Run compaction even if automatic summarization thresholds are not met")
|
|
keep: ContextSize | None = Field(default=None, description="Optional retention policy for this compaction only")
|
|
agent_name: str | None = Field(default=None, max_length=128, description="Optional custom agent name for memory attribution")
|
|
model_name: str | None = Field(default=None, max_length=128, description="Optional model to summarize with; resolved request override -> custom-agent model -> default, mirroring run model selection")
|
|
|
|
|
|
class ThreadCompactResponse(BaseModel):
|
|
"""Response model for manual thread-context compaction."""
|
|
|
|
thread_id: str
|
|
compacted: bool
|
|
reason: str | None = None
|
|
removed_message_count: int = 0
|
|
preserved_message_count: int = 0
|
|
summary_updated: bool = False
|
|
checkpoint_id: str | None = None
|
|
total_tokens: int = 0
|
|
|
|
|
|
class HistoryEntry(_MetadataRedactingResponse):
|
|
"""Single checkpoint history entry."""
|
|
|
|
checkpoint_id: str
|
|
parent_checkpoint_id: str | None = None
|
|
metadata: dict[str, Any] = Field(default_factory=dict)
|
|
values: dict[str, Any] = Field(default_factory=dict)
|
|
created_at: str | None = None
|
|
next: list[str] = Field(default_factory=list)
|
|
|
|
|
|
class ThreadHistoryRequest(BaseModel):
|
|
"""Request body for checkpoint history."""
|
|
|
|
limit: int = Field(default=10, ge=1, le=100, description="Maximum entries")
|
|
before: str | None = Field(default=None, description="Cursor for pagination")
|
|
|
|
|
|
class ThreadBranchRequest(BaseModel):
|
|
"""Request body for creating a branch from a completed assistant turn."""
|
|
|
|
message_id: str = Field(..., min_length=1, description="Target assistant message ID to branch from")
|
|
message_ids: list[str] = Field(default_factory=list, description="All assistant message IDs in the target turn")
|
|
title: str | None = Field(default=None, max_length=256, description="Optional title for the branched thread")
|
|
|
|
|
|
class ThreadBranchResponse(BaseModel):
|
|
"""Response model for a thread branch."""
|
|
|
|
thread_id: str
|
|
parent_thread_id: str
|
|
parent_checkpoint_id: str
|
|
branched_from_message_id: str
|
|
workspace_clone_mode: str
|
|
# "seeded" | "skipped_empty" | "failed" — whether the parent history was
|
|
# copied into the branch's run-event feed (see branch_thread).
|
|
history_seed_mode: str
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _delete_thread_data(thread_id: str, paths: Paths | None = None, *, user_id: str | None = None) -> ThreadDeleteResponse:
|
|
"""Delete local persisted filesystem data for a thread."""
|
|
path_manager = paths or get_paths()
|
|
try:
|
|
path_manager.delete_thread_dir(thread_id, user_id=user_id)
|
|
except ValueError as exc:
|
|
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
|
except FileNotFoundError:
|
|
# Not critical — thread data may not exist on disk
|
|
logger.debug("No local thread data to delete for %s", sanitize_log_param(thread_id))
|
|
return ThreadDeleteResponse(success=True, message=f"No local data for {thread_id}")
|
|
except Exception as exc:
|
|
logger.exception("Failed to delete thread data for %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to delete local thread data.") from exc
|
|
|
|
logger.info("Deleted local thread data for %s", sanitize_log_param(thread_id))
|
|
return ThreadDeleteResponse(success=True, message=f"Deleted local thread data for {thread_id}")
|
|
|
|
|
|
async def _fetch_raw_pending_writes(checkpointer: Any, config: dict[str, Any]) -> list[Any]:
|
|
"""Fetch pending writes attached to a specific checkpoint.
|
|
|
|
Snapshot ``tasks`` only reflect writes that were pending while a task was
|
|
still scheduled; writes attached to the latest checkpoint afterwards
|
|
(rollback reattachment, worker error fallback) never surface there, so the
|
|
status derivation needs one raw tuple fetch on the resolved checkpoint.
|
|
"""
|
|
raw_tuple = await checkpointer.aget_tuple(config)
|
|
if raw_tuple is None:
|
|
return []
|
|
return list(getattr(raw_tuple, "pending_writes", ()) or ())
|
|
|
|
|
|
def _derive_thread_status(snapshot: Any, pending_writes: list[Any], *, fallback_status: str = "idle") -> str:
|
|
"""Derive thread status from the materialized snapshot plus the raw
|
|
pending writes attached to the resolved checkpoint."""
|
|
if snapshot is None:
|
|
return "idle"
|
|
|
|
for write in pending_writes:
|
|
if isinstance(write, (list, tuple)) and len(write) >= 2 and write[1] == "__error__":
|
|
return "error"
|
|
|
|
tasks = getattr(snapshot, "tasks", None) or ()
|
|
for task in tasks:
|
|
if getattr(task, "error", None) is not None:
|
|
return "error"
|
|
|
|
if not getattr(snapshot, "tasks_known", True):
|
|
return fallback_status
|
|
|
|
if tasks:
|
|
return "interrupted"
|
|
|
|
return "idle"
|
|
|
|
|
|
async def _ensure_thread_for_goal(thread_id: str, request: Request) -> None:
|
|
"""Ensure a thread_meta row and root checkpoint exist for goal commands."""
|
|
from app.gateway.deps import get_thread_store
|
|
|
|
thread_store = get_thread_store(request)
|
|
checkpointer = get_checkpointer(request)
|
|
thread_owner_user_id = get_trusted_internal_owner_user_id(request)
|
|
thread_owner_kwargs = {"user_id": thread_owner_user_id} if thread_owner_user_id else {}
|
|
|
|
record = await thread_store.get(thread_id, **thread_owner_kwargs)
|
|
if record is None and thread_owner_user_id:
|
|
unscoped_record = await thread_store.get(thread_id, user_id=None)
|
|
if unscoped_record is not None:
|
|
if unscoped_record.get("user_id") != thread_owner_user_id:
|
|
await thread_store.update_owner(thread_id, thread_owner_user_id, user_id=None)
|
|
record = await thread_store.get(thread_id, **thread_owner_kwargs)
|
|
if record is None:
|
|
try:
|
|
await thread_store.create(thread_id, metadata={}, **thread_owner_kwargs)
|
|
except Exception:
|
|
logger.exception("Failed to create thread_meta for goal thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to create thread") from None
|
|
|
|
try:
|
|
await ensure_thread_checkpoint(checkpointer, thread_id)
|
|
except Exception:
|
|
logger.exception("Failed to create goal checkpoint for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to create thread checkpoint") from None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Endpoints
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@router.delete("/{thread_id}", response_model=ThreadDeleteResponse)
|
|
@require_permission("threads", "delete", owner_check=True, require_existing=True)
|
|
async def delete_thread_data(thread_id: str, request: Request) -> ThreadDeleteResponse:
|
|
"""Delete local persisted filesystem data for a thread.
|
|
|
|
Cleans DeerFlow-managed thread directories, removes checkpoint data,
|
|
and removes the thread_meta row from the configured ThreadMetaStore
|
|
(sqlite or memory).
|
|
"""
|
|
run_manager = get_run_manager(request)
|
|
try:
|
|
async with goal_thread_lock(thread_id):
|
|
async with run_manager.reserve_thread_operation(
|
|
thread_id,
|
|
kind=ThreadOperationKind.delete,
|
|
user_id=get_effective_user_id(),
|
|
):
|
|
return await _delete_thread_data_with_reservation(thread_id, request)
|
|
except ConflictError:
|
|
raise HTTPException(
|
|
status_code=409,
|
|
detail="Thread has work in flight. Delete it after the work finishes.",
|
|
) from None
|
|
|
|
|
|
async def _delete_thread_data_with_reservation(thread_id: str, request: Request) -> ThreadDeleteResponse:
|
|
"""Delete a thread while its durable exclusive reservation is held."""
|
|
from app.gateway.deps import get_thread_store
|
|
|
|
# Legacy IDs may predate the canonical filesystem-safe contract. They can
|
|
# still be removed from metadata/checkpoint stores, but must never be
|
|
# interpolated into a host path during cleanup.
|
|
try:
|
|
validate_thread_id(thread_id)
|
|
except ValueError:
|
|
response = ThreadDeleteResponse(
|
|
success=True,
|
|
message="Skipped local data cleanup for legacy thread ID",
|
|
)
|
|
else:
|
|
response = _delete_thread_data(thread_id, user_id=get_effective_user_id())
|
|
|
|
# Remove checkpoints (best-effort)
|
|
checkpointer = getattr(request.app.state, "checkpointer", None)
|
|
if checkpointer is not None:
|
|
try:
|
|
if hasattr(checkpointer, "adelete_thread"):
|
|
await checkpointer.adelete_thread(thread_id)
|
|
except Exception:
|
|
logger.debug("Could not delete checkpoints for thread %s (not critical)", sanitize_log_param(thread_id))
|
|
|
|
# Remove thread_meta row (best-effort) — required for sqlite backend
|
|
# so the deleted thread no longer appears in /threads/search.
|
|
try:
|
|
thread_store = get_thread_store(request)
|
|
await thread_store.delete(thread_id)
|
|
except Exception:
|
|
logger.debug("Could not delete thread_meta for %s (not critical)", sanitize_log_param(thread_id))
|
|
|
|
# Tear down any live browser session (best-effort). Sessions are keyed only
|
|
# by thread_id, so leaving one alive after the owner deletes the thread lets
|
|
# a later caller who guesses the id reuse the retained page/cookies.
|
|
try:
|
|
from deerflow.community.browser_automation import get_browser_session_manager
|
|
|
|
await get_browser_session_manager().close_session(thread_id)
|
|
except ImportError:
|
|
pass # Playwright is an optional dependency.
|
|
except Exception:
|
|
logger.debug("Could not close browser session for %s (not critical)", sanitize_log_param(thread_id))
|
|
|
|
return response
|
|
|
|
|
|
async def _resolve_existing_thread(
|
|
thread_store: Any,
|
|
thread_id: str,
|
|
thread_owner_user_id: str | None,
|
|
thread_owner_kwargs: dict[str, Any],
|
|
) -> dict | None:
|
|
"""Return the existing thread_meta record for an idempotent create.
|
|
|
|
When the caller carries a trusted internal owner but only a legacy unscoped
|
|
(``user_id=None``) row exists, claim it for that owner before returning.
|
|
Both the fast path and the insert-race recovery path resolve through here so
|
|
a thread's ownership does not diverge based on which path found the record.
|
|
"""
|
|
existing_record = await thread_store.get(thread_id, **thread_owner_kwargs)
|
|
if existing_record is None and thread_owner_user_id:
|
|
await thread_store.claim_unowned(thread_id, thread_owner_user_id)
|
|
existing_record = await thread_store.get(thread_id, **thread_owner_kwargs)
|
|
return existing_record
|
|
|
|
|
|
def _existing_thread_response(thread_id: str, record: dict) -> ThreadResponse:
|
|
return ThreadResponse(
|
|
thread_id=thread_id,
|
|
status=record.get("status", "idle"),
|
|
created_at=coerce_iso(record.get("created_at", "")),
|
|
updated_at=coerce_iso(record.get("updated_at", "")),
|
|
metadata=record.get("metadata", {}),
|
|
)
|
|
|
|
|
|
@router.post("", response_model=ThreadResponse)
|
|
@require_permission("threads", "write")
|
|
async def create_thread(body: ThreadCreateRequest, request: Request) -> ThreadResponse:
|
|
"""Create a new thread.
|
|
|
|
Writes a thread_meta record (so the thread appears in /threads/search)
|
|
and an empty checkpoint (so state endpoints work immediately).
|
|
Idempotent: returns the existing record when ``thread_id`` already exists.
|
|
"""
|
|
from app.gateway.deps import get_thread_store
|
|
|
|
checkpointer = get_checkpointer(request)
|
|
thread_store = get_thread_store(request)
|
|
thread_id = resolve_thread_id(body.thread_id)
|
|
now = now_iso()
|
|
thread_owner_user_id = get_trusted_internal_owner_user_id(request)
|
|
thread_owner_kwargs = {"user_id": thread_owner_user_id} if thread_owner_user_id else {}
|
|
# ``body.metadata`` is already stripped of server-reserved keys by
|
|
# ``ThreadCreateRequest._strip_reserved`` — see the model definition.
|
|
|
|
# Idempotency: return existing record when already present
|
|
existing_record = await _resolve_existing_thread(thread_store, thread_id, thread_owner_user_id, thread_owner_kwargs)
|
|
if existing_record is not None:
|
|
return _existing_thread_response(thread_id, existing_record)
|
|
|
|
# Write thread_meta so the thread appears in /threads/search immediately
|
|
from deerflow.persistence.projects import ProjectNotAssignableError
|
|
|
|
try:
|
|
created_record = await thread_store.create(
|
|
thread_id,
|
|
assistant_id=getattr(body, "assistant_id", None),
|
|
**thread_owner_kwargs,
|
|
metadata=body.metadata,
|
|
project_id=body.project_id,
|
|
)
|
|
except ProjectNotAssignableError:
|
|
# Fail closed: missing, foreign, or archived projects are
|
|
# indistinguishable at the API surface.
|
|
raise HTTPException(status_code=404, detail="Project not found") from None
|
|
except ThreadOwnershipConflictError:
|
|
# Do not reveal that a caller-chosen id belongs to another user.
|
|
raise HTTPException(status_code=404, detail="Thread not found") from None
|
|
except IntegrityError:
|
|
# The idempotency read above and this insert are not atomic: a
|
|
# concurrent request for the same thread_id can commit in between, so
|
|
# the SQL-backed store rejects ours on the duplicate primary key.
|
|
# Honour the documented idempotency contract by resolving the
|
|
# now-existing record — running the same owner reconciliation the fast
|
|
# path does — instead of surfacing the conflict as a 500. The memory
|
|
# store serializes same-id creates under its per-thread lock and keeps
|
|
# its historical overwrite behavior, so it does not reach this branch.
|
|
existing_record = await _resolve_existing_thread(thread_store, thread_id, thread_owner_user_id, thread_owner_kwargs)
|
|
if existing_record is not None:
|
|
return _existing_thread_response(thread_id, existing_record)
|
|
# A duplicate-key error with no row we can read back is a real failure.
|
|
logger.exception("Failed to write thread_meta for %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to create thread")
|
|
except Exception:
|
|
# Any non-race failure must surface, not be silently swallowed as a 200.
|
|
logger.exception("Failed to write thread_meta for %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to create thread")
|
|
|
|
# Write an empty checkpoint so state endpoints work immediately
|
|
config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
|
|
try:
|
|
ckpt_metadata = {
|
|
"step": -1,
|
|
"source": "input",
|
|
"writes": None,
|
|
"parents": {},
|
|
**body.metadata,
|
|
"created_at": now,
|
|
}
|
|
await checkpointer.aput(config, empty_checkpoint(), ckpt_metadata, {})
|
|
except Exception:
|
|
logger.exception("Failed to create checkpoint for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to create thread")
|
|
|
|
logger.info("Thread created: %s", sanitize_log_param(thread_id))
|
|
# Respond from the persisted record — the store stamps
|
|
# ``metadata.deerflow_project_id`` from the assigned project_id column, so
|
|
# echoing ``body.metadata`` here would omit the membership the retry path
|
|
# (``_existing_thread_response``) reports.
|
|
return _existing_thread_response(thread_id, created_record)
|
|
|
|
|
|
@router.post("/{thread_id}/branches", response_model=ThreadBranchResponse)
|
|
@require_permission("threads", "write", owner_check=True, require_existing=True)
|
|
async def branch_thread(thread_id: ThreadId, body: ThreadBranchRequest, request: Request) -> ThreadBranchResponse:
|
|
"""Create a new main-thread branch from a completed assistant turn."""
|
|
try:
|
|
async with goal_thread_lock(thread_id):
|
|
async with get_run_manager(request).reserve_thread_operation(
|
|
thread_id,
|
|
kind=ThreadOperationKind.branch,
|
|
user_id=get_effective_user_id(),
|
|
):
|
|
return await _branch_thread_with_reservation(thread_id, body, request)
|
|
except ConflictError:
|
|
raise HTTPException(
|
|
status_code=409,
|
|
detail="Thread has work in flight. Branch it after the work finishes.",
|
|
) from None
|
|
|
|
|
|
async def _branch_thread_with_reservation(
|
|
thread_id: ThreadId,
|
|
body: ThreadBranchRequest,
|
|
request: Request,
|
|
) -> ThreadBranchResponse:
|
|
"""Create a branch while holding the source thread's exclusive reservation."""
|
|
from app.gateway.deps import get_thread_store
|
|
|
|
thread_store = get_thread_store(request)
|
|
|
|
source_record = await thread_store.get(thread_id)
|
|
if source_record is None:
|
|
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
|
|
|
|
source_metadata = source_record.get("metadata") or {}
|
|
if source_metadata.get(_SIDECAR_METADATA_KEY) is True:
|
|
raise HTTPException(status_code=409, detail="Branching is only available in the main conversation.")
|
|
source_accessor, source_config = await abuild_checkpoint_state_accessor(
|
|
request,
|
|
thread_id=thread_id,
|
|
assistant_id=source_record.get("assistant_id"),
|
|
)
|
|
|
|
target_message_ids = {body.message_id, *body.message_ids}
|
|
snapshot = await _find_branch_checkpoint(source_accessor, source_config, target_message_ids)
|
|
parent_checkpoint_id = _checkpoint_id(snapshot)
|
|
if not parent_checkpoint_id:
|
|
raise HTTPException(status_code=409, detail="This turn can no longer be branched from.")
|
|
target_human = _branch_target_human_message(_checkpoint_messages(snapshot), target_message_ids)
|
|
target_human_id = _message_id(target_human)
|
|
if not target_human_id:
|
|
raise HTTPException(status_code=409, detail="This turn can no longer be branched from.")
|
|
replay_base_tuple = await _find_branch_replay_base(
|
|
source_accessor,
|
|
source_config,
|
|
snapshot,
|
|
target_human_id,
|
|
)
|
|
|
|
# Workspace files are not checkpointed, so they only reflect the *current* thread
|
|
# state. Cloning them onto a branch from an older turn would leak files created
|
|
# after that turn (message history rolls back, workspace would not). Restrict the
|
|
# best-effort clone to branches taken from the latest turn so history and workspace
|
|
# stay consistent.
|
|
branch_from_latest_turn = await _branch_targets_latest_turn(source_accessor, source_config, target_message_ids)
|
|
|
|
new_thread_id = str(uuid.uuid4())
|
|
now = now_iso()
|
|
branch_metadata = {
|
|
_BRANCH_METADATA_KEY: True,
|
|
"branch_parent_thread_id": thread_id,
|
|
"branch_parent_checkpoint_id": parent_checkpoint_id,
|
|
"branch_parent_message_id": body.message_id,
|
|
"branch_created_at": now,
|
|
}
|
|
# A branch extends its source conversation, so the new row inherits the
|
|
# source thread's project membership — an unassigned branch would surface
|
|
# under Recent chats instead of the source thread's project group. The
|
|
# store re-validates the project inside the insert (same fail-closed path
|
|
# as create/move), and the inherited id may be stale only when the project
|
|
# was archived or deleted after the source read.
|
|
from deerflow.persistence.projects import ProjectNotAssignableError
|
|
|
|
source_project_id = (source_metadata or {}).get(THREAD_PROJECT_METADATA_KEY)
|
|
|
|
if body.title:
|
|
display_name = body.title
|
|
else:
|
|
sibling_records = await _branch_sibling_records(thread_store, thread_id)
|
|
display_name, title_sequence = _default_branch_title(
|
|
source_record.get("display_name"),
|
|
source_is_branch=source_metadata.get(_BRANCH_METADATA_KEY) is True,
|
|
source_sequence=source_metadata.get(_BRANCH_TITLE_SEQUENCE_METADATA_KEY),
|
|
sibling_records=sibling_records,
|
|
)
|
|
if title_sequence is not None:
|
|
branch_metadata[_BRANCH_TITLE_SEQUENCE_METADATA_KEY] = title_sequence
|
|
thread_owner_user_id = get_trusted_internal_owner_user_id(request)
|
|
thread_owner_kwargs = {"user_id": thread_owner_user_id} if thread_owner_user_id else {}
|
|
|
|
# Copy materialized values with replace semantics: reducer channels must
|
|
# not re-merge an already-aggregated value, so every copied reducer value
|
|
# is wrapped in Overwrite (not just messages).
|
|
branch_accessor, new_config = build_checkpoint_state_mutation_accessor(
|
|
request,
|
|
thread_id=new_thread_id,
|
|
as_node="branch",
|
|
# The branch write carries the full materialized snapshot; use the
|
|
# source assistant's effective schema so extension middleware channels
|
|
# survive instead of being silently discarded as unknown channels.
|
|
state_schema=graph_state_schema(getattr(source_accessor, "graph", None)),
|
|
)
|
|
branch_reducer_fields = graph_reducer_channels(getattr(branch_accessor, "graph", None))
|
|
if branch_reducer_fields is None:
|
|
branch_reducer_fields = THREAD_STATE_REDUCER_FIELDS
|
|
|
|
def branch_values(source_snapshot: Any) -> dict[str, Any]:
|
|
values: dict[str, Any] = {}
|
|
for key, value in dict(source_snapshot.values).items():
|
|
if key in _BRANCH_EXCLUDED_CHANNELS:
|
|
continue
|
|
if key in branch_reducer_fields:
|
|
values[key] = Overwrite(list(value) if key == "messages" and isinstance(value, list) else value)
|
|
else:
|
|
values[key] = value
|
|
if display_name is not None:
|
|
values["title"] = display_name
|
|
return values
|
|
|
|
# Stamp both synthetic checkpoints with the branch-creation time because
|
|
# serializers fall back to metadata when snapshot.created_at is absent.
|
|
checkpoint_metadata_updates = {
|
|
**branch_metadata,
|
|
"source": "branch",
|
|
"updated_at": now,
|
|
"created_at": now,
|
|
}
|
|
new_config.setdefault("metadata", {}).update(checkpoint_metadata_updates)
|
|
try:
|
|
head_config = new_config
|
|
if replay_base_tuple is not None:
|
|
head_config = await branch_accessor.aupdate(
|
|
new_config,
|
|
branch_values(replay_base_tuple),
|
|
as_node="branch",
|
|
)
|
|
head_config.setdefault("metadata", {}).update(checkpoint_metadata_updates)
|
|
await branch_accessor.aupdate(
|
|
head_config,
|
|
branch_values(snapshot),
|
|
as_node="branch",
|
|
)
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, new_thread_id) from exc
|
|
except Exception:
|
|
logger.exception("Failed to write branch checkpoint for thread %s", sanitize_log_param(new_thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to create branch") from None
|
|
|
|
async def _write_branch_row(project_id: str | None) -> None:
|
|
try:
|
|
await thread_store.create(
|
|
new_thread_id,
|
|
assistant_id=source_record.get("assistant_id"),
|
|
display_name=display_name,
|
|
metadata=branch_metadata,
|
|
project_id=project_id,
|
|
**thread_owner_kwargs,
|
|
)
|
|
except ProjectNotAssignableError:
|
|
raise
|
|
except Exception:
|
|
logger.exception("Failed to write branch thread_meta for %s", sanitize_log_param(new_thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to create branch") from None
|
|
|
|
try:
|
|
await _write_branch_row(source_project_id)
|
|
except ProjectNotAssignableError:
|
|
# The source project became unassignable (archived, or deleted in a
|
|
# race that cleared the source row's membership after our read): keep
|
|
# the branch usable as an unassigned thread — the pre-inheritance
|
|
# behavior for sources without an active project.
|
|
await _write_branch_row(None)
|
|
|
|
# The thread feed (GET /messages, /messages/page) reads the run-event
|
|
# store, not checkpoints, and a fresh branch has no run_events — so the
|
|
# inherited history would vanish from the UI as soon as the branch's
|
|
# first run refreshes the feed (#4380 problem 2). Seed the branch's
|
|
# run_events from the same checkpoint snapshot the branch was created
|
|
# from. Best-effort: on failure the branch stays usable, with history
|
|
# visible only through the checkpoint overlay until it is re-branched.
|
|
try:
|
|
seed_events = build_branch_history_seed_events(
|
|
_checkpoint_messages(snapshot),
|
|
thread_id=new_thread_id,
|
|
run_id_prefix=f"branch-seed-{new_thread_id}",
|
|
parent_thread_id=thread_id,
|
|
)
|
|
if seed_events:
|
|
await get_run_event_store(request).put_batch(seed_events)
|
|
history_seed_mode = "seeded"
|
|
else:
|
|
history_seed_mode = "skipped_empty"
|
|
except Exception:
|
|
logger.exception("Failed to seed branch history run-events for thread %s", sanitize_log_param(new_thread_id))
|
|
history_seed_mode = "failed"
|
|
|
|
if branch_from_latest_turn:
|
|
workspace_clone_mode = await _copy_branch_user_data(thread_id, new_thread_id)
|
|
else:
|
|
workspace_clone_mode = "skipped_historical_turn"
|
|
return ThreadBranchResponse(
|
|
thread_id=new_thread_id,
|
|
parent_thread_id=thread_id,
|
|
parent_checkpoint_id=parent_checkpoint_id,
|
|
branched_from_message_id=body.message_id,
|
|
workspace_clone_mode=workspace_clone_mode,
|
|
history_seed_mode=history_seed_mode,
|
|
)
|
|
|
|
|
|
@router.post("/search", response_model=list[ThreadResponse])
|
|
@require_permission("threads", "read")
|
|
async def search_threads(body: ThreadSearchRequest, request: Request) -> list[ThreadResponse]:
|
|
"""Search and list threads.
|
|
|
|
Delegates to the configured ThreadMetaStore implementation
|
|
(SQL-backed for sqlite/postgres, Store-backed for memory mode).
|
|
"""
|
|
from app.gateway.deps import get_thread_store
|
|
from deerflow.persistence.thread_meta import InvalidMetadataFilterError
|
|
|
|
repo = get_thread_store(request)
|
|
# Three-state project filter: key absent → no filter; explicit null →
|
|
# unassigned threads only; string → members of that project.
|
|
project_filter = body.project_id if "project_id" in body.model_fields_set else PROJECT_FILTER_UNSET
|
|
try:
|
|
rows = await repo.search(
|
|
metadata=body.metadata or None,
|
|
status=body.status,
|
|
**({"archived": body.archived} if body.archived is not None else {}),
|
|
project_id=project_filter,
|
|
limit=body.limit,
|
|
offset=body.offset,
|
|
)
|
|
except InvalidMetadataFilterError as exc:
|
|
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
|
return [
|
|
ThreadResponse(
|
|
thread_id=r["thread_id"],
|
|
status=r.get("status", "idle"),
|
|
# ``coerce_iso`` heals legacy unix-second values that
|
|
# ``MemoryThreadMetaStore`` historically wrote with ``time.time()``;
|
|
# SQL-backed rows already arrive as ISO strings and pass through.
|
|
created_at=coerce_iso(r.get("created_at", "")),
|
|
updated_at=coerce_iso(r.get("updated_at", "")),
|
|
metadata=r.get("metadata", {}),
|
|
values={"title": r["display_name"]} if r.get("display_name") else {},
|
|
interrupts={},
|
|
)
|
|
for r in rows
|
|
]
|
|
|
|
|
|
@router.patch("/{thread_id}", response_model=ThreadResponse)
|
|
@require_permission("threads", "write", owner_check=True, require_existing=True)
|
|
async def patch_thread(thread_id: ThreadId, body: ThreadPatchRequest, request: Request) -> ThreadResponse:
|
|
"""Merge metadata into a thread record."""
|
|
from app.gateway.deps import get_thread_store
|
|
|
|
thread_store = get_thread_store(request)
|
|
record = await thread_store.get(thread_id)
|
|
if record is None:
|
|
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
|
|
|
|
# ``body.metadata`` already stripped by ``ThreadPatchRequest._strip_reserved``.
|
|
# Pin/unpin and archive/restore are not conversation activity, so it must not bump ``updated_at``.
|
|
# Other metadata PATCH callers keep the public endpoint's existing recency
|
|
# contract unless they get their own explicit no-touch API surface.
|
|
touch = not _is_organization_metadata_patch(body.metadata)
|
|
try:
|
|
await thread_store.update_metadata(thread_id, body.metadata, touch=touch)
|
|
except Exception:
|
|
logger.exception("Failed to patch thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to update thread")
|
|
|
|
# Re-read to get the merged metadata and the store's timestamp decision.
|
|
record = await thread_store.get(thread_id) or record
|
|
return ThreadResponse(
|
|
thread_id=thread_id,
|
|
status=record.get("status", "idle"),
|
|
created_at=coerce_iso(record.get("created_at", "")),
|
|
updated_at=coerce_iso(record.get("updated_at", "")),
|
|
metadata=record.get("metadata", {}),
|
|
)
|
|
|
|
|
|
class ThreadMoveRequest(BaseModel):
|
|
"""Request body for moving a thread into/out of a project."""
|
|
|
|
project_id: str | None = Field(..., description="Target project id, or null to unassign")
|
|
|
|
|
|
@router.post("/{thread_id}/move", response_model=ThreadResponse)
|
|
@require_permission("threads", "write", owner_check=True, require_existing=True)
|
|
async def move_thread(thread_id: ThreadId, body: ThreadMoveRequest, request: Request) -> ThreadResponse:
|
|
"""Move a thread between projects (or out). Organizational only: history,
|
|
run state, and per-thread files are untouched (RFC v2 §6)."""
|
|
from app.gateway.deps import get_thread_store
|
|
|
|
thread_store = get_thread_store(request)
|
|
moved = await thread_store.set_project(thread_id, body.project_id)
|
|
if not moved:
|
|
raise HTTPException(status_code=404, detail="Thread or project not found")
|
|
record = await thread_store.get(thread_id)
|
|
return ThreadResponse(
|
|
thread_id=thread_id,
|
|
status=record.get("status", "idle"),
|
|
created_at=coerce_iso(record.get("created_at", "")),
|
|
updated_at=coerce_iso(record.get("updated_at", "")),
|
|
metadata=record.get("metadata", {}),
|
|
)
|
|
|
|
|
|
@router.get("/{thread_id}", response_model=ThreadResponse)
|
|
@require_permission("threads", "read", owner_check=True)
|
|
async def get_thread(thread_id: ThreadId, request: Request) -> ThreadResponse:
|
|
"""Get thread info from metadata plus the graph's materialized state."""
|
|
from app.gateway.deps import get_thread_store
|
|
|
|
thread_store = get_thread_store(request)
|
|
checkpointer = get_checkpointer(request)
|
|
record: dict | None = await thread_store.get(thread_id)
|
|
try:
|
|
accessor, config = await abuild_checkpoint_state_accessor(
|
|
request,
|
|
thread_id=thread_id,
|
|
assistant_id=record.get("assistant_id") if record is not None else None,
|
|
)
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
|
|
try:
|
|
snapshot = await accessor.aget(config)
|
|
checkpoint_id = (snapshot.config or {}).get("configurable", {}).get("checkpoint_id")
|
|
pending_writes = await _fetch_raw_pending_writes(checkpointer, snapshot.config) if checkpoint_id else []
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
except Exception:
|
|
logger.exception("Failed to get checkpoint for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to get thread")
|
|
|
|
if record is None and not checkpoint_id:
|
|
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
|
|
|
|
metadata = snapshot.metadata or {}
|
|
if record is None:
|
|
record = {
|
|
"thread_id": thread_id,
|
|
"status": "idle",
|
|
"created_at": coerce_iso(snapshot.created_at or metadata.get("created_at", "")),
|
|
"updated_at": coerce_iso(metadata.get("updated_at", snapshot.created_at or metadata.get("created_at", ""))),
|
|
"metadata": {key: value for key, value in metadata.items() if key not in ("created_at", "updated_at", "step", "source", "writes", "parents")},
|
|
}
|
|
stored_status = record.get("status", "idle")
|
|
status = _derive_thread_status(snapshot, pending_writes, fallback_status=stored_status) if checkpoint_id else stored_status
|
|
|
|
return ThreadResponse(
|
|
thread_id=thread_id,
|
|
status=status,
|
|
created_at=coerce_iso(record.get("created_at", "")),
|
|
updated_at=coerce_iso(record.get("updated_at", "")),
|
|
metadata=record.get("metadata", {}),
|
|
values=serialize_channel_values_for_api(snapshot.values),
|
|
)
|
|
|
|
|
|
@router.get("/{thread_id}/goal", response_model=ThreadGoalResponse)
|
|
@require_permission("threads", "read", owner_check=True)
|
|
async def get_thread_goal(thread_id: ThreadId, request: Request) -> ThreadGoalResponse:
|
|
"""Return the active Claude-style goal for a thread, if any."""
|
|
checkpointer = get_checkpointer(request)
|
|
try:
|
|
goal = await read_thread_goal(checkpointer, thread_id)
|
|
except Exception:
|
|
logger.exception("Failed to read goal for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to read thread goal") from None
|
|
return ThreadGoalResponse(goal=goal)
|
|
|
|
|
|
@router.put("/{thread_id}/goal", response_model=ThreadGoalResponse)
|
|
@require_permission("threads", "write", owner_check=True)
|
|
async def set_thread_goal(thread_id: ThreadId, body: ThreadGoalRequest, request: Request) -> ThreadGoalResponse:
|
|
"""Set or replace the active goal for a thread.
|
|
|
|
``/chats/new`` pages already hold a generated UUID before the first run, so
|
|
this endpoint creates the missing thread checkpoint on demand.
|
|
"""
|
|
checkpointer = get_checkpointer(request)
|
|
try:
|
|
goal = build_goal_state(body.objective, max_continuations=body.max_continuations)
|
|
async with reserve_checkpoint_write(request, thread_id, user_id=get_effective_user_id()):
|
|
await _ensure_thread_for_goal(thread_id, request)
|
|
await write_thread_goal(checkpointer, thread_id, goal, as_node="goal", create_if_missing=True)
|
|
except ValueError as exc:
|
|
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
|
except ConflictError:
|
|
raise HTTPException(status_code=409, detail="Thread has a run in flight. Set the goal after the run finishes.") from None
|
|
except HTTPException:
|
|
raise
|
|
except Exception:
|
|
logger.exception("Failed to set goal for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to set thread goal") from None
|
|
return ThreadGoalResponse(goal=goal)
|
|
|
|
|
|
@router.delete("/{thread_id}/goal", response_model=ThreadGoalResponse)
|
|
@require_permission("threads", "write", owner_check=True)
|
|
async def clear_thread_goal(thread_id: ThreadId, request: Request) -> ThreadGoalResponse:
|
|
"""Clear the active goal for a thread."""
|
|
checkpointer = get_checkpointer(request)
|
|
try:
|
|
async with reserve_checkpoint_write(request, thread_id, user_id=get_effective_user_id()):
|
|
await write_thread_goal(checkpointer, thread_id, None, as_node="goal")
|
|
except ConflictError:
|
|
raise HTTPException(status_code=409, detail="Thread has a run in flight. Clear the goal after the run finishes.") from None
|
|
except LookupError:
|
|
return ThreadGoalResponse(goal=None)
|
|
except Exception:
|
|
logger.exception("Failed to clear goal for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to clear thread goal") from None
|
|
return ThreadGoalResponse(goal=None)
|
|
|
|
|
|
def _thread_compact_response(result: ThreadCompactionResult) -> ThreadCompactResponse:
|
|
return ThreadCompactResponse(
|
|
thread_id=result.thread_id,
|
|
compacted=result.compacted,
|
|
reason=result.reason,
|
|
removed_message_count=result.removed_message_count,
|
|
preserved_message_count=result.preserved_message_count,
|
|
summary_updated=result.summary_updated,
|
|
checkpoint_id=result.checkpoint_id,
|
|
total_tokens=result.total_tokens,
|
|
)
|
|
|
|
|
|
@router.post("/{thread_id}/compact", response_model=ThreadCompactResponse)
|
|
@require_permission("threads", "write", owner_check=True, require_existing=True)
|
|
async def compact_thread(thread_id: ThreadId, body: ThreadCompactRequest, request: Request) -> ThreadCompactResponse:
|
|
"""Manually summarize old thread context while preserving the visible history."""
|
|
# Compaction writes only base-schema channels (messages + summary_text);
|
|
# every other channel — including middleware-contributed ones — is carried
|
|
# forward by checkpoint fork inheritance, so the base-schema mutation
|
|
# graph is sufficient (and avoids building the full lead graph per call).
|
|
try:
|
|
accessor, _ = build_checkpoint_state_mutation_accessor(
|
|
request,
|
|
thread_id=thread_id,
|
|
as_node="manual_compaction",
|
|
)
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
keep = body.keep.to_tuple() if body.keep is not None else None
|
|
try:
|
|
async with reserve_checkpoint_write(request, thread_id, user_id=get_effective_user_id()):
|
|
result = await compact_thread_context(
|
|
accessor,
|
|
thread_id,
|
|
keep=keep,
|
|
force=body.force,
|
|
user_id=get_effective_user_id(),
|
|
agent_name=body.agent_name,
|
|
model_name=body.model_name,
|
|
)
|
|
except ConflictError:
|
|
raise HTTPException(status_code=409, detail="Thread has a run in flight. Compact after the run finishes.") from None
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
except ContextCompactionDisabled:
|
|
raise HTTPException(status_code=409, detail="Context compaction is disabled.") from None
|
|
except ContextCompactionFailed:
|
|
raise HTTPException(status_code=500, detail="Failed to compact thread context.") from None
|
|
except LookupError:
|
|
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found") from None
|
|
except HTTPException:
|
|
raise
|
|
except Exception:
|
|
logger.exception("Failed to compact thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to compact thread context.") from None
|
|
return _thread_compact_response(result)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
@router.get("/{thread_id}/state", response_model=ThreadStateResponse)
|
|
@require_permission("threads", "read", owner_check=True)
|
|
async def get_thread_state(thread_id: ThreadId, request: Request) -> ThreadStateResponse:
|
|
"""Get the latest materialized graph state for a thread."""
|
|
# Resolve through the thread's assistant so custom middleware channels
|
|
# appear in the response instead of being dropped by the default schema.
|
|
try:
|
|
accessor, config = await build_thread_checkpoint_state_accessor(request, thread_id=thread_id)
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
try:
|
|
snapshot = await accessor.aget(config)
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
except Exception:
|
|
logger.exception("Failed to get state for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to get thread state")
|
|
|
|
snapshot_config = snapshot.config or {}
|
|
checkpoint_id = snapshot_config.get("configurable", {}).get("checkpoint_id")
|
|
if not checkpoint_id:
|
|
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
|
|
|
|
parent_config = snapshot.parent_config or {}
|
|
parent_checkpoint_id = parent_config.get("configurable", {}).get("checkpoint_id")
|
|
metadata = snapshot.metadata or {}
|
|
created_at = snapshot.created_at or metadata.get("created_at", "")
|
|
tasks_raw = snapshot.tasks or ()
|
|
tasks = [{"id": getattr(task, "id", ""), "name": getattr(task, "name", "")} for task in tasks_raw]
|
|
|
|
values = serialize_channel_values_for_api(snapshot.values)
|
|
messages = values.get("messages")
|
|
if isinstance(messages, list) and messages:
|
|
# Same reason as the history endpoint: a client reading the checkpoint
|
|
# over REST needs the feed position the stream would have stamped.
|
|
values["messages"] = await stamp_messages_with_seq(_optional_run_event_store(request), thread_id, messages)
|
|
|
|
return ThreadStateResponse(
|
|
values=values,
|
|
next=list(snapshot.next or ()),
|
|
metadata=metadata,
|
|
checkpoint={"id": checkpoint_id, "ts": coerce_iso(created_at)},
|
|
checkpoint_id=checkpoint_id,
|
|
parent_checkpoint_id=parent_checkpoint_id,
|
|
created_at=coerce_iso(created_at),
|
|
tasks=tasks,
|
|
)
|
|
|
|
|
|
@router.post("/{thread_id}/state", response_model=ThreadStateResponse)
|
|
@require_permission("threads", "write", owner_check=True, require_existing=True)
|
|
async def update_thread_state(thread_id: ThreadId, body: ThreadStateUpdateRequest, request: Request) -> ThreadStateResponse:
|
|
"""Replace selected thread-state fields through the materialized graph."""
|
|
from app.gateway.deps import get_thread_store
|
|
|
|
thread_store = get_thread_store(request)
|
|
if body.checkpoint_id is not None:
|
|
if not body.checkpoint_id:
|
|
raise HTTPException(status_code=404, detail="Checkpoint not found")
|
|
selected_config = {
|
|
"configurable": {
|
|
"thread_id": thread_id,
|
|
"checkpoint_ns": "",
|
|
"checkpoint_id": body.checkpoint_id,
|
|
}
|
|
}
|
|
try:
|
|
checkpoint_tuple = await get_checkpointer(request).aget_tuple(selected_config)
|
|
except Exception:
|
|
logger.exception("Failed to get state for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to get thread state")
|
|
if checkpoint_tuple is None:
|
|
raise HTTPException(status_code=404, detail=f"Checkpoint {body.checkpoint_id} not found")
|
|
|
|
mutation_node = body.as_node or "manual_state_update"
|
|
# Resolve through the shared boundary (thread metadata -> assistant_id ->
|
|
# effective schema) so extension middleware channels stay writable.
|
|
accessor, read_config = await build_thread_checkpoint_state_mutation_accessor(
|
|
request,
|
|
thread_id=thread_id,
|
|
as_node=mutation_node,
|
|
checkpoint_id=body.checkpoint_id,
|
|
)
|
|
# These values go straight into a checkpoint, so they need the same
|
|
# server-owned-metadata stripping the run path gets inside normalize_input.
|
|
# Without it an authenticated client can persist forged provenance and
|
|
# transform trails, which later readers are entitled to treat as facts
|
|
# about what the host itself did.
|
|
values = strip_server_owned_state_metadata(dict(body.values or {}))
|
|
writable_channels = graph_writable_channels(getattr(accessor, "graph", None))
|
|
if writable_channels is not None:
|
|
unknown_fields = sorted(set(values) - writable_channels)
|
|
if unknown_fields:
|
|
raise HTTPException(
|
|
status_code=422,
|
|
detail=f"Unknown thread-state field(s): {', '.join(unknown_fields)}",
|
|
)
|
|
reducer_fields = graph_reducer_channels(getattr(accessor, "graph", None))
|
|
if reducer_fields is None:
|
|
reducer_fields = THREAD_STATE_REDUCER_FIELDS
|
|
updates = {key: Overwrite(value) if key in reducer_fields else value for key, value in values.items()}
|
|
try:
|
|
async with reserve_checkpoint_write(request, thread_id, user_id=get_effective_user_id()):
|
|
updated_config = await accessor.aupdate(
|
|
read_config,
|
|
updates,
|
|
as_node=mutation_node,
|
|
)
|
|
snapshot = await accessor.aget(updated_config)
|
|
except ConflictError:
|
|
raise HTTPException(status_code=409, detail="Thread has a run in flight. Update state after the run finishes.") from None
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
except Exception:
|
|
logger.exception("Failed to update state for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to update thread state")
|
|
|
|
if thread_store and body.values and "title" in body.values:
|
|
new_title = body.values["title"]
|
|
if new_title:
|
|
try:
|
|
await thread_store.update_display_name(
|
|
thread_id,
|
|
new_title,
|
|
remove_metadata_keys=(_BRANCH_TITLE_SEQUENCE_METADATA_KEY,),
|
|
)
|
|
except Exception:
|
|
logger.debug("Failed to sync title to thread_meta for %s (non-fatal)", sanitize_log_param(thread_id))
|
|
|
|
snapshot_config = snapshot.config or {}
|
|
checkpoint_id = snapshot_config.get("configurable", {}).get("checkpoint_id")
|
|
parent_config = snapshot.parent_config or {}
|
|
parent_checkpoint_id = parent_config.get("configurable", {}).get("checkpoint_id")
|
|
metadata = snapshot.metadata or {}
|
|
created_at = snapshot.created_at or metadata.get("created_at", "")
|
|
tasks_raw = snapshot.tasks or ()
|
|
tasks = [{"id": getattr(task, "id", ""), "name": getattr(task, "name", "")} for task in tasks_raw]
|
|
|
|
return ThreadStateResponse(
|
|
values=serialize_channel_values_for_api(snapshot.values),
|
|
next=list(snapshot.next or ()),
|
|
metadata=metadata,
|
|
checkpoint={"id": checkpoint_id, "ts": coerce_iso(created_at)},
|
|
checkpoint_id=checkpoint_id,
|
|
parent_checkpoint_id=parent_checkpoint_id,
|
|
created_at=coerce_iso(created_at),
|
|
tasks=tasks,
|
|
)
|
|
|
|
|
|
def _checkpoint_run_durations(metadata: Any) -> dict[str, int]:
|
|
raw_durations = metadata.get("run_durations") if isinstance(metadata, dict) else None
|
|
if not isinstance(raw_durations, dict):
|
|
return {}
|
|
return {run_id: duration_seconds for run_id, duration_seconds in raw_durations.items() if valid_duration_entry(run_id, duration_seconds)}
|
|
|
|
|
|
def _checkpoint_run_message_ids(metadata: Any) -> dict[str, str]:
|
|
raw_message_run_ids = metadata.get(RUN_MESSAGE_IDS_METADATA_KEY) if isinstance(metadata, dict) else None
|
|
if not isinstance(raw_message_run_ids, dict):
|
|
return {}
|
|
return {message_id: run_id for message_id, run_id in raw_message_run_ids.items() if valid_run_message_id_entry(message_id, run_id)}
|
|
|
|
|
|
async def _load_run_durations(
|
|
*,
|
|
run_manager: Any,
|
|
thread_id: str,
|
|
user_id: str | None,
|
|
run_ids: set[str],
|
|
) -> dict[str, int]:
|
|
"""Batch-hydrate the requested runs and compute their latest durations."""
|
|
if not run_ids:
|
|
return {}
|
|
|
|
from app.gateway.routers.thread_runs import compute_run_durations
|
|
|
|
runs = await run_manager.list_by_thread(
|
|
thread_id,
|
|
user_id=user_id,
|
|
limit=max(100, len(run_ids)),
|
|
)
|
|
known_run_ids = {run.run_id for run in runs}
|
|
for run_id in sorted(run_ids - known_run_ids):
|
|
run = await run_manager.get(run_id, user_id=user_id)
|
|
if run is not None:
|
|
runs.append(run)
|
|
known_run_ids.add(run_id)
|
|
|
|
computed_durations = compute_run_durations(runs)
|
|
return {run_id: duration for run_id, duration in computed_durations.items() if run_id in run_ids}
|
|
|
|
|
|
async def _persist_run_history_metadata_background(
|
|
*,
|
|
request: Request,
|
|
checkpointer: Any,
|
|
thread_id: str,
|
|
user_id: str | None,
|
|
duration_run_ids: set[str],
|
|
message_run_ids: dict[str, str],
|
|
audited_message_ids: set[str],
|
|
) -> None:
|
|
"""Best-effort history migration behind durable checkpoint admission."""
|
|
from deerflow.runtime.runs.worker import persist_run_history_metadata
|
|
|
|
try:
|
|
async with reserve_checkpoint_write(request, thread_id, user_id=user_id):
|
|
from app.gateway.deps import get_run_event_store, get_run_manager
|
|
|
|
authoritative_message_run_ids = dict(message_run_ids)
|
|
authoritative_duration_run_ids = set(duration_run_ids)
|
|
if audited_message_ids:
|
|
exact_after_admission = await get_run_event_store(request).find_latest_ai_message_run_ids(
|
|
thread_id,
|
|
audited_message_ids,
|
|
user_id=user_id,
|
|
)
|
|
for message_id in audited_message_ids:
|
|
exact_run_id = exact_after_admission.get(message_id)
|
|
if valid_run_message_id_entry(message_id, exact_run_id):
|
|
if authoritative_message_run_ids.get(message_id) != exact_run_id:
|
|
authoritative_duration_run_ids.add(exact_run_id)
|
|
authoritative_message_run_ids[message_id] = exact_run_id
|
|
|
|
authoritative_durations = await _load_run_durations(
|
|
run_manager=get_run_manager(request),
|
|
thread_id=thread_id,
|
|
user_id=user_id,
|
|
run_ids=authoritative_duration_run_ids,
|
|
)
|
|
|
|
await persist_run_history_metadata(
|
|
checkpointer=checkpointer,
|
|
thread_id=thread_id,
|
|
durations=authoritative_durations,
|
|
message_run_ids=authoritative_message_run_ids,
|
|
)
|
|
except ConflictError:
|
|
# A live run or another checkpoint writer owns the thread. The mapping
|
|
# is a read-through optimization, so the next history request can retry
|
|
# instead of racing a user-visible state mutation.
|
|
logger.debug("Skipped run-history metadata migration for busy thread %s", sanitize_log_param(thread_id))
|
|
except Exception:
|
|
logger.warning("Failed to persist run-history metadata for thread %s", sanitize_log_param(thread_id), exc_info=True)
|
|
|
|
|
|
@router.post("/{thread_id}/history", response_model=list[HistoryEntry])
|
|
@require_permission("threads", "read", owner_check=True)
|
|
async def get_thread_history(
|
|
thread_id: ThreadId,
|
|
body: ThreadHistoryRequest,
|
|
request: Request,
|
|
background_tasks: BackgroundTasks,
|
|
) -> list[HistoryEntry]:
|
|
"""Get materialized graph state history for a thread.
|
|
|
|
Only the latest (first) checkpoint carries the ``messages`` key to
|
|
avoid duplicating the complete conversation across every entry.
|
|
"""
|
|
checkpointer = get_checkpointer(request)
|
|
try:
|
|
accessor, config = await build_thread_checkpoint_state_accessor(
|
|
request,
|
|
thread_id=thread_id,
|
|
checkpoint_id=body.before,
|
|
)
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
|
|
entries: list[HistoryEntry] = []
|
|
is_latest_checkpoint = True
|
|
try:
|
|
snapshots = await accessor.ahistory(config, limit=body.limit)
|
|
for snapshot in snapshots:
|
|
snapshot_config = snapshot.config or {}
|
|
parent_config = snapshot.parent_config or {}
|
|
metadata = snapshot.metadata or {}
|
|
materialized_values = snapshot.values if isinstance(snapshot.values, dict) else {}
|
|
|
|
checkpoint_id = snapshot_config.get("configurable", {}).get("checkpoint_id", "")
|
|
parent_id = parent_config.get("configurable", {}).get("checkpoint_id")
|
|
|
|
values: dict[str, Any] = {}
|
|
if title := materialized_values.get("title"):
|
|
values["title"] = title
|
|
if thread_data := materialized_values.get("thread_data"):
|
|
values["thread_data"] = thread_data
|
|
|
|
if is_latest_checkpoint:
|
|
messages = materialized_values.get("messages")
|
|
if messages:
|
|
serialized_msgs = serialize_channel_values_for_api({"messages": messages}).get("messages", [])
|
|
try:
|
|
from app.gateway.routers.thread_runs import stamp_turn_duration_on_last_ai
|
|
|
|
# Human messages define turn boundaries. New checkpoints
|
|
# carry the completed turns' durations in metadata, so the
|
|
# messages channel stays unchanged.
|
|
checkpoint_run_durations = _checkpoint_run_durations(metadata)
|
|
checkpoint_run_message_ids = _checkpoint_run_message_ids(metadata)
|
|
current_turn_run_id = None
|
|
turn_run_ids: set[str] = set()
|
|
legacy_ai_message_ids: set[str] = set()
|
|
for msg in serialized_msgs:
|
|
if msg.get("type") == "human":
|
|
additional_kwargs = msg.get("additional_kwargs")
|
|
if isinstance(additional_kwargs, dict):
|
|
run_id = additional_kwargs.get("run_id")
|
|
if isinstance(run_id, str) and run_id:
|
|
current_turn_run_id = run_id
|
|
continue
|
|
|
|
message_type = msg.get("type")
|
|
if message_type not in {"ai", "tool"}:
|
|
continue
|
|
|
|
if message_type == "ai":
|
|
message_id = msg.get("id")
|
|
persisted_run_id = checkpoint_run_message_ids.get(message_id) if isinstance(message_id, str) else None
|
|
if persisted_run_id:
|
|
msg["run_id"] = persisted_run_id
|
|
elif not isinstance(msg.get("run_id"), str) or not msg.get("run_id"):
|
|
if current_turn_run_id:
|
|
msg["run_id"] = current_turn_run_id
|
|
if isinstance(message_id, str) and message_id:
|
|
legacy_ai_message_ids.add(message_id)
|
|
|
|
run_id = msg.get("run_id")
|
|
if isinstance(run_id, str) and run_id:
|
|
turn_run_ids.add(run_id)
|
|
elif current_turn_run_id:
|
|
msg.setdefault("run_id", current_turn_run_id)
|
|
|
|
# Runs referenced by this checkpoint's AI messages but
|
|
# absent from duration metadata are either legacy
|
|
# (never migrated) or just completed. Exact attribution
|
|
# has its own completeness condition: duration-only
|
|
# checkpoints written before #4949 still need their AI
|
|
# IDs correlated and persisted. Correlate once via the
|
|
# event store, then hydrate only the run rows whose
|
|
# durations are actually required.
|
|
resolved_run_durations = dict(checkpoint_run_durations)
|
|
missing_run_ids = turn_run_ids - set(checkpoint_run_durations)
|
|
if missing_run_ids or legacy_ai_message_ids:
|
|
from app.gateway.deps import get_run_event_store, get_run_manager
|
|
|
|
run_mgr = get_run_manager(request)
|
|
event_store = get_run_event_store(request)
|
|
user_id = get_effective_user_id()
|
|
ai_message_ids = set(legacy_ai_message_ids)
|
|
try:
|
|
msg_to_run = (
|
|
await event_store.find_latest_ai_message_run_ids(
|
|
thread_id,
|
|
ai_message_ids,
|
|
user_id=user_id,
|
|
)
|
|
if ai_message_ids
|
|
else {}
|
|
)
|
|
except Exception:
|
|
# A failed exact lookup must not masquerade as a
|
|
# successful boundary attribution. Removing the
|
|
# synthesized ids leaves the response incomplete
|
|
# rather than deterministically wrong. Durations
|
|
# backed by persisted mappings remain provable
|
|
# and should still survive this degraded path.
|
|
for msg in serialized_msgs:
|
|
if msg.get("type") == "ai" and msg.get("id") in ai_message_ids:
|
|
msg.pop("run_id", None)
|
|
stamp_turn_duration_on_last_ai(
|
|
serialized_msgs,
|
|
checkpoint_run_durations,
|
|
)
|
|
raise
|
|
|
|
for msg in serialized_msgs:
|
|
if msg.get("type") != "ai":
|
|
continue
|
|
exact_run_id = msg_to_run.get(msg.get("id"))
|
|
if exact_run_id:
|
|
msg["run_id"] = exact_run_id
|
|
|
|
# Cache the complete audited attribution, including
|
|
# boundary fallbacks for IDs with no event. Without
|
|
# those negative-result entries, every history read
|
|
# would rescan the same pre-event-store prefix.
|
|
message_run_ids_to_persist = {
|
|
message_id: run_id
|
|
for msg in serialized_msgs
|
|
if msg.get("type") == "ai" and isinstance((message_id := msg.get("id")), str) and message_id in ai_message_ids and isinstance((run_id := msg.get("run_id")), str) and run_id
|
|
}
|
|
required_run_ids = {run_id for msg in serialized_msgs if msg.get("type") == "ai" and isinstance((run_id := msg.get("run_id")), str) and run_id and run_id not in checkpoint_run_durations}
|
|
run_durations = await _load_run_durations(
|
|
run_manager=run_mgr,
|
|
thread_id=thread_id,
|
|
user_id=user_id,
|
|
run_ids=required_run_ids,
|
|
)
|
|
resolved_run_durations.update(run_durations)
|
|
|
|
# Intentional, best-effort write-on-read migration:
|
|
# persist both exact attribution and duration after
|
|
# the response so subsequent reads stay exact without
|
|
# waiting on an active stream's checkpoint lock.
|
|
if required_run_ids or message_run_ids_to_persist:
|
|
background_tasks.add_task(
|
|
_persist_run_history_metadata_background,
|
|
request=request,
|
|
checkpointer=checkpointer,
|
|
thread_id=thread_id,
|
|
user_id=user_id,
|
|
duration_run_ids=required_run_ids,
|
|
message_run_ids=message_run_ids_to_persist,
|
|
audited_message_ids=ai_message_ids,
|
|
)
|
|
|
|
# Stamp only after exact attribution is final. Stamping
|
|
# the synthesized boundary first can leave its duration
|
|
# attached to a message whose run ID is later corrected.
|
|
stamp_turn_duration_on_last_ai(
|
|
serialized_msgs,
|
|
resolved_run_durations,
|
|
)
|
|
|
|
except Exception:
|
|
logger.warning("Failed to inject turn_duration for thread %s", sanitize_log_param(thread_id), exc_info=True)
|
|
|
|
# The stream stamps `values` frames as they are published, but a
|
|
# client that only opens a conversation never sees one — this is
|
|
# the read it does instead, and without a seq a rescued early turn
|
|
# has no absolute position to be placed at (#4666).
|
|
values["messages"] = await stamp_messages_with_seq(
|
|
_optional_run_event_store(request),
|
|
thread_id,
|
|
serialized_msgs,
|
|
)
|
|
|
|
is_latest_checkpoint = False
|
|
|
|
next_tasks = list(snapshot.next or ())
|
|
|
|
# Strip LangGraph internal keys from metadata
|
|
user_meta = {k: v for k, v in metadata.items() if k not in ("created_at", "updated_at", "step", "source", "writes", "parents", "run_durations", RUN_MESSAGE_IDS_METADATA_KEY)}
|
|
# Keep step for ordering context
|
|
if "step" in metadata:
|
|
user_meta["step"] = metadata["step"]
|
|
|
|
entries.append(
|
|
HistoryEntry(
|
|
checkpoint_id=checkpoint_id,
|
|
parent_checkpoint_id=parent_id,
|
|
metadata=user_meta,
|
|
values=values,
|
|
created_at=coerce_iso(snapshot.created_at or metadata.get("created_at", "")),
|
|
next=next_tasks,
|
|
)
|
|
)
|
|
except _CHECKPOINT_MODE_ERRORS as exc:
|
|
raise _checkpoint_mode_http_error(exc, thread_id) from exc
|
|
except Exception:
|
|
logger.exception("Failed to get history for thread %s", sanitize_log_param(thread_id))
|
|
raise HTTPException(status_code=500, detail="Failed to get thread history")
|
|
|
|
return entries
|