deer-flow/backend/packages/harness/deerflow/tracing/AGENTS.md

Tracing System (packages/harness/deerflow/tracing/)

LangSmith and Langfuse are both supported. The wiring lives in two layers:

• factory.py::build_tracing_callbacks() — returns the LangChain CallbackHandler list for the providers currently enabled via env vars (LANGSMITH_TRACING, LANGFUSE_TRACING, etc.). The handlers are attached at the graph invocation root for in-graph runs (make_lead_agent and DeerFlowClient.stream both append them to config["callbacks"] before invoking the graph) so a single run produces one trace with all node / LLM / tool calls as child spans. Standalone callers — anything that invokes a model outside such a graph (e.g. MemoryUpdater) — keep create_chat_model's default attach_tracing=True, which falls back to model-level callback attachment.
• metadata.py::build_langfuse_trace_metadata() — builds the Langfuse-reserved trace attributes for RunnableConfig.metadata. The Langfuse v4 langchain.CallbackHandler lifts these onto the root trace (see its _parse_langfuse_trace_attributes), but only when it sees on_chain_start(parent_run_id=None) — which is why the callbacks have to live at the graph root, not the model.

Trace-attribute injection points: both runtime/runs/worker.py::run_agent (gateway path) and client.py::DeerFlowClient.stream (embedded path) merge the metadata into config["metadata"] right before constructing the graph. subagents/executor.py::_aexecute does the same for every subagent run so subagent traces group under the parent thread's session card (carrying the parent thread_id → langfuse_session_id, the user_id captured at task_tool → langfuse_user_id, and a subagent:<normalized-name> trace name). Caller-supplied keys win via setdefault, so an external session_id override is preserved. Field mapping:

Langfuse field	Source
langfuse_session_id	LangGraph thread_id
langfuse_user_id	get_effective_user_id() (default in no-auth); for subagents, captured from runtime.context at task_tool time via resolve_runtime_user_id()
langfuse_trace_name	RunRecord.assistant_id / client agent_name (defaults to lead-agent); for subagents, subagent:<name> (lowercased, _ → -)
langfuse_tags	env:<DEER_FLOW_ENV> + model:<model_name>
deerflow_trace_id	Current request/entry trace id from deerflow.trace_context; matches X-Trace-Id for enhanced Gateway HTTP requests. Gated by logging.enhance.enabled in both gateway and embedded paths via is_trace_correlation_enabled — off by default; embedded callers can still opt in per-turn by wrapping stream() in request_trace_context(...)

Returns {} when Langfuse is not in the enabled providers — LangSmith-only deployments are unaffected. Set DEER_FLOW_ENV (or ENVIRONMENT) to tag traces by deployment environment. Tests live in tests/test_tracing_factory.py, tests/test_tracing_metadata.py, tests/test_worker_langfuse_metadata.py, tests/test_client_langfuse_metadata.py, and tests/test_subagent_executor.py::TestSubagentTracingWiring.

Monocle telemetry is a third provider, structurally unlike LangSmith/Langfuse. It is not a LangChain callback: tracing/monocle.py::setup_monocle_tracing_if_enabled() calls monocle_apptrace.setup_monocle_telemetry() once, which installs a process-global OTel TracerProvider, patches span serialization, and auto-instruments the openai/langchain/langgraph clients. Because that is a one-time, process-global side effect (not a per-run callback), it is initialized from the Gateway lifespan (app/gateway/app.py) — never from build_tracing_callbacks() — and it is off by default. The setup call was deliberately moved out of agents/__init__.py, so import deerflow.agents must never start tracing (pinned by tests/test_monocle_tracing.py::test_no_import_time_setup). The Gateway lifespan is the sole call site (pinned by test_gateway_lifespan_initializes_monocle), so unlike LangSmith/Langfuse — which attach at the graph roots and cover every path — the embedded DeerFlowClient and the TUI are not instrumented; embedded users who want Monocle traces call setup_monocle_tracing_if_enabled() themselves before running the agent.

Unlike the Langfuse metadata above, DeerFlow injects no per-run fields into Monocle traces — the only attribute it sets is workflow_name="deer-flow"; every span attribute (span.type, entity.*, token usage, span inputs/outputs, scope.agentic.session) is produced by Monocle's own metamodel and auto-instrumentation, so there is no DeerFlow trace-attribute layer to maintain here.

Config is env-driven like the others — MonocleTracingConfig, built in get_tracing_config() and gated by is_monocle_tracing_enabled(). MONOCLE_TRACING enables it; MONOCLE_EXPORTERS selects exporters (default file → trace JSON in .monocle/; also console, okahu, s3, blob, gcs, where okahu requires OKAHU_API_KEY). setup_monocle_tracing_if_enabled() stays a thin wrapper on purpose: monocle_apptrace already guards duplicate setup (instrumentor.py::check_duplicate_setup) and never force-overrides an existing global provider, so the wrapper only gates on config. Coexistence with Langfuse (v4, also OTel-based) is verified: whichever library initializes second reuses the existing global TracerProvider and attaches its own span processor, so neither side loses spans (pinned by test_coexists_with_langfuse). Both processors see all spans, so Monocle's exporters also capture Langfuse's spans when both are enabled. (LangSmith is a plain callback and coexists trivially.) Tests: tests/test_monocle_tracing.py.