mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-11-08 03:30:34 +00:00
1dd6d05627
* feat(extension-api): add task identity fields to CompactionEvent (0.2.10) CompactionEvent carried the transform, its hashes and its counts, but nothing that names the task, run or thread the compaction happened in. An observer that wants to relate a compaction to the run that produced it had to match hashes across sessions, which is unreliable. Append three optional fields, task_id, run_id and thread_id (str | None, default None). Existing keyword and positional construction is unchanged, so already-released extensions keep working. The host fills them in the next commit. Bump deerflow-extension-api to 0.2.10 and the harness's exact pin with it. * fix(extensions): hand compaction observers the live task store and its identity The summarization middleware already recovers the live task store from the runtime to observe its summary system call, and the registry allocates that store precisely because a compaction observer is registered. notify_context_compacted() then discarded it and handed every observer a detached store, with a docstring claiming there was no live task. - The lead run worker and the subagent executor now seed every task store they allocate with the task's TaskInfo, not only when a lifecycle contributor is registered; the lifecycle notification reuses the seeded object. A subagent without a parent run_id still gets no TaskInfo. - compact_state() and acompact_state() pass the store from task_store_from_runtime() to _record_compaction(). - notify_context_compacted() takes an optional task_store. A live store goes to the observers and fills the event's task_id/run_id/thread_id from its TaskInfo; without one (manual /compact runs outside any run) observers still receive a detached store and the ids stay None. For a background subagent the store keeps its public scope id (the provider tool_call_id) while TaskInfo.task_id stays the execution id, as lifecycle contributors already see it; the event follows TaskInfo. * docs(extensions): document compaction event identity and seeded TaskInfo The extensions guide and the developer manual still said compaction observers always receive a detached store because no task is live at the call site. Describe what the host does now: lead runs and subagents with a parent run seed every task store with TaskInfo, compaction observers get the live store, and CompactionEvent's task_id/run_id/thread_id come from it; only manual /compact falls back to a detached store with None ids. Adds the 0.2.10 row to the contract version table (PR number pending). * docs(changelog): note compaction observers receive the live task identity * docs(extensions): list compaction observers among task-store triggers * docs: reference #7831 for the compaction event identity change * docs(extensions): backfill contract history rows 0.2.5 to 0.2.9 The version table jumped from 0.2.4 to 0.2.10 although 0.2.5-0.2.9 shipped. Add those rows, attributing names merged without a bump to the first release that carried them (#5796 in 0.2.5, #6920 and #5645 in 0.2.8, #7729 in 0.2.10). The page intro now says it documents 0.2.4 plus the 0.2.10 CompactionEvent fields, and points to the history for the rest.
69 lines
3.0 KiB
Python
69 lines
3.0 KiB
Python
"""The transform that replaces many messages with one.
|
|
|
|
Summarization is destructive by design: N messages leave the context and one
|
|
summary enters it. Afterwards only the summary exists, so nothing downstream can
|
|
answer "which messages became this?" — the mapping is gone. It has to be emitted
|
|
where it is still true.
|
|
|
|
Fields are keyed on canonical content hashes (``deerflow_extension_api.canonical_hash``),
|
|
not a producer-stamped identity key: nothing in the host currently mints a stable
|
|
per-message identity for the messages a compaction consumes or the summary it
|
|
produces, so an identity-keyed field would ship permanently empty in every event.
|
|
Every message has content, so hashing it is always available.
|
|
|
|
The exact recipe both sides must use: ``canonical_hash(message.content)`` — the
|
|
message's ``content`` attribute passed directly, never pre-stringified. DeerFlow
|
|
messages are routinely multimodal (``list[dict]`` content), and ``str()`` on a
|
|
dict renders insertion order, so two logically identical messages would hash
|
|
differently if stringified first. ``canonical_hash`` already normalizes through
|
|
``canonical_json`` (sorted keys, deterministic separators); passing it anything
|
|
else throws that normalization away.
|
|
|
|
Trap for consumers: ``DurableContextMiddleware`` is the only place the produced summary
|
|
text later reaches a message (the ``durable_context_data`` block), but
|
|
``_render_durable_context_data`` renders a *bounded, HTML-escaped* projection of
|
|
``summary_text`` for the model prompt, not the summary itself. Hashing that rendering
|
|
will not equal ``output_content_hash``. A consumer must join a compaction to what came
|
|
after it through this event alone, never by re-hashing a later projection of the same
|
|
text.
|
|
|
|
``task_id`` / ``run_id`` / ``thread_id`` name the task the compaction happened in. The
|
|
host copies them from the ``TaskInfo`` it seeds into the live task store — the same
|
|
store the observer receives — so a consumer can key a compaction on
|
|
``(thread_id, run_id)`` instead of matching hashes across sessions. They stay ``None``
|
|
when no task identity exists: a manual ``/compact`` outside any run (observers then
|
|
receive a detached store) or a subagent executed without a parent run.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from dataclasses import dataclass
|
|
from typing import Protocol
|
|
|
|
from deerflow_extension_api.state import ExtensionData
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class CompactionEvent:
|
|
"""What a compaction consumed and produced, captured while both still exist."""
|
|
|
|
transform_kind: str
|
|
transform_version: str
|
|
source_content_hashes: tuple[str, ...]
|
|
output_content_hash: str
|
|
compacted_message_count: int
|
|
kept_message_count: int
|
|
task_id: str | None = None
|
|
run_id: str | None = None
|
|
thread_id: str | None = None
|
|
|
|
|
|
class ContextCompactionObserver(Protocol):
|
|
async def on_context_compacted(
|
|
self,
|
|
app_store: ExtensionData,
|
|
task_store: ExtensionData,
|
|
event: CompactionEvent,
|
|
) -> None:
|
|
return None
|