mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-19 11:06:18 +00:00
feat(skills): rank deferred discovery by agent intent (#5369)
* feat(skills): rank deferred discovery by intent * fix(skills): preserve exact selections and cache search metadata --------- Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
PeaceMaker-best
GitHub
parent
f2857ad3ff
commit
e831720304
@ -964,6 +964,8 @@ A standard Agent Skill is a structured capability module — a Markdown file tha
|
||||
|
||||
Skills are loaded progressively — only when the task needs them, not all at once. This keeps the context window lean and makes DeerFlow work well even with token-sensitive models.
|
||||
|
||||
When deferred skill discovery is enabled, `describe_skill` ranks installed skills by bounded, Unicode-normalized intent-term coverage across names and descriptions. Natural multi-term requests can therefore find a relevant skill without requiring one exact phrase, while exact `select:` and required-name `+prefix` lookups remain available. Ranked searches use up to 256 characters and return up to five results; exact `select:` lists are not truncated and return all requested catalog matches.
|
||||
|
||||
A skill directory is a package boundary: once DeerFlow finds its `SKILL.md`, nested `SKILL.md` files under that package (for example evaluation fixtures) remain supporting data and are not registered as runtime skills. Namespace directories without their own `SKILL.md` can still group nested skills.
|
||||
|
||||
Skill Markdown and bundled text resources use UTF-8. Skill-creator CLI and review utilities read and write text explicitly as UTF-8 so localized skills behave consistently across operating systems.
|
||||
|
||||
@ -8,7 +8,7 @@
|
||||
- **Sandbox projection**: `skills/projection.py` materializes enabled-only shared trees at `{base_dir}/skills_view/public` and `{base_dir}/users/{user_id}/skills_view/{custom,legacy,integrations}`. A lead Agent with an explicit `skills` allowlist (including `[]`) gets the intersection of enabled public/user-visible skills and that allowlist at `{base_dir}/users/{user_id}/threads/{thread_id}/skills_view/{public,custom,legacy,integrations}`. `skills=None` keeps the shared zero-copy mount until a thread has used an explicit policy; later unrestricted runs repopulate the same stable thread root with all enabled skills. Rebuilds sign source state, view state, and normalized policy in a manifest, revoke every old category before adding the new policy, stage copies in temporary directories, and atomically replace files. Category root inodes stay stable for live bind mounts; concurrent readers can briefly see fewer skills during a policy change, never a skill revoked by the new policy. Policy-scoped copies reject absolute symlinks and relative symlinks that resolve outside their own skill package, preventing a permitted package from linking back to an omitted source. It copies files into the view (`_copy_into_view`) so a sandbox write cannot mutate the canonical skill inode; the operational trade-off is an O(total bytes) I/O and per-user/thread storage multiplier across rebuilds, prioritized for write isolation over zero-copy hardlinks. Steady-state freshness checks combine source and view metadata tree digests, so in-sandbox view tampering is detected and repaired on the next acquire. Storage writes, archive installs, deletes, and toggles rebuild shared scopes under a cross-process lock; Gateway boot ensures only the shared public view, user views are repaired lazily on acquire, and Agent thread views are recomputed before sandbox reuse. Managed integration packages are global, but their projected category is per-user because enabled state is isolated. User projection rebuilds re-read global enable state from disk instead of the process singleton, so a toggle handled by another Gateway worker is reflected on the next acquire. Gateway public-skill toggles take the public projection lock before the shared `extensions_config_write_lock`, re-read an existing config raw from disk, change only that skill's entry, and rebuild before responding; keep this as one worker-owned critical section so MCP writes cannot interleave and request cancellation cannot release either lock while the worker still runs. The shared public steady-state signature check runs without the global projection lock; stale/error paths take the lock and re-check before rebuilding or clearing. User and thread scope checks are serialized per scope. Projection failures clear the affected view before raising.
|
||||
- **Injection (legacy / default)**: Enabled skills are listed in the agent system prompt with full metadata and container paths (`<available_skills>` block). Controlled by `skills.deferred_discovery: false` (default).
|
||||
- **Deferred discovery** (`skills.deferred_discovery: true`): Skills are listed by name only in a compact `<skill_index>` block, keeping the system prompt prefix-cache friendly. The agent calls the `describe_skill` tool at runtime to fetch full metadata for skills it wants to use, then loads the SKILL.md via `read_file`. Two new modules support this path:
|
||||
- `skills/catalog.py` — `SkillCatalog` (immutable, searchable; query forms: `select:a,b`, `+prefix`, free-text regex); `select:` returns all requested skills without a result cap; other modes cap at `MAX_RESULTS=5`.
|
||||
- `skills/catalog.py` — immutable `SkillCatalog`; query forms: `select:a,b`, `+prefix`, free-text intent terms. Ranked queries use up to 256 characters / 16 unique literal terms, preferring name matches over description-only matches at equal coverage; ties keep catalog order. A lazy per-catalog index caches normalized names/descriptions. Unlike tool search, ranking uses literal intent terms, not regexes. `select:` is parsed before search limits and returns all exact catalog matches without query-length or result caps; other modes cap at `MAX_RESULTS=5`.
|
||||
- `skills/describe.py` — `build_describe_skill_tool(catalog)` builds the `describe_skill` tool as a closure; `build_skill_search_setup(skills, enabled, ...)` produces a `SkillSearchSetup(describe_skill_tool, skill_names)` that is wired into both the LangGraph agent factory (`agent.py`) and the embedded client (`client.py`).
|
||||
- **Slash activation**: `/skill-name task` loads that enabled skill's `SKILL.md` for the current model call only. The resolver rejects leading whitespace, missing separators, reserved channel commands (`/new`, `/help`, `/bootstrap`, `/status`, `/models`, `/memory`, `/goal`, `/agent`), disabled skills, and skills outside a custom agent's whitelist.
|
||||
- **Installation**: `POST /api/skills/install` extracts .skill ZIP archive to custom/ directory
|
||||
|
||||
@ -1,8 +1,9 @@
|
||||
"""Skill catalog — deferred skill discovery at runtime.
|
||||
|
||||
Mirrors ``DeferredToolCatalog`` from ``tool_search.py``: an immutable, searchable
|
||||
catalog that lets the LLM discover skill metadata on demand rather than having
|
||||
every skill's full description baked into the system prompt.
|
||||
Like ``DeferredToolCatalog`` from ``tool_search.py``, this immutable catalog
|
||||
exposes metadata on demand instead of embedding full descriptions in prompts.
|
||||
Query forms are shared, but skills intentionally use literal intent ranking
|
||||
rather than the tool catalog's free-text regex matching.
|
||||
|
||||
The agent sees skill names in ``<skill_index>`` but cannot read their metadata
|
||||
until it calls ``describe_skill``. This keeps the system prompt compact and
|
||||
@ -13,6 +14,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
import unicodedata
|
||||
from dataclasses import dataclass
|
||||
from functools import cached_property
|
||||
|
||||
@ -21,18 +23,97 @@ from deerflow.skills.types import Skill
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
MAX_RESULTS = 5
|
||||
MAX_QUERY_CHARS = 256
|
||||
MAX_QUERY_TERMS = 16
|
||||
|
||||
_NAME_SEPARATOR_RE = re.compile(r"[-_./]+")
|
||||
_TOKEN_RE = re.compile(r"[^\W_]+", re.UNICODE)
|
||||
_WHITESPACE_RE = re.compile(r"\s+")
|
||||
_IGNORED_SINGLE_ASCII_TERMS = frozenset({"a", "i"})
|
||||
|
||||
|
||||
def _compile_catalog_regex(pattern: str) -> re.Pattern[str]:
|
||||
"""Compile ``pattern`` case-insensitively, falling back to literal match.
|
||||
def _normalize_search_text(value: str) -> str:
|
||||
"""Return Unicode-normalized, separator-aware text for matching."""
|
||||
normalized = unicodedata.normalize("NFKC", value).casefold()
|
||||
normalized = _NAME_SEPARATOR_RE.sub(" ", normalized)
|
||||
return _WHITESPACE_RE.sub(" ", normalized).strip()
|
||||
|
||||
Search queries come from the model, so an invalid regex (e.g. an unbalanced
|
||||
paren) must degrade to a literal substring match rather than raise.
|
||||
|
||||
def _query_terms(query: str) -> tuple[str, ...]:
|
||||
"""Extract a bounded set of unique literal intent terms.
|
||||
|
||||
The English article/pronoun ``a``/``I`` are discarded because they would
|
||||
otherwise match almost every catalog entry. Other single-character terms
|
||||
stay meaningful for skills such as C++ or R.
|
||||
"""
|
||||
try:
|
||||
return re.compile(pattern, re.IGNORECASE)
|
||||
except re.error:
|
||||
return re.compile(re.escape(pattern), re.IGNORECASE)
|
||||
terms: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for term in _TOKEN_RE.findall(_normalize_search_text(query[:MAX_QUERY_CHARS])):
|
||||
if term in _IGNORED_SINGLE_ASCII_TERMS:
|
||||
continue
|
||||
if term in seen:
|
||||
continue
|
||||
seen.add(term)
|
||||
terms.append(term)
|
||||
if len(terms) == MAX_QUERY_TERMS:
|
||||
break
|
||||
return tuple(terms)
|
||||
|
||||
|
||||
def _contains_term(text: str, term: str) -> bool:
|
||||
if len(term) == 1 and term.isascii():
|
||||
return term in _TOKEN_RE.findall(text)
|
||||
return term in text
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _SearchEntry:
|
||||
skill: Skill
|
||||
normalized_name: str
|
||||
normalized_description: str
|
||||
|
||||
|
||||
def _intent_score(entry: _SearchEntry, *, normalized_query: str, terms: tuple[str, ...]) -> tuple[int, int, int, int, int] | None:
|
||||
"""Score one skill by intent coverage without external retrieval state."""
|
||||
normalized_name = entry.normalized_name
|
||||
normalized_description = entry.normalized_description
|
||||
name_matches = tuple(_contains_term(normalized_name, term) for term in terms)
|
||||
description_matches = tuple(_contains_term(normalized_description, term) for term in terms)
|
||||
name_hits = sum(name_matches)
|
||||
matched_terms = sum(name_match or description_match for name_match, description_match in zip(name_matches, description_matches, strict=True))
|
||||
if not matched_terms:
|
||||
return None
|
||||
|
||||
return (
|
||||
int(normalized_name == normalized_query),
|
||||
matched_terms,
|
||||
int(normalized_query in normalized_name),
|
||||
name_hits,
|
||||
int(normalized_query in normalized_description),
|
||||
)
|
||||
|
||||
|
||||
def _rank_by_intent(entries: tuple[_SearchEntry, ...], query: str, *, include_unmatched: bool = False) -> list[Skill]:
|
||||
normalized_query = _normalize_search_text(query)
|
||||
terms = _query_terms(query)
|
||||
if not normalized_query or not terms:
|
||||
return [entry.skill for entry in entries[:MAX_RESULTS]] if include_unmatched else []
|
||||
|
||||
scored: list[tuple[tuple[int, int, int, int, int], Skill]] = []
|
||||
unmatched: list[Skill] = []
|
||||
for entry in entries:
|
||||
score = _intent_score(entry, normalized_query=normalized_query, terms=terms)
|
||||
if score is None:
|
||||
unmatched.append(entry.skill)
|
||||
else:
|
||||
scored.append((score, entry.skill))
|
||||
|
||||
# Python's sort is stable, so equal-score skills retain catalog order.
|
||||
scored.sort(key=lambda item: item[0], reverse=True)
|
||||
ranked = [skill for _, skill in scored]
|
||||
if include_unmatched:
|
||||
ranked.extend(unmatched)
|
||||
return ranked[:MAX_RESULTS]
|
||||
|
||||
|
||||
# NOTE: frozen=True without slots=True keeps __dict__, which is what lets the
|
||||
@ -42,11 +123,11 @@ def _compile_catalog_regex(pattern: str) -> re.Pattern[str]:
|
||||
class SkillCatalog:
|
||||
"""Immutable catalog of skills. Pure search, no mutation.
|
||||
|
||||
Query forms (mirror ``DeferredToolCatalog.search``):
|
||||
Query forms (shared with tool search; ranking semantics differ):
|
||||
|
||||
- ``"select:data-analysis,deep-research"`` — exact match by name.
|
||||
- ``"+podcast gen"`` — require *podcast* in the name, rank by *gen*.
|
||||
- ``"chart visualization"`` — regex match on name + description.
|
||||
- ``"chart visualization"`` — multi-term intent match on name + description.
|
||||
"""
|
||||
|
||||
skills: tuple[Skill, ...]
|
||||
@ -56,10 +137,17 @@ class SkillCatalog:
|
||||
"""All skill names in insertion order."""
|
||||
return frozenset(s.name for s in self.skills)
|
||||
|
||||
@cached_property
|
||||
def _search_index(self) -> tuple[_SearchEntry, ...]:
|
||||
"""Normalize immutable skill metadata once per catalog, in catalog order."""
|
||||
return tuple(_SearchEntry(skill, _normalize_search_text(skill.name), _normalize_search_text(skill.description or "")) for skill in self.skills)
|
||||
|
||||
def search(self, query: str) -> list[Skill]:
|
||||
"""Match *query* against skill names and descriptions.
|
||||
|
||||
Returns at most ``MAX_RESULTS`` skills, ranked by relevance.
|
||||
Exact ``select:`` queries have no query-length or result cap.
|
||||
Other queries use at most ``MAX_QUERY_CHARS`` characters and return
|
||||
at most ``MAX_RESULTS`` skills, ranked by relevance.
|
||||
"""
|
||||
query = query.strip()
|
||||
if not query:
|
||||
@ -70,33 +158,20 @@ class SkillCatalog:
|
||||
wanted = {n.strip() for n in query[7:].split(",")}
|
||||
return [s for s in self.skills if s.name in wanted]
|
||||
|
||||
query = query[:MAX_QUERY_CHARS]
|
||||
|
||||
# ── Required-prefix search ─────────────────────────────────────
|
||||
if query.startswith("+"):
|
||||
parts = query[1:].split(None, 1)
|
||||
if not parts:
|
||||
return [] # bare "+" with no required token
|
||||
required = parts[0].lower()
|
||||
candidates = [s for s in self.skills if required in s.name.lower()]
|
||||
required = _normalize_search_text(parts[0])
|
||||
if not _TOKEN_RE.search(required):
|
||||
return []
|
||||
candidates = tuple(entry for entry in self._search_index if required in entry.normalized_name)
|
||||
if len(parts) > 1:
|
||||
pattern = _compile_catalog_regex(parts[1])
|
||||
candidates.sort(
|
||||
key=lambda s: _catalog_regex_score(pattern, s),
|
||||
reverse=True,
|
||||
)
|
||||
return candidates[:MAX_RESULTS]
|
||||
return _rank_by_intent(candidates, parts[1], include_unmatched=True)
|
||||
return [entry.skill for entry in candidates[:MAX_RESULTS]]
|
||||
|
||||
# ── Free-text regex search ─────────────────────────────────────
|
||||
regex = _compile_catalog_regex(query)
|
||||
scored: list[tuple[int, Skill]] = []
|
||||
for s in self.skills:
|
||||
searchable = f"{s.name} {s.description or ''}"
|
||||
if regex.search(searchable):
|
||||
# Name match scores higher than description-only match.
|
||||
scored.append((2 if regex.search(s.name) else 1, s))
|
||||
scored.sort(key=lambda x: x[0], reverse=True)
|
||||
return [s for _, s in scored][:MAX_RESULTS]
|
||||
|
||||
|
||||
def _catalog_regex_score(pattern: re.Pattern[str], s: Skill) -> int:
|
||||
"""Count regex hits across name + description for ranking."""
|
||||
return len(pattern.findall(f"{s.name} {s.description or ''}"))
|
||||
# ── Free-text intent search ────────────────────────────────────
|
||||
return _rank_by_intent(self._search_index, query)
|
||||
|
||||
@ -4,8 +4,9 @@ Builds the ``describe_skill`` tool as a closure over a :class:`SkillCatalog`.
|
||||
The tool returns structured metadata (description, allowed tools, file location)
|
||||
so the LLM can decide whether to ``read_file`` the full SKILL.md.
|
||||
|
||||
Mirrors ``build_tool_search_tool`` from ``tool_search.py``: same query syntax,
|
||||
same ``Command`` + ``ToolMessage`` return shape, same fail-safe degradation.
|
||||
Shares query forms, ``Command`` + ``ToolMessage`` return shape, and fail-safe
|
||||
degradation with ``build_tool_search_tool`` from ``tool_search.py``. Skill
|
||||
queries intentionally rank literal intent terms rather than matching regexes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@ -1,10 +1,11 @@
|
||||
"""Tests for SkillCatalog — deferred skill discovery search engine."""
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from deerflow.skills.catalog import MAX_RESULTS, SkillCatalog
|
||||
from deerflow.skills.catalog import MAX_QUERY_CHARS, MAX_RESULTS, SkillCatalog, _normalize_search_text
|
||||
from deerflow.skills.types import Skill, SkillCategory
|
||||
|
||||
# ── Fixtures ──────────────────────────────────────────────────────────────────
|
||||
@ -102,6 +103,45 @@ def test_select_returns_all_requested(catalog: SkillCatalog, sample_skills: list
|
||||
assert len(result) == len(sample_skills)
|
||||
|
||||
|
||||
def test_long_select_preserves_exact_names_and_catalog_order():
|
||||
skills = tuple(_make_skill(f"skill-number-{i:02d}-with-a-longish-name") for i in range(30))
|
||||
catalog = SkillCatalog(skills)
|
||||
requested = [s.name for s in reversed(skills)] + [skills[0].name, "missing", "SKILL-NUMBER-00-WITH-A-LONGISH-NAME"]
|
||||
query = " select:" + ", ".join(requested) + " "
|
||||
assert len(query) > MAX_QUERY_CHARS
|
||||
|
||||
assert catalog.search(query) == list(skills)
|
||||
|
||||
|
||||
def test_select_does_not_match_a_name_cut_at_search_limit():
|
||||
skills = (_make_skill("data"), _make_skill("data-analysis"))
|
||||
prefix = "select:" + "," * (MAX_QUERY_CHARS - len("select:data"))
|
||||
|
||||
assert SkillCatalog(skills).search(f"{prefix}data-analysis") == [skills[1]]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("prefix", ["", "+report "])
|
||||
def test_ranked_search_still_ignores_terms_beyond_character_limit(prefix: str):
|
||||
catalog = SkillCatalog((_make_skill("report", "needle"),))
|
||||
query = prefix + "unknown " * MAX_QUERY_CHARS + "needle"
|
||||
|
||||
assert catalog.search(query) == catalog.search(query[:MAX_QUERY_CHARS])
|
||||
|
||||
|
||||
def test_search_normalizes_catalog_metadata_once(catalog: SkillCatalog):
|
||||
with patch("deerflow.skills.catalog._normalize_search_text", wraps=_normalize_search_text) as normalize:
|
||||
assert catalog.search("select:data-analysis")
|
||||
normalize.assert_not_called()
|
||||
assert catalog.search("data")
|
||||
assert catalog.search("+data Python")
|
||||
assert catalog.search("+data")
|
||||
assert catalog.search("research")
|
||||
normalized_inputs = [call.args[0] for call in normalize.call_args_list]
|
||||
for skill in catalog.skills:
|
||||
assert normalized_inputs.count(skill.name) == 1
|
||||
assert normalized_inputs.count(skill.description) == 1
|
||||
|
||||
|
||||
# ── Required-prefix search (+) ────────────────────────────────────────────────
|
||||
|
||||
|
||||
@ -127,7 +167,13 @@ def test_required_prefix_no_match(catalog: SkillCatalog):
|
||||
assert result == []
|
||||
|
||||
|
||||
# ── Free-text regex search ────────────────────────────────────────────────────
|
||||
def test_required_prefix_keeps_single_letter_semantics():
|
||||
catalog = SkillCatalog((_make_skill("r-analysis"), _make_skill("python-analysis")))
|
||||
|
||||
assert [skill.name for skill in catalog.search("+r")] == ["r-analysis"]
|
||||
|
||||
|
||||
# ── Free-text intent search ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_keyword_matches_name(catalog: SkillCatalog):
|
||||
@ -150,19 +196,92 @@ def test_name_match_scores_higher_than_description(catalog: SkillCatalog):
|
||||
assert result[0].name == "chart-visualization"
|
||||
|
||||
|
||||
def test_regex_case_insensitive(catalog: SkillCatalog):
|
||||
def test_search_is_case_insensitive(catalog: SkillCatalog):
|
||||
result_lower = catalog.search("data")
|
||||
result_upper = catalog.search("DATA")
|
||||
assert {s.name for s in result_lower} == {s.name for s in result_upper}
|
||||
|
||||
|
||||
def test_invalid_regex_falls_back_to_literal(catalog: SkillCatalog):
|
||||
"""Unbalanced paren should degrade to literal match, not raise."""
|
||||
def test_regex_punctuation_is_treated_as_literal_input(catalog: SkillCatalog):
|
||||
"""Model-generated punctuation must not be compiled or raise."""
|
||||
result = catalog.search("(invalid")
|
||||
# Should not raise; may or may not match anything
|
||||
assert isinstance(result, list)
|
||||
|
||||
|
||||
def test_multi_term_query_matches_across_name_separators(catalog: SkillCatalog):
|
||||
result = catalog.search("chart visualization")
|
||||
|
||||
assert result[0].name == "chart-visualization"
|
||||
|
||||
|
||||
def test_multi_term_query_matches_noncontiguous_description(catalog: SkillCatalog):
|
||||
result = catalog.search("analyze Python")
|
||||
|
||||
assert result[0].name == "data-analysis"
|
||||
|
||||
|
||||
def test_more_intent_terms_outrank_incidental_match():
|
||||
catalog = SkillCatalog(
|
||||
(
|
||||
_make_skill("python-style", "Format Python source code"),
|
||||
_make_skill("spreadsheet-analysis", "Analyze spreadsheet data with Python"),
|
||||
)
|
||||
)
|
||||
|
||||
result = catalog.search("analyze spreadsheet python")
|
||||
|
||||
assert [skill.name for skill in result] == ["spreadsheet-analysis", "python-style"]
|
||||
|
||||
|
||||
def test_name_match_outranks_description_only_at_equal_coverage():
|
||||
catalog = SkillCatalog(
|
||||
(
|
||||
_make_skill("scripting", "Automate work with Python"),
|
||||
_make_skill("python-workflow", "Automate developer work"),
|
||||
)
|
||||
)
|
||||
|
||||
result = catalog.search("python")
|
||||
|
||||
assert [skill.name for skill in result] == ["python-workflow", "scripting"]
|
||||
|
||||
|
||||
def test_score_ties_preserve_catalog_order():
|
||||
catalog = SkillCatalog(
|
||||
(
|
||||
_make_skill("first", "Generate reports"),
|
||||
_make_skill("second", "Generate reports"),
|
||||
)
|
||||
)
|
||||
|
||||
assert [skill.name for skill in catalog.search("reports")] == ["first", "second"]
|
||||
|
||||
|
||||
def test_unicode_compatibility_normalization(catalog: SkillCatalog):
|
||||
result = catalog.search("ＤＡＴＡ")
|
||||
|
||||
assert result[0].name == "data-analysis"
|
||||
|
||||
|
||||
def test_single_letter_language_term_remains_searchable():
|
||||
catalog = SkillCatalog((_make_skill("cpp-analysis", "Analyze C++ code"),))
|
||||
|
||||
assert catalog.search("C++")[0].name == "cpp-analysis"
|
||||
|
||||
|
||||
def test_cjk_terms_rank_by_coverage():
|
||||
catalog = SkillCatalog(
|
||||
(
|
||||
_make_skill("generic-chart", "生成可视化图表"),
|
||||
_make_skill("data-visualization", "执行数据分析和可视化"),
|
||||
)
|
||||
)
|
||||
|
||||
result = catalog.search("数据 可视化")
|
||||
|
||||
assert [skill.name for skill in result] == ["data-visualization", "generic-chart"]
|
||||
|
||||
|
||||
def test_empty_query(catalog: SkillCatalog):
|
||||
result = catalog.search("")
|
||||
assert result == []
|
||||
@ -173,6 +292,16 @@ def test_whitespace_only_query(catalog: SkillCatalog):
|
||||
assert result == []
|
||||
|
||||
|
||||
def test_punctuation_only_query(catalog: SkillCatalog):
|
||||
assert catalog.search("((...---___") == []
|
||||
|
||||
|
||||
def test_long_query_is_bounded_and_does_not_raise(catalog: SkillCatalog):
|
||||
result = catalog.search("data " * 100_000)
|
||||
|
||||
assert result[0].name == "data-analysis"
|
||||
|
||||
|
||||
def test_max_results_cap(catalog: SkillCatalog):
|
||||
"""Free-text search should cap results at MAX_RESULTS."""
|
||||
# 'generation' matches many descriptions
|
||||
|
||||
@ -264,16 +264,17 @@ def test_describe_tool_keyword_search(catalog: SkillCatalog):
|
||||
assert "deep-research" in messages[0].content
|
||||
|
||||
|
||||
def test_describe_tool_select_uncapped(tmp_path):
|
||||
def test_describe_tool_select_uncapped():
|
||||
"""select: must return ALL requested skills, not capped at MAX_RESULTS."""
|
||||
from deerflow.skills.catalog import MAX_RESULTS
|
||||
from deerflow.skills.catalog import MAX_QUERY_CHARS, MAX_RESULTS
|
||||
|
||||
# Build more skills than MAX_RESULTS so the cap would visibly truncate
|
||||
many_skills = [_make_skill(f"skill-{i:02d}") for i in range(MAX_RESULTS + 2)]
|
||||
many_skills = [_make_skill(f"skill-number-{i:02d}-with-a-longish-name") for i in range(MAX_RESULTS + 10)]
|
||||
big_catalog = SkillCatalog(tuple(many_skills))
|
||||
tool = build_describe_skill_tool(big_catalog)
|
||||
|
||||
names_csv = ",".join(s.name for s in many_skills)
|
||||
assert len(names_csv) > MAX_QUERY_CHARS
|
||||
result = tool.invoke(
|
||||
{"args": {"name": f"select:{names_csv}"}, "name": "describe_skill", "type": "tool_call", "id": "test_select_uncapped"},
|
||||
)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user