github/deer-flow
1
0
Fork 0
mirror of https://github.com/bytedance/deer-flow.git synced 2026-11-12 14:00:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(skills): give a bracketed literal in the userinfo slot no host on every Python (#8017)

Browse Source 
* fix(skills): give a bracketed literal in the userinfo slot no host on every Python

`_http_host` answered from `urlsplit(url).hostname`, which decides the
ambiguous authority `http://[::1]@Example.COM/api` differently across
interpreters: 3.12 reads the bracketed part as userinfo and returns the host
`example.com`, 3.13 rejects the authority and `urlsplit` raises, so the helper
returns None. The same URL got two different answers depending on the runtime.

The answer is not cosmetic: `_is_outbound_url` compares it against
`_LOCAL_HTTP_HOSTS`, so whether a URL counts as local or outbound moved with the
interpreter. The test table added with the surrounding refactor (#7950) already
states the intended answer, None, which is why `tests/test_skillscan_native.py`
fails on 3.12 and passes on 3.13.

The bracketed-literal case is now rejected before `hostname` is read, so the
answer no longer depends on that URL-parsing change, and the shape keeps the
treatment the other malformed authorities get: no host, therefore outbound.

The test table also gains a bracketed literal with an IPv6 host and a port
(`http://[2001:DB8::1]@example.com:8443/api`), so a fix that only special-cased
the exact URL from the failure cannot pass.

On 3.12: `tests/test_skillscan_native.py` goes from 1 failed / 387 passed to 388
passed, and the skillscan suite (834 tests) passes. Reverting only the
orchestrator change puts both bracketed cases back to failing.

* fix(skills): reject raw brackets in URL userinfo

---------

Co-authored-by: lxcxjxhx <lxcxjxhx@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
Hyacinth-of-Security 2026-11-09 16:49:37 +08:00 committed by GitHub
parent 7bbccda575
commit c738c5ef8b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 15 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@ -2165,7 +2165,8 @@ Windows scripts (`.bat`, `.cmd`, `.ps1`, `.psm1`, `.js`, `.jse`, `.vbs`, `.vbe`,
SkillScan warns about remote downloads piped into common shells, including `env` launchers, sudo, interpreter paths, and shell line continuations. It distinguishes environment dumps from `env` commands, including assignments and GNU split-string escapes. Shell parsing and split expansion are bounded; reaching a limit preserves critical findings already detected. Pipes to non-shell tools such as `jq` and `tee` do not trigger this warning.
SkillScan treats HTTP/HTTPS scheme spellings and HTTP hostnames case-insensitively and recognizes bracketed IPv6
loopback (`[::1]`) URLs as local. External IPv6 endpoints still trigger network findings,
loopback (`[::1]`) hosts as local. Raw brackets in the URL's userinfo make it count as outbound for
exfiltration checks. External IPv6 endpoints still trigger network findings,
and cloud-metadata hostname detection is case-insensitive.
SkillScan recognizes unquoted integer model-usage counters such as `total_tokens` and `input_tokens` in skill examples as measurements. Benchmark token statistics and signed deltas are also recognized; credential fields and other string values remain subject to secret detection.

2
backend/packages/harness/deerflow/skills/AGENTS.md
View File

File diff suppressed because one or more lines are too long

10
backend/packages/harness/deerflow/skills/skillscan/orchestrator.py
View File

@ -1446,11 +1446,15 @@ def _http_host(url: str) -> str | None:
if not url[:8].lower().startswith(("http://", "https://")):
return None
try:
# Parse the authority so IPv6 brackets and userinfo cannot be mistaken
# for the host. Malformed URLs remain outbound in _is_outbound_url.
host = urlsplit(url).hostname
parts = urlsplit(url)
except ValueError:
return None
# Python versions differ on which brackets in userinfo they accept.
# Treat raw brackets there as malformed so the URL stays outbound.
userinfo, at_sign, _hostport = parts.netloc.rpartition("@")
if at_sign and ("[" in userinfo or "]" in userinfo):
return None
host = parts.hostname
return host.lower() if host else None

5
backend/tests/test_skillscan_native.py
View File

@ -2214,6 +2214,11 @@ def test_shell_curl_without_pipe_finishes_on_repeated_backslash_text(tmp_path: P
("http://[2001:DB8::1]:8080/api", "2001:db8::1"),
("http://user@[::1]:8080/api", "::1"),
("http://[::1]@Example.COM/api", None),
("http://[2001:DB8::1]@example.com:8443/api", None),
("http://[::1]@[::1]:8443/api", None),
("http://[@[::1]", None),
("http://x[@[::1]", None),
("http://]@[::1]", None),
("http://localhost@Example.COM/api", "example.com"),
("http://[::1/api", None),
("ftp://LOCALHOST/api", None),