mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 05:06:59 +00:00
fix(docker): strip the IPv6 listen in the production nginx launcher too (#5900)
* fix(docker): strip the IPv6 listen in the production nginx launcher too docker/nginx/nginx.conf listens on 2026 and [::]:2026. On a host booted with ipv6.disable=1 (or a kernel without IPv6) nginx refuses to start — socket() [::]:2026 failed (97: Address family not supported by protocol) — so the container restart-loops under `restart: unless-stopped` and `make up` waits on a readiness probe that never turns healthy. #2027 added a launcher guard that deletes the IPv6 listen when /proc/net/if_inet6 is absent, but only to docker-compose-dev.yaml; the Helm chart mirrors it; docker-compose.yaml — the file behind `make up` and scripts/deploy.sh — still ran the bare copy-and-start command. Give the production nginx service the same launcher as the dev file: `set -e`, copy the template, guard, and `exec nginx` so nginx is PID 1 and receives docker stop's signal directly. The new test pins the guard in all three launchers (prod, dev, Helm), requires the three sed patterns to be identical so they cannot drift again, checks the guard edits the file nginx actually loads, and applies the pattern to the shipped compose and Helm configs to confirm it removes only the IPv6 listen. Verified on nginx:alpine: BusyBox sed strips exactly that line and `nginx -t` accepts the result; the rendered production launcher runs unchanged when IPv6 is present. * docs(changelog): reference #5900 in the nginx IPv6 guard entries
This commit is contained in:
parent
bf9a019f28
commit
1dfd307c92
@ -323,6 +323,14 @@ This release closes that milestone with **181 merged pull requests**.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **docker:** The production stack (`make up` / `scripts/deploy.sh`) now starts
|
||||
on hosts with IPv6 disabled. `docker/nginx/nginx.conf` also listens on
|
||||
`[::]:2026`; on a kernel booted with `ipv6.disable=1` that listen makes nginx
|
||||
exit at startup, so the container restart-looped and `make up` never became
|
||||
healthy. The dev compose file has stripped the IPv6 listen when
|
||||
`/proc/net/if_inet6` is absent since #2027, and the Helm chart mirrors it;
|
||||
the production compose file was the one launcher still without the guard.
|
||||
It now uses the same launcher, and starts nginx with `exec` so it is PID 1. ([#5900])
|
||||
- **skills:** `skill_manage(action="remove_file")` and `write_file` now work on
|
||||
binary support files, and reject directories cleanly. A `.skill` archive may
|
||||
carry `assets/logo.png` (the installer only rejects *executable* binaries),
|
||||
@ -6120,4 +6128,5 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
|
||||
[#5879]: https://github.com/bytedance/deer-flow/pull/5879
|
||||
[#5881]: https://github.com/bytedance/deer-flow/pull/5881
|
||||
[#5893]: https://github.com/bytedance/deer-flow/pull/5893
|
||||
[#5900]: https://github.com/bytedance/deer-flow/pull/5900
|
||||
|
||||
|
||||
@ -328,6 +328,12 @@
|
||||
|
||||
### 修复
|
||||
|
||||
- **Docker:** 生产栈(`make up` / `scripts/deploy.sh`)现在可以在禁用 IPv6 的主机上启动。
|
||||
`docker/nginx/nginx.conf` 同时监听 `[::]:2026`;在以 `ipv6.disable=1` 启动的内核上,这条监听会让
|
||||
nginx 在启动时退出,容器因此反复重启,`make up` 永远无法就绪。开发用 compose 文件自 #2027 起
|
||||
会在 `/proc/net/if_inet6` 不存在时去掉 IPv6 监听,Helm chart 也做了同样处理;生产用 compose
|
||||
文件是唯一还没有这层保护的启动器。现在它使用相同的启动脚本,并以 `exec` 启动 nginx 使其成为
|
||||
PID 1。([#5900])
|
||||
- **技能:** `skill_manage(action="remove_file")` 与 `write_file` 现在可以处理二进制支持文件,
|
||||
并会干净地拒绝目录。`.skill` 压缩包可以包含 `assets/logo.png`(安装器只拒绝*可执行*二进制),
|
||||
但这两个操作在改动文件之前会先把原有内容按 UTF-8 文本读出(仅用于历史记录),于是二进制
|
||||
@ -5205,3 +5211,4 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写,核心包含子
|
||||
[#5879]: https://github.com/bytedance/deer-flow/pull/5879
|
||||
[#5881]: https://github.com/bytedance/deer-flow/pull/5881
|
||||
[#5893]: https://github.com/bytedance/deer-flow/pull/5893
|
||||
[#5900]: https://github.com/bytedance/deer-flow/pull/5900
|
||||
|
||||
115
backend/tests/test_compose_nginx_ipv6_guard.py
Normal file
115
backend/tests/test_compose_nginx_ipv6_guard.py
Normal file
@ -0,0 +1,115 @@
|
||||
"""Every nginx launcher must survive a host with IPv6 disabled.
|
||||
|
||||
``docker/nginx/nginx.conf`` listens on both ``2026`` and ``[::]:2026``. On a
|
||||
host booted with ``ipv6.disable=1`` (or a kernel without IPv6) nginx refuses to
|
||||
start — ``socket() [::]:2026 failed (97: Address family not supported by
|
||||
protocol)`` — and a ``restart: unless-stopped`` container loops until the
|
||||
``--wait`` readiness check gives up. #2027 added a launcher guard that strips
|
||||
the IPv6 ``listen`` when ``/proc/net/if_inet6`` is absent, but only to the dev
|
||||
compose file; the Helm chart mirrors it; the production compose file used by
|
||||
``make up`` / ``scripts/deploy.sh`` did not. These tests pin the guard in every
|
||||
launcher, keep the three copies identical, and check the pattern really
|
||||
removes the IPv6 line from the config it is applied to.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
COMPOSE_PATHS = {
|
||||
"prod": REPO_ROOT / "docker" / "docker-compose.yaml",
|
||||
"dev": REPO_ROOT / "docker" / "docker-compose-dev.yaml",
|
||||
}
|
||||
HELM_NGINX_DEPLOYMENT = REPO_ROOT / "deploy" / "helm" / "deer-flow" / "templates" / "nginx-deployment.yaml"
|
||||
NGINX_CONFS = {
|
||||
"compose": REPO_ROOT / "docker" / "nginx" / "nginx.conf",
|
||||
"helm": REPO_ROOT / "deploy" / "helm" / "deer-flow" / "templates" / "configmap-nginx.yaml",
|
||||
}
|
||||
|
||||
GUARD_RE = re.compile(r"test -e /proc/net/if_inet6 \|\| sed -i '(?P<pattern>[^']+)' (?P<target>\S+)")
|
||||
|
||||
|
||||
def _compose_nginx_script(compose_path: Path) -> str:
|
||||
compose = yaml.safe_load(compose_path.read_text(encoding="utf-8"))
|
||||
command = compose["services"]["nginx"]["command"]
|
||||
if isinstance(command, list):
|
||||
assert command[:2] == ["sh", "-c"], f"{compose_path.name}: nginx command must be `sh -c <script>`; got {command[:2]!r}"
|
||||
return command[2]
|
||||
return str(command)
|
||||
|
||||
|
||||
def _helm_nginx_script() -> str:
|
||||
# The template carries Helm expressions elsewhere; the nginx container's
|
||||
# ``args`` block is plain YAML, so pull the folded scalar out textually.
|
||||
text = HELM_NGINX_DEPLOYMENT.read_text(encoding="utf-8")
|
||||
match = re.search(r"args:\n\s+- >-\n((?:\s{14}.*\n)+)", text)
|
||||
assert match, "could not find the nginx args block in the Helm deployment"
|
||||
return " ".join(line.strip() for line in match.group(1).splitlines())
|
||||
|
||||
|
||||
LAUNCHERS = {
|
||||
"prod": lambda: _compose_nginx_script(COMPOSE_PATHS["prod"]),
|
||||
"dev": lambda: _compose_nginx_script(COMPOSE_PATHS["dev"]),
|
||||
"helm": _helm_nginx_script,
|
||||
}
|
||||
|
||||
|
||||
def _guard(script: str) -> re.Match[str]:
|
||||
match = GUARD_RE.search(script)
|
||||
assert match, f"nginx launcher has no `test -e /proc/net/if_inet6 || sed -i ...` guard:\n{script}"
|
||||
return match
|
||||
|
||||
|
||||
@pytest.mark.parametrize("launcher", sorted(LAUNCHERS))
|
||||
def test_every_nginx_launcher_strips_the_ipv6_listen_when_ipv6_is_unavailable(launcher: str):
|
||||
script = LAUNCHERS[launcher]()
|
||||
|
||||
guard = _guard(script)
|
||||
# The guard must run on the copied config, between the copy and the start.
|
||||
assert script.index("cp /etc/nginx/nginx.conf.template") < guard.start() < script.index("nginx -")
|
||||
start = script[guard.end() :]
|
||||
explicit = re.search(r"nginx (?:.*\s)?-c (\S+)", start)
|
||||
loaded = explicit.group(1) if explicit else "/etc/nginx/nginx.conf" # nginx's compiled-in default
|
||||
assert guard.group("target") == loaded, f"the guard edits {guard.group('target')} but nginx loads {loaded}"
|
||||
|
||||
|
||||
def test_all_nginx_launchers_use_one_identical_guard_pattern():
|
||||
"""Three copies of one rule: pin them to the same sed expression so they cannot drift."""
|
||||
patterns = {launcher: _guard(script())["pattern"] for launcher, script in LAUNCHERS.items()}
|
||||
|
||||
assert len(set(patterns.values())) == 1, f"nginx launchers disagree on the IPv6 guard pattern: {patterns}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("variant", sorted(COMPOSE_PATHS))
|
||||
def test_compose_nginx_execs_nginx_as_pid_1(variant: str):
|
||||
"""`exec` hands PID 1 to nginx so `docker stop` delivers SIGTERM/SIGQUIT to it, not to `sh`."""
|
||||
script = _compose_nginx_script(COMPOSE_PATHS[variant])
|
||||
|
||||
assert re.search(r"(^|\n|&&\s*)exec nginx -g 'daemon off;'", script), f"{variant} compose must start nginx with `exec`:\n{script}"
|
||||
assert "set -e" in script, f"{variant} compose must fail the container when the copy or the guard fails:\n{script}"
|
||||
|
||||
|
||||
def _apply_sed_delete(bre_pattern: str, text: str) -> str:
|
||||
"""Apply `sed -i '/<pattern>/d'` with Python's regex engine (BusyBox BRE → Python re)."""
|
||||
assert bre_pattern.startswith("/") and bre_pattern.endswith("/d"), bre_pattern
|
||||
body = bre_pattern[1:-2].replace("[[:space:]]", r"\s").replace(r"\+", "+")
|
||||
return "".join(line for line in text.splitlines(keepends=True) if not re.search(body, line))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("conf", sorted(NGINX_CONFS))
|
||||
def test_guard_pattern_removes_only_the_ipv6_listen_from_the_shipped_config(conf: str):
|
||||
"""The pattern is only useful if it matches the exact `listen [::]:2026 ...;` line we ship."""
|
||||
pattern = _guard(LAUNCHERS["dev"]())["pattern"]
|
||||
original = NGINX_CONFS[conf].read_text(encoding="utf-8")
|
||||
assert re.search(r"^\s*listen \[::\]:2026 default_server;\s*$", original, re.M), f"{conf} nginx config no longer has the IPv6 listen this guard targets"
|
||||
|
||||
stripped = _apply_sed_delete(pattern, original)
|
||||
|
||||
assert not re.search(r"^\s*listen\s+\[::\]", stripped, re.M), f"{conf}: IPv6 listen survived the guard"
|
||||
assert re.search(r"^\s*listen 2026 default_server;\s*$", stripped, re.M), f"{conf}: the IPv4 listen must survive"
|
||||
assert len(original.splitlines()) - len(stripped.splitlines()) == 1
|
||||
@ -54,9 +54,20 @@ services:
|
||||
- "${BIND_HOST:-127.0.0.1}:${PORT:-2026}:2026"
|
||||
volumes:
|
||||
- ./nginx/nginx.conf:/etc/nginx/nginx.conf.template:ro
|
||||
command: >
|
||||
sh -c "cp /etc/nginx/nginx.conf.template /etc/nginx/nginx.conf
|
||||
&& nginx -g 'daemon off;'"
|
||||
# Same launcher as the dev compose file and the Helm chart: nginx.conf
|
||||
# listens on [::]:2026 as well, and on a host with IPv6 disabled that
|
||||
# listen makes nginx exit at startup (socket() ... 97: Address family not
|
||||
# supported by protocol), so the container restart-loops and `make up`
|
||||
# never becomes healthy. Strip the IPv6 listen when the kernel has no
|
||||
# IPv6, and exec nginx so it is PID 1 for signal handling.
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
cp /etc/nginx/nginx.conf.template /etc/nginx/nginx.conf
|
||||
test -e /proc/net/if_inet6 || sed -i '/^[[:space:]]*listen[[:space:]]\+\[::\]:2026[[:space:]]/d' /etc/nginx/nginx.conf
|
||||
exec nginx -g 'daemon off;'
|
||||
depends_on:
|
||||
frontend:
|
||||
condition: service_started
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user