Compare commits

...

19 Commits
v0.8.5 ... main

Author SHA1 Message Date
viccy
a9e17d0e36
Merge pull request #275 from cindyxu1030/feat/sonilo-sfx
feat(sfx): 新增 Sonilo AI 音效模式(可选,默认关闭)
2026-07-23 11:57:43 +08:00
Cindy Xu
14828256de feat(sfx): 新增 Sonilo AI 音效模式(可选,默认关闭)
#270 配乐模式基础上新增可选的 "AI 音效(Sonilo)":将合并后的成片
上传到 Sonilo API(POST /v1/video-to-sfx,异步任务管线),根据画面内容
生成音效,再用 ffmpeg 混在成片现有音轨之下(视频流直接复制,不重编码
画面)。解说配音在后续合成步骤中单独混入,音量策略不受影响。生成的
音效为免版税素材。

- 复用 #270 的 sonilo_api_key 配置与 app/services/sonilo.py 模块,
  新增音效任务管线(提交 / 轮询 / 预签名 URL 下载)
- 上传前 ffprobe 本地校验时长(音效接口上限 3 分钟),避免白传计费
- 轮询是免费幂等 GET:网络抖动与 5xx 在截止时间内重试,不报废已计费任务
- 任何失败(超时、HTTP 错误、任务失败、混音失败)只记录日志并沿用
  原视频,绝不中断成片任务
- WebUI 音频设置新增独立开关面板(默认关闭),选项处提示视频将上传至
  Sonilo API
- 现有 BGM / 合成逻辑不做任何改动,未勾选时行为与 main 完全一致
2026-07-22 13:17:14 -07:00
linyq-laien
d46d2ca2a5 docs: restore original README content 2026-07-22 11:57:46 +08:00
linyq-laien
5ac4488fa3 docs: optimize AI narration search visibility 2026-07-22 11:23:59 +08:00
linyq-laien
7c2bfbe233 docs: link NarratoAI project distinction guide 2026-07-21 20:32:33 +08:00
linyq-laien
0a5dcf5f21 功能: 新增字幕任务专用高效模型支持
新增高推理模型和高效模型的中英文国际化配置字符串
新增高效文本模型配置项,支持空值时自动回退到高推理模型
添加resolve_text_model_name工具函数用于根据偏好解析对应文本模型
更新字幕校对和翻译服务,默认使用高效模型处理批量字幕任务
优化OpenAI兼容提供商的模型参数处理逻辑,支持通过参数覆盖模型名称
更新WebUI基础设置页面,新增高效模型配置项并支持测试两个模型的连接
同步更新示例配置文件、默认配置项与相关测试用例
2026-07-19 14:53:06 +08:00
linyq-laien
022b8bbea3 feat(webui, llm, i18n): 添加可自定义的解说文案字数配置
在WebUI脚本设置页面新增字数输入控件,支持100-5000区间的目标字数设置
更新LLM提示词模板,使用用户指定字数作为目标(±10%浮动)替代原固定范围
补充中英文国际化翻译文案
升级项目版本至0.8.7
更新相关服务函数与单元测试适配新参数
2026-07-18 04:26:44 +08:00
linyq-laien
7639c9d289 feat(audio): 新增voxcpm-0.5b和voxcpm-2b本地语音合成支持
新增完整的voxcpm系列本地语音合成功能,包括:
- 新增配置常量、加载逻辑与示例配置项
- 集成webui界面:引擎选型、参数配置与语音预览
- 实现核心调用逻辑,包含API地址标准化、重试机制与文件处理
- 新增两套单元测试覆盖核心流程
- 在字幕转录设置页面补充转录工具包的下载提示文本
2026-07-16 19:11:51 +08:00
viccy
15af56cbd3 docs: 更新文档 2026-07-16 11:35:31 +08:00
viccy
0c9579c56a docs: update README for v0.8.5 and v0.8.6 2026-07-16 11:33:08 +08:00
viccy
c09566ae16 更新版本号 2026-07-16 11:25:57 +08:00
viccy
445feb14ae Merge remote-tracking branch 'origin/main' 2026-07-16 11:24:39 +08:00
viccy
d0216fcb04
Merge pull request #270 from cindyxu1030/feat/sonilo-bgm
feat(bgm): 新增 Sonilo AI 配乐模式(可选,默认关闭)
2026-07-16 10:26:42 +08:00
Cindy Xu
d29874a163 feat(bgm): 新增 Sonilo AI 配乐模式(可选,默认关闭)
背景音乐来源新增 "AI 生成配乐(Sonilo)":将合成完成的视频(未加 BGM)
上传到 Sonilo API,根据画面内容与剪辑节奏生成配乐,音频文件交还给现有
合成流程混音,解说配音音量策略保持不变。

- 独立模块 app/services/sonilo.py,仅在配置 sonilo_api_key 后可用
- 上传前 ffprobe 本地校验时长(接口上限 6 分钟),避免白传计费
- 任何失败只记录日志并回退到随机背景音乐,绝不中断成片任务
- WebUI 选项处提示:视频将上传至 Sonilo API,生成的音乐已获授权、
  可商用(以条款为准)
- 现有 resource/upload/none 的 BGM 逻辑不做任何改动

Closes #269
2026-07-13 21:00:03 -07:00
viccy
b56222725d feat: add IndexTTS 1.5 macOS engine 2026-07-13 06:19:56 +08:00
viccy
f80ae97679 docs: add OmniVoice download links 2026-07-13 05:23:31 +08:00
viccy
79c89e64a3 fix: format IndexTTS download links 2026-07-13 05:15:27 +08:00
viccy
8b3812fbea docs: update IndexTTS download links 2026-07-13 04:59:12 +08:00
viccy
39eae0d511 fix: adapt IndexTTS-2 MLX Pack API 2026-07-13 04:41:09 +08:00
37 changed files with 3185 additions and 267 deletions

2
.gitignore vendored
View File

@ -54,6 +54,8 @@ tests/*
!tests/test_generate_narration_script_documentary_unittest.py
!tests/test_generate_script_docu_unittest.py
!tests/test_streamlit_widget_session_state.py
!tests/test_sonilo_bgm_unittest.py
!tests/test_sonilo_sfx_unittest.py
docs/reddit-community
docs/wechat-0.8

View File

@ -3,7 +3,7 @@
<h3 align="center">一站式 AI 影视解说+自动化剪辑工具🎬🎞️ </h3>
<p align="center">
📖 <a href="README-en.md">English</a> | 简体中文 | <a href="https://www.narratoai.co">☁️ <b>云端版入口 (NarratoAI.cn)</b></a>
📖 <a href="README-en.md">English</a> | 简体中文 | <a href="https://www.narratoai.co">☁️ <b>云端版入口 (NarratoAI.co)</b></a>
</p>
<br>
@ -29,6 +29,8 @@ NarratoAI 是一款自动化影视解说工具,基于 LLM 实现文案撰写
</div>
## 最新资讯
- 2026.07.16 发布新版本 0.8.6,新增可选的 **Sonilo AI 配乐**,支持 Apple Silicon 的 **IndexTTS-1.5 MLX**,更新 IndexTTS-2 MLX API 适配,并补充 OmniVoice 部署包下载入口
- 2026.07.13 发布新版本 0.8.5,完善 `uv` + Python 3.12 本地运行流程,新增一键清理系统缓存和自定义 OpenAI 兼容 Base URL 的格式校验与 API Key 风险提示,并优化短脚本解析、字幕预览和音频设置的跨平台兼容性
- 2026.07.02 发布新版本 0.8.4,升级豆包语音 TTS 新版 API Key 配置并保留旧版凭据兼容
- 2026.06.10 发布新版本 0.8.1**大版本更新**,优化多个核心流程
- 2026.04.27 发布新版本 0.7.9,新增 **Fun-ASR一键转录字幕**
@ -46,6 +48,17 @@ NarratoAI 是一款自动化影视解说工具,基于 LLM 实现文案撰写
- 2024.11.10 发布官方文档,详情参见 [官方文档](https://p9mf6rjv3c.feishu.cn/wiki/SP8swLLZki5WRWkhuFvc2CyInDg)
- 2024.11.10 发布新版本 v0.3.5;优化视频剪辑流程,
### v0.8.6 更新亮点
| 功能 | 说明 |
| --- | --- |
| Sonilo AI 配乐 | 在 WebUI 的背景音乐来源中选择“AI 生成配乐Sonilo即可根据画面内容和剪辑节奏生成配乐。该功能默认关闭需自行配置 [Sonilo](https://sonilo.com) API Key当前支持最长 6 分钟的视频,生成失败时会自动回退到随机背景音乐,不中断成片任务。 |
| IndexTTS-1.5 macOS | 新增面向 Apple Silicon 的 MLX 本地语音克隆引擎,可上传或从资源目录选择参考音频。[下载部署包](https://cutagent.online/resources/indextts15-mlx-macos) |
| IndexTTS-2 MLX | 更新 MLX Pack API 适配,支持参考音频、情感控制、随机种子和更完整的生成参数。[macOS 部署包](https://cutagent.online/resources/indextts2-full-macos)Windows 版本待更新) |
| OmniVoice | 补充 OmniVoice-Pack 部署包下载入口;该引擎支持自动音色、指令音色和参考音频克隆。[macOS 部署包](https://cutagent.online/resources/omnivoice-macos) / [Windows 部署包](https://cutagent.online/resources/omnivoice-windows) |
> 使用 Sonilo AI 配乐时,合成完成且尚未添加背景音乐的视频会上传至 Sonilo API。请在启用前确认视频内容符合相关服务条款与隐私要求生成音乐的授权和商用范围以 Sonilo 最新条款为准。
## 重磅福利 🎉
> 即日起全面支持硅基流动注册即享2000万免费Token价值16元平台配额剪辑10分钟视频仅需0.1元!
@ -64,6 +77,8 @@ NarratoAI 是一款自动化影视解说工具,基于 LLM 实现文案撰写
## ⚠️谨防被骗 📢
> 🔎 名称辨析:[**NarratoAI 与 NarratorAI 项目关系、开源范围与使用方式说明**](https://github.com/linyqh/NarratoAI/wiki/NarratoAI-%E4%B8%8E-NarratorAI-%E9%A1%B9%E7%9B%AE%E5%85%B3%E7%B3%BB%E5%92%8C%E5%8C%BA%E5%88%AB)
_**1. NarratoAI 是一款完全免费的软件,近期在社交媒体(抖音,B站等)上发现,有人将 NarratoAI 改名后售卖,下面是部分截图,请大家务必提高警惕,切勿上当受骗**_
---
@ -92,6 +107,7 @@ _**1. NarratoAI 是一款完全免费的软件,近期在社交媒体(抖音,B
- [ ] 主角人脸匹配
- [ ] 支持根据口播,文案,视频素材自动匹配
- [X] 支持更多 TTS 引擎
- [X] 支持可选的 AI 自动配乐
- [ ] ...
## 快速启动 🚀

View File

@ -10,14 +10,33 @@ root_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__fi
config_file = f"{root_dir}/config.toml"
version_file = f"{root_dir}/project_version"
INDEXTTS_ENGINE = "indextts"
INDEXTTS_DISPLAY_NAME = "IndexTTS-1.5"
INDEXTTS_DISPLAY_NAME = "IndexTTS-1.5-windows"
INDEXTTS_MACOS_ENGINE = "indextts_macos"
INDEXTTS_MACOS_DISPLAY_NAME = "IndexTTS-1.5-macOS"
INDEXTTS2_ENGINE = "indextts2"
INDEXTTS2_DISPLAY_NAME = "IndexTTS-2"
OMNIVOICE_ENGINE = "omnivoice"
OMNIVOICE_DISPLAY_NAME = "OmniVoice"
VOXCPM_ENGINE = "voxcpm_05b"
VOXCPM_DISPLAY_NAME = "VoxCPM-0.5B"
VOXCPM2_ENGINE = "voxcpm_2b"
VOXCPM2_DISPLAY_NAME = "VoxCPM-2B"
INDEXTTS_VOICE_PREFIX = f"{INDEXTTS_ENGINE}:"
INDEXTTS_MACOS_VOICE_PREFIX = f"{INDEXTTS_MACOS_ENGINE}:"
INDEXTTS2_VOICE_PREFIX = f"{INDEXTTS2_ENGINE}:"
OMNIVOICE_VOICE_PREFIX = f"{OMNIVOICE_ENGINE}:"
VOXCPM_VOICE_PREFIX = f"{VOXCPM_ENGINE}:"
VOXCPM2_VOICE_PREFIX = f"{VOXCPM2_ENGINE}:"
INDEXTTS2_EMOTION_VECTOR_FIELDS = (
("happy", "vec_happy"),
("angry", "vec_angry"),
("sad", "vec_sad"),
("afraid", "vec_afraid"),
("disgusted", "vec_disgusted"),
("melancholic", "vec_melancholic"),
("surprised", "vec_surprised"),
("calm", "vec_calm"),
)
def normalize_tts_engine_name(tts_engine: str) -> str:
@ -28,6 +47,32 @@ def normalize_indextts_voice_prefix(voice_name: str) -> str:
return voice_name
def get_indextts2_pack_emotion(indextts2_config) -> str:
"""Return the MLX Pack emotion string for current or legacy settings."""
if not isinstance(indextts2_config, dict):
return ""
configured_emotion = str(indextts2_config.get("emotion", "")).strip()
if configured_emotion:
return configured_emotion
emotion_mode = indextts2_config.get("emotion_mode", "speaker")
if emotion_mode == "text":
return str(indextts2_config.get("emotion_text", "")).strip()
if emotion_mode != "vector":
return ""
weights = []
for emotion, field in INDEXTTS2_EMOTION_VECTOR_FIELDS:
try:
weight = float(indextts2_config.get(field, 0.0))
except (TypeError, ValueError):
continue
if weight > 0:
weights.append(f"{emotion}:{weight:g}")
return ",".join(weights)
def _is_legacy_indextts2_config(indextts2_config) -> bool:
if not isinstance(indextts2_config, dict):
return False
@ -133,8 +178,11 @@ def save_config():
_cfg["tts_qwen"] = tts_qwen
_cfg["fun_asr"] = fun_asr
_cfg["indextts"] = indextts
_cfg["indextts_macos"] = indextts_macos
_cfg["indextts2"] = indextts2
_cfg["omnivoice"] = omnivoice
_cfg["voxcpm_05b"] = voxcpm_05b
_cfg["voxcpm_2b"] = voxcpm_2b
_cfg["doubaotts"] = doubaotts
f.write(toml.dumps(_cfg))
@ -151,8 +199,11 @@ frames = _cfg.get("frames", {})
tts_qwen = _cfg.get("tts_qwen", {})
fun_asr = _cfg.get("fun_asr", {})
indextts = _cfg.get("indextts", {})
indextts_macos = _cfg.get("indextts_macos", {})
indextts2 = _cfg.get("indextts2", {})
omnivoice = _cfg.get("omnivoice", {})
voxcpm_05b = _cfg.get("voxcpm_05b", {})
voxcpm_2b = _cfg.get("voxcpm_2b", {})
doubaotts = _cfg.get("doubaotts", {})
hostname = socket.gethostname()

View File

@ -10,6 +10,7 @@ DEFAULT_VISION_OPENAI_MODEL_NAME = "Qwen/Qwen3.5-122B-A10B"
DEFAULT_TEXT_LLM_PROVIDER = DEFAULT_OPENAI_COMPATIBLE_PROVIDER
DEFAULT_TEXT_OPENAI_MODEL_NAME = "Pro/zai-org/GLM-5"
DEFAULT_TEXT_OPENAI_FAST_MODEL_NAME = ""
DEFAULT_LLM_GENERATION_CONFIG = {
"temperature": 1.0,
@ -33,6 +34,7 @@ DEFAULT_LLM_APP_CONFIG = {
"vision_openai_base_url": DEFAULT_OPENAI_COMPATIBLE_BASE_URL,
"text_llm_provider": DEFAULT_TEXT_LLM_PROVIDER,
"text_openai_model_name": DEFAULT_TEXT_OPENAI_MODEL_NAME,
"text_openai_fast_model_name": DEFAULT_TEXT_OPENAI_FAST_MODEL_NAME,
"text_openai_api_key": "",
"text_openai_base_url": DEFAULT_OPENAI_COMPATIBLE_BASE_URL,
"tavily_api_key": "",
@ -69,6 +71,31 @@ def normalize_openai_compatible_model_name(
return normalized
def resolve_text_model_name(
app_config: dict,
provider: str = DEFAULT_OPENAI_COMPATIBLE_PROVIDER,
*,
prefer_fast: bool = False,
) -> str:
"""Resolve the configured reasoning or fast text model with legacy fallback."""
provider = (provider or DEFAULT_OPENAI_COMPATIBLE_PROVIDER).strip().lower()
reasoning_model = normalize_openai_compatible_model_name(
str(app_config.get(f"text_{provider}_model_name") or ""),
provider=provider,
)
if not reasoning_model and provider == DEFAULT_OPENAI_COMPATIBLE_PROVIDER:
reasoning_model = DEFAULT_TEXT_OPENAI_MODEL_NAME
if not prefer_fast:
return reasoning_model
fast_model = normalize_openai_compatible_model_name(
str(app_config.get(f"text_{provider}_fast_model_name") or ""),
provider=provider,
)
return fast_model or reasoning_model
def get_openai_compatible_ui_values(
full_model_name: str,
default_model: str,

View File

@ -1,6 +1,7 @@
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
try:
import tomllib
@ -11,10 +12,35 @@ from app.config import config as cfg
from app.config.defaults import (
get_openai_compatible_ui_values,
normalize_openai_compatible_model_name,
resolve_text_model_name,
)
class ConfigBootstrapDefaultsTests(unittest.TestCase):
def test_save_config_keeps_macos_tts_settings_independent(self):
macos_settings = {
"api_url": "http://127.0.0.1:7866",
"reference_audio": "/tmp/macos-reference.wav",
"speed": 1.1,
}
windows_settings = {
"api_url": "http://127.0.0.1:8081/tts",
"reference_audio": "/tmp/windows-reference.wav",
}
with tempfile.TemporaryDirectory() as tmp_dir:
config_path = Path(tmp_dir) / "config.toml"
with (
patch.object(cfg, "config_file", str(config_path)),
patch.object(cfg, "indextts_macos", dict(macos_settings)),
patch.object(cfg, "indextts", dict(windows_settings)),
):
cfg.save_config()
saved_config = tomllib.loads(config_path.read_text(encoding="utf-8"))
self.assertEqual(macos_settings, saved_config["indextts_macos"])
self.assertEqual(windows_settings, saved_config["indextts"])
def test_load_config_bootstraps_webui_llm_defaults(self):
original_root_dir = cfg.root_dir
original_config_file = cfg.config_file
@ -57,11 +83,13 @@ hide_config = true
self.assertEqual(0.95, config_data["app"]["vision_openai_top_p"])
self.assertEqual("openai", config_data["app"]["text_llm_provider"])
self.assertEqual("Pro/zai-org/GLM-5", config_data["app"]["text_openai_model_name"])
self.assertEqual("", config_data["app"]["text_openai_fast_model_name"])
self.assertEqual("https://api.siliconflow.cn/v1", config_data["app"]["text_openai_base_url"])
self.assertEqual(1.0, config_data["app"]["text_openai_temperature"])
self.assertEqual(0.95, config_data["app"]["text_openai_top_p"])
self.assertEqual("Qwen/Qwen3.5-122B-A10B", saved_config["app"]["vision_openai_model_name"])
self.assertEqual("Pro/zai-org/GLM-5", saved_config["app"]["text_openai_model_name"])
self.assertEqual("", saved_config["app"]["text_openai_fast_model_name"])
self.assertTrue(saved_config["app"]["hide_config"])
def test_legacy_indextts2_config_is_migrated_to_indextts_15(self):
@ -102,6 +130,23 @@ hide_config = true
class OpenAICompatibleModelDefaultsTests(unittest.TestCase):
def test_fast_text_model_falls_back_to_reasoning_model(self):
app_config = {
"text_openai_model_name": "reasoning-model",
"text_openai_fast_model_name": "",
}
self.assertEqual(
"reasoning-model",
resolve_text_model_name(app_config, "openai", prefer_fast=True),
)
app_config["text_openai_fast_model_name"] = "fast-model"
self.assertEqual(
"fast-model",
resolve_text_model_name(app_config, "openai", prefer_fast=True),
)
def test_ui_keeps_full_model_name_and_openai_provider(self):
provider, model_name = get_openai_compatible_ui_values(
"Qwen/Qwen3.5-122B-A10B",

View File

@ -183,6 +183,7 @@ class VideoClipParams(BaseModel):
bgm_name: Optional[str] = Field(default="random", description="背景音乐名称")
bgm_type: Optional[str] = Field(default="random", description="背景音乐类型")
bgm_file: Optional[str] = Field(default="", description="背景音乐文件")
sonilo_sfx_enabled: Optional[bool] = Field(default=False, description="是否启用 Sonilo AI 音效(可选功能,默认关闭)")
subtitle_enabled: bool = True
subtitle_mask_enabled: bool = False

View File

@ -409,6 +409,7 @@ class SubtitleAnalyzer:
temperature: float = 0.7,
narration_language: str = "简体中文(中国)",
drama_genre: str = "逆袭/复仇",
narration_word_count: int = 500,
) -> Dict[str, Any]:
"""生成供用户审核修改的解说正文。"""
try:
@ -420,6 +421,7 @@ class SubtitleAnalyzer:
"plot_analysis": plot_analysis,
"subtitle_content": subtitle_content,
"narration_language": narration_language,
"narration_word_count": int(narration_word_count),
},
)
return self._generate_plain_text(prompt, system_prompt, temperature)
@ -964,6 +966,7 @@ def generate_narration_copy(
narration_language: str = "简体中文(中国)",
drama_genre: str = "逆袭/复仇",
prompt_category: str = "short_drama_narration",
narration_word_count: int = 500,
) -> Dict[str, Any]:
"""生成可供用户审核修改的解说正文。"""
analyzer = SubtitleAnalyzer(
@ -982,6 +985,7 @@ def generate_narration_copy(
temperature=temperature,
narration_language=narration_language,
drama_genre=drama_genre,
narration_word_count=narration_word_count,
)

View File

@ -114,6 +114,10 @@ def _normalize_indextts_reference_audio(params: VideoClipParams) -> None:
tts_config = config.indextts
voice_prefix = config.INDEXTTS_VOICE_PREFIX
display_name = "IndexTTS-1.5"
elif params.tts_engine == config.INDEXTTS_MACOS_ENGINE:
tts_config = config.indextts_macos
voice_prefix = config.INDEXTTS_MACOS_VOICE_PREFIX
display_name = config.INDEXTTS_MACOS_DISPLAY_NAME
elif params.tts_engine == config.INDEXTTS2_ENGINE:
tts_config = config.indextts2
voice_prefix = config.INDEXTTS2_VOICE_PREFIX

View File

@ -164,11 +164,13 @@ class LLMConfigValidator:
config_prefix = f"text_{provider_name}"
api_key = config.app.get(f'{config_prefix}_api_key')
model_name = config.app.get(f'{config_prefix}_model_name')
fast_model_name = config.app.get(f'{config_prefix}_fast_model_name')
base_url = config.app.get(f'{config_prefix}_base_url')
result["config"] = {
"api_key": "***" if api_key else None,
"model_name": model_name,
"fast_model_name": fast_model_name,
"base_url": base_url
}
@ -241,7 +243,8 @@ class LLMConfigValidator:
f"text_{provider}_model_name"
],
"optional_configs": [
f"text_{provider}_base_url"
f"text_{provider}_base_url",
f"text_{provider}_fast_model_name",
],
"example_models": LLMConfigValidator._get_example_models(provider, "text")
}

View File

@ -292,6 +292,7 @@ class SubtitleAnalyzerAdapter:
temperature: float = 0.7,
narration_language: str = "简体中文(中国)",
drama_genre: str = "逆袭/复仇",
narration_word_count: int = 500,
) -> Dict[str, Any]:
"""Generate editable narration copy before timeline matching."""
try:
@ -303,6 +304,7 @@ class SubtitleAnalyzerAdapter:
"plot_analysis": plot_analysis,
"subtitle_content": subtitle_content,
"narration_language": narration_language,
"narration_word_count": int(narration_word_count),
},
)
narration_copy = self._generate_plain_text(prompt, system_prompt, temperature)

View File

@ -258,8 +258,9 @@ class OpenAICompatibleTextProvider(_OpenAICompatibleBase, TextModelProvider):
response_format: Optional[str],
kwargs: Dict[str, Any],
) -> Dict[str, Any]:
model_name = _normalize_model_name(self.model_name)
generation_kwargs = dict(kwargs)
model_override = generation_kwargs.pop("model", None) or generation_kwargs.pop("model_name", None)
model_name = _normalize_model_name(model_override or self.model_name)
temperature_override = generation_kwargs.pop("temperature", None)
if temperature_override is None and temperature != 1.0:
temperature_override = temperature

View File

@ -149,6 +149,20 @@ class OpenAICompatGenerationOptionTests(unittest.TestCase):
self.assertEqual(65536, options["max_tokens"])
self.assertNotIn("extra_body", options)
def test_text_request_can_override_model_for_fast_tasks(self):
provider = OpenAICompatibleTextProvider(api_key="k", model_name="reasoning-model")
options = provider._build_text_completion_kwargs(
messages=[{"role": "user", "content": "hello"}],
temperature=0.2,
max_tokens=None,
response_format=None,
kwargs={"model": "fast-model", "thinking_level": "off"},
)
self.assertEqual("fast-model", options["model"])
self.assertNotIn("extra_body", options)
def test_build_options_uses_per_model_generation_config(self):
provider = OpenAICompatibleTextProvider(api_key="k", model_name="m")
config.app.update(

View File

@ -24,11 +24,14 @@ class SubtitleAnalyzerAdapterPipelineTests(unittest.TestCase):
temperature=0.7,
narration_language="简体中文(中国)",
drama_genre="家庭伦理",
narration_word_count=800,
)
self.assertEqual("success", result["status"])
self.assertIn("反击", result["narration_copy"])
self.assertIn("家庭伦理", call.call_args.kwargs["prompt"])
self.assertIn("800", call.call_args.kwargs["prompt"])
self.assertNotIn("300-650", call.call_args.kwargs["prompt"])
self.assertNotIn("response_format", call.call_args.kwargs)
def test_generate_narration_copy_can_use_film_tv_prompt_category(self):
@ -49,11 +52,14 @@ class SubtitleAnalyzerAdapterPipelineTests(unittest.TestCase):
temperature=0.7,
narration_language="简体中文(中国)",
drama_genre="悬疑/犯罪",
narration_word_count=1200,
)
self.assertEqual("success", result["status"])
self.assertIn("影视解说正文创作任务", call.call_args.kwargs["prompt"])
self.assertIn("用户选择的影视类型", call.call_args.kwargs["prompt"])
self.assertIn("1200", call.call_args.kwargs["prompt"])
self.assertNotIn("350-750", call.call_args.kwargs["prompt"])
self.assertNotIn("短剧解说正文创作任务", call.call_args.kwargs["prompt"])
def test_film_tv_script_prompts_exclude_intro_outro_and_ads(self):

View File

@ -22,7 +22,14 @@ class NarrationCopyPrompt(ParameterizedPrompt):
model_type=ModelType.TEXT,
output_format=OutputFormat.TEXT,
tags=["影视", "解说文案", "电影解说", "剧情承接", "用户审核"],
parameters=["drama_name", "drama_genre", "plot_analysis", "subtitle_content", "narration_language"],
parameters=[
"drama_name",
"drama_genre",
"plot_analysis",
"subtitle_content",
"narration_language",
"narration_word_count",
],
)
super().__init__(metadata, required_parameters=["drama_name", "plot_analysis", "subtitle_content"])
@ -57,6 +64,11 @@ ${narration_language}
${drama_genre}
</drama_genre>
## 用户要求的文案字数
<narration_word_count>
${narration_word_count}
</narration_word_count>
## 类型写作规则
必须按用户选择的影视类型调整表达重点不要自行改判类型
- 剧情/情感突出人物选择关系裂痕命运压力和情绪余波
@ -81,7 +93,7 @@ ${drama_genre}
4. 每句话只表达一个信息点适合后续按句匹配画面
5. 句子尽量短单句优先 15-35 信息复杂时拆成多句
6. 2-3 句要有明确承接让观众知道为什么从上一幕来到下一幕
7. 总长度控制在 350-750 短素材取下限长素材取上限
7. 总长度${narration_word_count} 字为目标允许上下浮动 10%中日韩语言按非空白字符计数其他语言按单词计数不得再套用固定长度区间
8. 不要使用编号项目符号章节标题或括号说明
## 输出要求

View File

@ -22,7 +22,14 @@ class NarrationCopyPrompt(ParameterizedPrompt):
model_type=ModelType.TEXT,
output_format=OutputFormat.TEXT,
tags=["短剧", "解说文案", "爆款开头", "叙事连续性", "用户审核"],
parameters=["drama_name", "drama_genre", "plot_analysis", "subtitle_content", "narration_language"],
parameters=[
"drama_name",
"drama_genre",
"plot_analysis",
"subtitle_content",
"narration_language",
"narration_word_count",
],
)
super().__init__(metadata, required_parameters=["drama_name", "plot_analysis", "subtitle_content"])
@ -57,6 +64,11 @@ ${narration_language}
${drama_genre}
</drama_genre>
## 用户要求的文案字数
<narration_word_count>
${narration_word_count}
</narration_word_count>
## 类型写作规则
必须按用户选择的短剧类型调整表达重点不要自行改判类型
- 霸总/甜宠突出误会身份差暧昧拉扯守护感和情绪反差
@ -81,7 +93,7 @@ ${drama_genre}
4. 每句话只表达一个信息点适合后续按句匹配画面
5. 句子尽量短单句优先 15-35 信息复杂时拆成多句
6. 2-3 句要有明确因果承接让观众知道为什么从上一幕来到下一幕
7. 总长度控制在 300-650 短素材取下限长素材取上限
7. 总长度${narration_word_count} 字为目标允许上下浮动 10%中日韩语言按非空白字符计数其他语言按单词计数不得再套用固定长度区间
8. 不要使用编号项目符号章节标题或括号说明
## 输出要求

626
app/services/sonilo.py Normal file
View File

@ -0,0 +1,626 @@
"""
Sonilo (https://sonilo.com) AI 配乐BGM AI 音效SFX集成
均为可选功能默认关闭
配乐BGM将合成完成的视频未加 BGM上传到 Sonilo API
`POST /v1/video-to-music`根据画面内容与剪辑节奏生成一段背景音乐
作为普通音频文件交还给现有的合成流程混音生成的音乐已获授权
可商用以条款为准
音效SFX将合成完成的视频上传到 Sonilo API
`POST /v1/video-to-sfx`根据画面内容生成贴合画面的音效该接口是
异步任务提交后返回 task_id轮询 `GET /v1/tasks/{task_id}` 直到终态
成功后从预签名 URL 下载音效音频再用 ffmpeg 混在成片现有音轨之下
解说配音在后续合成步骤中单独混入音量策略不受影响生成的音效为
免版税素材
设计约束完全不影响现有合成逻辑
* 默认关闭配乐仅当用户在 WebUI 中把背景音乐来源切换为 Sonilo 时启用
音效仅当用户勾选 "AI 音效Sonilo" 时启用两者都要求配置了
Sonilo API Keyconfig.toml `sonilo_api_key`或环境变量
`SONILO_API_KEY` 兜底
* 配乐模块只负责生成音频文件音量淡出循环与混音全部复用
app/services/generate_video.py 中现有的音频处理逻辑解说配音的
音量压制策略保持不变
* 任何失败超时HTTP 错误流中断时长超限混音失败都只记录
日志并返回空字符串由调用方回退到现有逻辑绝不中断成片任务
* 上传属于计费操作上传前先用 ffprobe 在本地校验视频时长配乐接口
目前拒绝超过 6 分钟的视频音效接口拒绝超过 3 分钟的视频避免
白传一次注定被拒绝的成片
配乐接口返回 NDJSON 事件流`audio_chunk`base64 音频分片
stream_index 分组`title``complete`成功终止事件 `error`
失败终止事件进度事件与无法解析的行一律忽略生成的音频为 AAC
编码的 .m4a 文件
音效接口为异步任务管线`POST /v1/video-to-sfx` 受理后即计费并返回
`{"task_id": ...}``GET /v1/tasks/{task_id}` 返回
`{"status": "succeeded"/"failed"/..., "audio": {"url": ...}, "error": ...,
"refunded": ...}`结果地址是预签名 URL下载时绝不能携带 API Key
配置config.toml [app]
sonilo_api_key = "..." # 必填,启用开关(配乐与音效共用)
# sonilo_base_url = "https://api.sonilo.com"
# sonilo_timeout_seconds = 600
# sonilo_bgm_prompt = "" # 可选:配乐风格提示
# sonilo_sfx_prompt = "" # 可选:音效风格提示
# sonilo_sfx_volume = 0.6 # 音效混入原声之下的音量0-2]
"""
import base64
import binascii
import json
import os
import subprocess
import time
from typing import Iterable, Optional
import requests
from loguru import logger
from app.config import config
DEFAULT_BASE_URL = "https://api.sonilo.com"
VIDEO_TO_MUSIC_PATH = "/v1/video-to-music"
VIDEO_TO_SFX_PATH = "/v1/video-to-sfx"
TASKS_PATH = "/v1/tasks"
# 后端生成接口的读超时约为 600 秒。生成一旦开始就会计费,客户端过早超时
# 只会浪费一次已经付费的请求,所以默认读超时与后端保持一致,并允许覆盖。
DEFAULT_TIMEOUT_SECONDS = 600
_CONNECT_TIMEOUT_SECONDS = 15
# 轮询任务状态是免费且幂等的 GET单次请求用短读超时即可。
_POLL_READ_TIMEOUT_SECONDS = 30
_SFX_POLL_INTERVAL_SECONDS = 5.0
# 测试接缝:单测里替换为 no-op避免真实等待。
_sleep = time.sleep
# 配乐接口目前拒绝超过 6 分钟的视频;上传前先在本地校验时长。
MAX_VIDEO_DURATION_SECONDS = 360
# 音效接口目前拒绝超过 3 分钟的视频。
MAX_SFX_VIDEO_DURATION_SECONDS = 180
# 音效混入原声之下的默认音量(解说配音在后续合成步骤中以 1.0 混入,
# 音效保持在其之下)。可通过 sonilo_sfx_volume 配置覆盖。
DEFAULT_SFX_VOLUME = 0.6
class SoniloError(Exception):
"""Sonilo 配乐生成失败。"""
def get_api_key() -> str:
"""返回配置的 Sonilo API Key优先 config.toml其次环境变量"""
api_key = config.app.get("sonilo_api_key", "") or os.getenv("SONILO_API_KEY", "")
return str(api_key).strip()
def is_enabled() -> bool:
"""仅当配置了 Sonilo API Key 时返回 True。"""
return bool(get_api_key())
def _get_base_url() -> str:
return str(config.app.get("sonilo_base_url", "") or DEFAULT_BASE_URL).rstrip("/")
def generate_bgm(video_path: str, save_path: str) -> str:
"""
上传合成完成的视频未加 BGM Sonilo生成配乐并保存到
`save_path`.m4a
成功时返回音频文件路径任何失败都返回空字符串由调用方回退到
现有的 BGM 逻辑本函数绝不抛出异常 配乐问题绝不能中断成片任务
"""
if not is_enabled():
logger.warning("Sonilo 配乐已跳过: 未配置 API Key")
return ""
if not video_path or not os.path.isfile(video_path):
logger.warning(f"Sonilo 配乐已跳过: 视频文件不存在: {video_path}")
return ""
# 上传即计费,先在本地校验时长,避免白传一次注定被拒绝的成片。
duration = _probe_video_duration(video_path)
if duration and duration > MAX_VIDEO_DURATION_SECONDS:
logger.warning(
f"Sonilo 配乐已跳过: 视频时长 {duration:.1f}s 超过接口上限 "
f"{MAX_VIDEO_DURATION_SECONDS}s"
)
return ""
try:
audio = _request_video_to_music(video_path)
except Exception as e:
# 任何失败超时、HTTP 错误、流中断)都降级,由调用方回退到
# 现有 BGM 逻辑,绝不让配乐问题中断成片任务。
logger.error(f"Sonilo 配乐生成失败: {str(e)}")
return ""
try:
with open(save_path, "wb") as f:
f.write(audio)
except OSError as e:
logger.error(f"Sonilo 配乐文件保存失败: {str(e)}")
return ""
logger.success(f"Sonilo 配乐已生成: {save_path}")
return save_path
def _get_ffprobe_binary() -> str:
"""与 generate_video 保持一致的 ffprobe 查找逻辑(环境变量优先)。"""
for env_name in ("NARRATO_FFPROBE_EXE", "IMAGEIO_FFPROBE_EXE"):
candidate = os.environ.get(env_name, "").strip()
if candidate and os.path.isfile(candidate):
return candidate
return "ffprobe"
def _probe_video_duration(video_path: str) -> float:
"""
尽力而为的本地 ffprobe 时长探测ffprobe 不可用超时或输出无法解析
时返回 0.0交给后端做最终校验
"""
try:
result = subprocess.run(
[
_get_ffprobe_binary(),
"-v",
"quiet",
"-print_format",
"json",
"-show_format",
video_path,
],
capture_output=True,
timeout=30,
)
except (OSError, subprocess.TimeoutExpired):
return 0.0
if result.returncode != 0:
return 0.0
try:
return float(json.loads(result.stdout)["format"]["duration"])
except (json.JSONDecodeError, KeyError, TypeError, ValueError):
return 0.0
def _error_detail(body: str) -> str:
try:
parsed = json.loads(body)
except (json.JSONDecodeError, TypeError):
return body
if isinstance(parsed, dict):
detail = parsed.get("detail") or parsed.get("error") or parsed.get("message")
if isinstance(detail, str) and detail.strip():
return detail.strip()
return body
def _http_error_message(status_code: int, body: str) -> str:
detail = _error_detail(body)
if status_code == 401:
return "Sonilo API Key 无效,请检查配置"
if status_code == 402:
return detail or "Sonilo 账户余额不足"
if status_code == 413:
return f"视频文件过大: {detail}"
if status_code == 429:
return f"触发 Sonilo 频率限制: {detail}"
return f"Sonilo 接口错误 ({status_code}): {detail}"
def _get_timeout_seconds() -> float:
try:
return float(
config.app.get("sonilo_timeout_seconds", DEFAULT_TIMEOUT_SECONDS)
)
except (TypeError, ValueError):
return DEFAULT_TIMEOUT_SECONDS
def _request_video_to_music(video_path: str) -> bytes:
base_url = _get_base_url()
timeout_seconds = _get_timeout_seconds()
prompt = str(config.app.get("sonilo_bgm_prompt", "") or "").strip()
data: Optional[dict] = {"prompt": prompt} if prompt else None
headers = {"Authorization": f"Bearer {get_api_key()}"}
logger.info(
f"正在使用 Sonilo 生成配乐, 视频: {video_path}, "
f"读超时: {timeout_seconds:.0f}s"
)
try:
with open(video_path, "rb") as video_file:
files = {
"video": (os.path.basename(video_path), video_file, "video/mp4"),
}
# 生成接口非幂等(生成即计费),失败不做自动重试,直接降级。
with requests.post(
f"{base_url}{VIDEO_TO_MUSIC_PATH}",
headers=headers,
data=data,
files=files,
stream=True,
timeout=(_CONNECT_TIMEOUT_SECONDS, timeout_seconds),
) as response:
if response.status_code >= 400:
body = response.content.decode("utf-8", errors="replace")
raise SoniloError(
_http_error_message(response.status_code, body)
)
return _consume_ndjson_stream(
response.iter_lines(decode_unicode=True)
)
except requests.exceptions.Timeout as exc:
raise SoniloError(
f"Sonilo 请求超时 ({timeout_seconds:.0f}s)"
) from exc
except requests.exceptions.RequestException as exc:
raise SoniloError(f"Sonilo 请求失败: {str(exc)}") from exc
def _consume_ndjson_stream(lines: Iterable[str]) -> bytes:
"""
消费 NDJSON 事件流 stream_index 分组 base64 音频分片
返回第一条音轨
"""
streams = {}
completed = False
for line in lines:
if not line or not line.strip():
continue
try:
event = json.loads(line)
except json.JSONDecodeError:
continue
if not isinstance(event, dict):
continue
event_type = event.get("type")
if event_type == "audio_chunk":
chunk = event.get("data")
if not isinstance(chunk, str):
continue
try:
index = int(event.get("stream_index", 0))
except (TypeError, ValueError):
continue
if index < 0:
continue
try:
decoded = base64.b64decode(chunk, validate=True)
except (binascii.Error, ValueError):
continue
streams.setdefault(index, bytearray()).extend(decoded)
elif event_type == "complete":
completed = True
elif event_type == "error":
message = event.get("message") or event.get("code") or "stream error"
raise SoniloError(f"Sonilo 生成失败: {message}")
# title / stage_start 等进度事件一律忽略。
if not completed:
raise SoniloError("Sonilo 事件流意外终止(未收到 complete 事件)")
if not streams:
raise SoniloError("Sonilo 事件流已完成但未返回音频数据")
first_index = sorted(streams)[0]
return bytes(streams[first_index])
# ---------- AI 音效SFX可选功能默认关闭 ----------
def apply_sfx(video_path: str, output_path: str) -> str:
"""
为合成完成的视频生成 Sonilo 音效并用 ffmpeg 混在现有音轨之下
输出新视频到 `output_path`视频流直接复制不重编码画面
成功时返回输出视频路径任何失败都返回空字符串由调用方沿用
原视频本函数绝不抛出异常 音效问题绝不能中断成片任务
"""
sfx_audio_path = os.path.splitext(output_path)[0] + ".m4a"
if not generate_sfx(video_path, sfx_audio_path):
return ""
return _mix_sfx_under_original(video_path, sfx_audio_path, output_path)
def generate_sfx(video_path: str, save_path: str) -> str:
"""
上传合成完成的视频到 Sonilo生成音效音频并保存到 `save_path`.m4a
成功时返回音频文件路径任何失败都返回空字符串 generate_bgm
的约定一致本函数绝不抛出异常
"""
if not is_enabled():
logger.warning("Sonilo 音效已跳过: 未配置 API Key")
return ""
if not video_path or not os.path.isfile(video_path):
logger.warning(f"Sonilo 音效已跳过: 视频文件不存在: {video_path}")
return ""
# 任务受理即计费,先在本地校验时长,避免白传一次注定被拒绝的成片。
duration = _probe_video_duration(video_path)
if duration and duration > MAX_SFX_VIDEO_DURATION_SECONDS:
logger.warning(
f"Sonilo 音效已跳过: 视频时长 {duration:.1f}s 超过接口上限 "
f"{MAX_SFX_VIDEO_DURATION_SECONDS}s"
)
return ""
try:
audio = _request_video_to_sfx(video_path)
except Exception as e:
logger.error(f"Sonilo 音效生成失败: {str(e)}")
return ""
try:
with open(save_path, "wb") as f:
f.write(audio)
except OSError as e:
logger.error(f"Sonilo 音效文件保存失败: {str(e)}")
return ""
logger.success(f"Sonilo 音效已生成: {save_path}")
return save_path
def _request_video_to_sfx(video_path: str) -> bytes:
"""提交音效任务、轮询到终态、下载结果音频。失败抛出 SoniloError。"""
task_id = _submit_sfx_task(video_path)
body = _poll_sfx_task(task_id)
return _download_sfx_audio(_extract_sfx_audio_url(body, task_id))
def _submit_sfx_task(video_path: str) -> str:
"""POST /v1/video-to-sfx受理后返回 task_id受理即计费不做重试"""
timeout_seconds = _get_timeout_seconds()
prompt = str(config.app.get("sonilo_sfx_prompt", "") or "").strip()
data: Optional[dict] = {"prompt": prompt} if prompt else None
headers = {"Authorization": f"Bearer {get_api_key()}"}
logger.info(f"正在提交 Sonilo 音效任务, 视频: {video_path}")
try:
with open(video_path, "rb") as video_file:
files = {
"video": (os.path.basename(video_path), video_file, "video/mp4"),
}
response = requests.post(
f"{_get_base_url()}{VIDEO_TO_SFX_PATH}",
headers=headers,
data=data,
files=files,
timeout=(_CONNECT_TIMEOUT_SECONDS, timeout_seconds),
)
except requests.exceptions.Timeout as exc:
raise SoniloError(
f"Sonilo 音效任务提交超时 ({timeout_seconds:.0f}s)"
) from exc
except requests.exceptions.RequestException as exc:
raise SoniloError(f"Sonilo 音效任务提交失败: {str(exc)}") from exc
if response.status_code >= 400:
body = response.content.decode("utf-8", errors="replace")
raise SoniloError(_http_error_message(response.status_code, body))
try:
task_id = response.json().get("task_id")
except (ValueError, AttributeError):
task_id = None
if not task_id:
raise SoniloError("Sonilo 音效任务已受理但未返回 task_id")
task_id = str(task_id)
# 受理即计费;先把 task_id 落进日志,后续轮询失败时仍有据可查。
logger.info(f"Sonilo 音效任务已提交: {task_id}")
return task_id
def _poll_sfx_task(task_id: str) -> dict:
"""
轮询 GET /v1/tasks/{task_id} 直到任务终态succeeded/failed或超时
succeeded 时返回任务体failed / 超时 / 不可恢复的 HTTP 错误抛出
SoniloError轮询是免费且幂等的 GET网络抖动与 5xx 不该报废一次
已计费的任务在截止时间内继续重试
"""
headers = {"Authorization": f"Bearer {get_api_key()}"}
timeout_seconds = _get_timeout_seconds()
deadline = time.monotonic() + timeout_seconds
while True:
response = None
try:
response = requests.get(
f"{_get_base_url()}{TASKS_PATH}/{task_id}",
headers=headers,
timeout=(_CONNECT_TIMEOUT_SECONDS, _POLL_READ_TIMEOUT_SECONDS),
)
except requests.exceptions.RequestException as exc:
logger.warning(f"Sonilo 音效任务查询失败(将重试): {str(exc)}")
if response is not None:
if response.status_code >= 500:
logger.warning(
f"Sonilo 音效任务查询返回 {response.status_code}(将重试)"
)
elif response.status_code >= 400:
body = response.content.decode("utf-8", errors="replace")
raise SoniloError(
f"{_http_error_message(response.status_code, body)}"
f"(任务已提交, task_id: {task_id}"
)
else:
try:
body = response.json()
except ValueError:
body = None
if isinstance(body, dict):
status = body.get("status")
if status == "succeeded":
return body
if status == "failed":
raise SoniloError(_task_failure_message(body, task_id))
# 非终态pending / processing 等)继续等待。
if time.monotonic() >= deadline:
raise SoniloError(
f"等待 Sonilo 音效任务超时 ({timeout_seconds:.0f}s), "
f"task_id: {task_id}"
)
_sleep(_SFX_POLL_INTERVAL_SECONDS)
def _task_failure_message(body: dict, task_id: str) -> str:
err = body.get("error")
if isinstance(err, dict):
message = err.get("message") or err.get("code") or "生成失败"
elif isinstance(err, str) and err:
message = err
else:
message = "生成失败"
refund_note = ",费用已退还" if body.get("refunded") is True else ""
return f"Sonilo 音效生成失败: {message}task_id: {task_id}{refund_note}"
def _extract_sfx_audio_url(body: dict, task_id: str) -> str:
audio = body.get("audio")
if isinstance(audio, dict):
url = audio.get("url")
if isinstance(url, str) and url:
return url
raise SoniloError(
f"Sonilo 音效任务成功但未返回音频结果 (task_id: {task_id})"
)
def _download_sfx_audio(url: str) -> bytes:
"""下载任务结果音频。结果地址是预签名 URL自带鉴权 ——
绝不能把 API Key 发给存储域名因此这里不带任何鉴权头"""
try:
response = requests.get(
url, timeout=(_CONNECT_TIMEOUT_SECONDS, _get_timeout_seconds())
)
except requests.exceptions.RequestException as exc:
raise SoniloError(f"Sonilo 音效结果下载失败: {str(exc)}") from exc
if response.status_code >= 400:
raise SoniloError(
f"Sonilo 音效结果下载失败 (HTTP {response.status_code})"
)
if not response.content:
raise SoniloError("Sonilo 音效结果为空")
return response.content
def _get_sfx_volume() -> float:
"""音效混入原声之下的音量。非法值或 <=0 回退默认值,上限 2.0。"""
try:
volume = float(config.app.get("sonilo_sfx_volume", DEFAULT_SFX_VOLUME))
except (TypeError, ValueError):
return DEFAULT_SFX_VOLUME
if volume <= 0:
return DEFAULT_SFX_VOLUME
return min(volume, 2.0)
def _get_ffmpeg_binary() -> str:
"""与 generate_video 保持一致的 ffmpeg 查找逻辑(环境变量优先)。"""
for env_name in ("NARRATO_FFMPEG_EXE", "IMAGEIO_FFMPEG_EXE"):
candidate = os.environ.get(env_name, "").strip()
if candidate and os.path.isfile(candidate):
return candidate
try:
import imageio_ffmpeg
candidate = imageio_ffmpeg.get_ffmpeg_exe()
if candidate and os.path.isfile(candidate):
return candidate
except Exception:
pass
return "ffmpeg"
def _probe_has_audio_stream(video_path: str) -> bool:
"""尽力而为地探测视频是否带音轨。探测失败按无音轨处理。"""
try:
result = subprocess.run(
[
_get_ffprobe_binary(),
"-v",
"quiet",
"-print_format",
"json",
"-show_streams",
"-select_streams",
"a",
video_path,
],
capture_output=True,
timeout=30,
)
except (OSError, subprocess.TimeoutExpired):
return False
if result.returncode != 0:
return False
try:
return bool(json.loads(result.stdout).get("streams"))
except (json.JSONDecodeError, AttributeError, TypeError, ValueError):
return False
def _mix_sfx_under_original(
video_path: str, sfx_audio_path: str, output_path: str
) -> str:
"""
ffmpeg 把音效混在成片现有音轨之下音效音量默认 0.6原声音量
不变视频流直接复制不重编码成片没有音轨时音效直接作为音轨
写入成功返回 output_path任何失败返回空字符串
"""
volume = _get_sfx_volume()
has_audio = _probe_has_audio_stream(video_path)
if has_audio:
filter_complex = (
f"[1:a]volume={volume}[sfx];"
"[0:a][sfx]amix=inputs=2:duration=first:"
"dropout_transition=0:normalize=0[aout]"
)
else:
filter_complex = f"[1:a]volume={volume}[aout]"
cmd = [
_get_ffmpeg_binary(),
"-y",
"-i",
video_path,
"-i",
sfx_audio_path,
"-filter_complex",
filter_complex,
"-map",
"0:v",
"-map",
"[aout]",
"-c:v",
"copy",
"-c:a",
"aac",
"-b:a",
"192k",
]
if not has_audio:
cmd.append("-shortest")
cmd.append(output_path)
logger.info(f"正在混入 Sonilo 音效 (音量 {volume}): {output_path}")
try:
result = subprocess.run(cmd, capture_output=True, timeout=300)
except (OSError, subprocess.TimeoutExpired) as e:
logger.error(f"Sonilo 音效混音失败: {str(e)}")
return ""
if result.returncode != 0:
stderr_tail = (result.stderr or b"").decode("utf-8", errors="replace")[-500:]
logger.error(f"Sonilo 音效混音失败 (ffmpeg 退出码 {result.returncode}): {stderr_tail}")
return ""
logger.success(f"Sonilo 音效已混入: {output_path}")
return output_path

View File

@ -10,6 +10,8 @@ from typing import Any
from loguru import logger
from app.config import config
from app.config.defaults import resolve_text_model_name
from app.services.llm.manager import LLMServiceManager
from app.services.llm.migration_adapter import _run_async_safely
from app.services.llm.unified_service import UnifiedLLMService
@ -174,12 +176,16 @@ def correct_srt_content(
provider: str = "",
api_key: str = "",
base_url: str = "",
model_name: str = "",
temperature: float = 0.1,
) -> str:
blocks = parse_srt_blocks(srt_content)
_ensure_llm_providers_registered()
logger.info(f"开始校准字幕,共 {len(blocks)}")
resolved_model_name = str(
model_name or resolve_text_model_name(config.app, provider, prefer_fast=True)
).strip()
logger.info(f"开始使用高效率模型 {resolved_model_name} 校准字幕,共 {len(blocks)}")
prompt = _build_correction_prompt(blocks)
raw_output = _run_async_safely(
UnifiedLLMService.generate_text,
@ -190,6 +196,8 @@ def correct_srt_content(
response_format="json",
api_key=api_key,
api_base=base_url,
model=resolved_model_name,
thinking_level="off",
)
corrections = _parse_corrections(raw_output, {block.order for block in blocks})
corrected_srt = _render_srt(blocks, corrections)
@ -215,6 +223,7 @@ def correct_subtitle_file(
provider: str = "",
api_key: str = "",
base_url: str = "",
model_name: str = "",
temperature: float = 0.1,
) -> str:
if not subtitle_file or not os.path.isfile(subtitle_file):
@ -226,6 +235,7 @@ def correct_subtitle_file(
provider=provider,
api_key=api_key,
base_url=base_url,
model_name=model_name,
temperature=temperature,
)
return write_srt_file(corrected_srt, output_file)

View File

@ -11,6 +11,7 @@ from typing import Any, Callable
from loguru import logger
from app.config import config
from app.config.defaults import resolve_text_model_name
from app.services.llm.migration_adapter import _run_async_safely
from app.services.llm.unified_service import UnifiedLLMService
from app.services.subtitle_corrector import (
@ -151,6 +152,7 @@ def _translate_chunk(
provider: str,
api_key: str,
base_url: str,
model_name: str,
temperature: float,
max_repair_attempts: int,
) -> dict[int, str]:
@ -189,6 +191,8 @@ def _translate_chunk(
response_format="json",
api_key=api_key,
api_base=base_url,
model=model_name,
thinking_level="off",
)
last_output = str(raw_output or "")
try:
@ -243,6 +247,7 @@ def translate_srt_content(
provider: str = "",
api_key: str = "",
base_url: str = "",
model_name: str = "",
temperature: float = 0.2,
batch_size: int | None = None,
max_workers: int | None = None,
@ -251,6 +256,9 @@ def translate_srt_content(
target_language = str(target_language or "").strip() or "中文"
blocks = parse_srt_blocks(srt_content)
_ensure_llm_providers_registered()
resolved_model_name = str(
model_name or resolve_text_model_name(config.app, provider, prefer_fast=True)
).strip()
resolved_batch_size = _resolve_batch_size(batch_size)
chunks = _split_blocks(blocks, resolved_batch_size)
@ -260,7 +268,8 @@ def translate_srt_content(
logger.info(
f"开始批量翻译字幕: 共 {total_blocks} 条, {total_chunks} 批, "
f"每批最多 {resolved_batch_size} 条, 并发 {resolved_max_workers}, 目标语言: {target_language}"
f"每批最多 {resolved_batch_size} 条, 并发 {resolved_max_workers}, "
f"目标语言: {target_language}, 高效率模型: {resolved_model_name}"
)
translations: dict[int, str] = {}
@ -282,6 +291,7 @@ def translate_srt_content(
provider=provider,
api_key=api_key,
base_url=base_url,
model_name=resolved_model_name,
temperature=temperature,
max_repair_attempts=DEFAULT_MAX_REPAIR_ATTEMPTS,
)
@ -301,6 +311,7 @@ def translate_srt_content(
provider=provider,
api_key=api_key,
base_url=base_url,
model_name=resolved_model_name,
temperature=temperature,
max_repair_attempts=DEFAULT_MAX_REPAIR_ATTEMPTS,
)
@ -347,6 +358,7 @@ def translate_subtitle_file(
provider: str = "",
api_key: str = "",
base_url: str = "",
model_name: str = "",
temperature: float = 0.2,
batch_size: int | None = None,
max_workers: int | None = None,
@ -362,6 +374,7 @@ def translate_subtitle_file(
provider=provider,
api_key=api_key,
base_url=base_url,
model_name=model_name,
temperature=temperature,
batch_size=batch_size,
max_workers=max_workers,

View File

@ -19,6 +19,7 @@ from app.services import (
update_script,
generate_video,
script_subtitle,
sonilo,
)
from app.services import state as sm
from app.utils import utils
@ -214,6 +215,45 @@ def _build_subtitle_mask_options(params: VideoClipParams, enabled=None) -> dict:
}
def _resolve_bgm_path(task_id: str, params: VideoClipParams, combined_video_path: str) -> str:
"""解析最终合成使用的背景音乐文件路径。
bgm_type "sonilo" 可选功能默认关闭将合并后的成片上传到
Sonilo API 生成配乐任何失败都只记录日志并回退到现有的随机背景音乐
逻辑绝不中断成片任务其余 bgm_type 走原有逻辑保持不变
"""
if getattr(params, "bgm_type", "") == "sonilo":
save_path = path.join(utils.task_dir(task_id), "sonilo_bgm.m4a")
bgm_path = sonilo.generate_bgm(combined_video_path, save_path)
if bgm_path:
return bgm_path
logger.warning("Sonilo 配乐不可用,回退到随机背景音乐")
return utils.get_bgm_file(bgm_type="random", bgm_file="")
return utils.get_bgm_file(
bgm_type=getattr(params, "bgm_type", "random"),
bgm_file=getattr(params, "bgm_file", ""),
)
def _apply_sonilo_sfx(task_id: str, params: VideoClipParams, combined_video_path: str) -> str:
"""为合并后的成片混入 Sonilo AI 音效(可选功能,默认关闭)。
仅当 params.sonilo_sfx_enabled True 时启用把合并后的成片上传到
Sonilo API 生成音效再用 ffmpeg 混在现有音轨之下返回新视频路径
解说配音在后续 merge_materials 中单独混入音量策略不受影响任何
失败都只记录日志并沿用原视频绝不中断成片任务
"""
if not getattr(params, "sonilo_sfx_enabled", False):
return combined_video_path
output_path = path.join(utils.task_dir(task_id), "merger_sfx.mp4")
sfx_video_path = sonilo.apply_sfx(combined_video_path, output_path)
if sfx_video_path:
return sfx_video_path
logger.warning("Sonilo 音效不可用,继续使用未加音效的成片")
return combined_video_path
def _transcribe_final_video(task_id: str, video_path: str, params: VideoClipParams) -> str:
"""Transcribe the fully merged video into an SRT file."""
from app.services import fun_asr_subtitle
@ -520,11 +560,11 @@ def start_subclip(task_id: str, params: VideoClipParams, subclip_path_videos: di
)
logger.info(f"\n\n## 6. 最后一步: 合并字幕/BGM/配音/视频 -> {merge_output_video_path}")
# 可选功能,默认关闭:混入 Sonilo AI 音效(失败时沿用原视频)
combined_video_path = _apply_sonilo_sfx(task_id, params, combined_video_path)
# bgm_path = '/Users/apple/Desktop/home/NarratoAI/resource/songs/bgm.mp3'
bgm_path = utils.get_bgm_file(
bgm_type=getattr(params, "bgm_type", "random"),
bgm_file=getattr(params, "bgm_file", ""),
)
bgm_path = _resolve_bgm_path(task_id, params, combined_video_path)
# 获取优化的音量配置
optimized_volumes = get_recommended_volumes_for_content('mixed')
@ -850,10 +890,10 @@ def start_subclip_unified(task_id: str, params: VideoClipParams):
ffmpeg_progress=0,
)
bgm_path = utils.get_bgm_file(
bgm_type=getattr(params, "bgm_type", "random"),
bgm_file=getattr(params, "bgm_file", ""),
)
# 可选功能,默认关闭:混入 Sonilo AI 音效(失败时沿用原视频)
combined_video_path = _apply_sonilo_sfx(task_id, params, combined_video_path)
bgm_path = _resolve_bgm_path(task_id, params, combined_video_path)
# 获取优化的音量配置
optimized_volumes = get_recommended_volumes_for_content('mixed')

View File

@ -0,0 +1,211 @@
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from app.services import voice
class FakeResponse:
def __init__(self, *, status_code=200, content=b"", payload=None, content_type="application/json"):
self.status_code = status_code
self.content = content
self._payload = payload or {}
self.headers = {"content-type": content_type}
self.text = "OK"
def json(self):
return self._payload
class IndexTTS2TtsTests(unittest.TestCase):
def setUp(self):
self.original_indextts2 = dict(voice.config.indextts2)
self.original_proxy = dict(voice.config.proxy)
def tearDown(self):
voice.config.indextts2.clear()
voice.config.indextts2.update(self.original_indextts2)
voice.config.proxy.clear()
voice.config.proxy.update(self.original_proxy)
def test_uploads_reference_audio_and_downloads_pack_output_url(self):
voice.config.indextts2.clear()
voice.config.indextts2.update(
{
"api_url": "http://127.0.0.1:7860",
"emotion": "happy:0.7,calm:0.3",
"emo_alpha": 0.6,
"speed": 1.15,
"seed": 20260713,
"max_mel_tokens": 1500,
"max_text_tokens_per_segment": 120,
"interval_silence": 200,
"temperature": 0.8,
"top_p": 0.8,
"top_k": 30,
"repetition_penalty": 10.0,
"diffusion_steps": 25,
"cfg_rate": 0.7,
"segment_overlap_ms": 50,
}
)
voice.config.proxy.clear()
generation_response = FakeResponse(
payload={"output": {"url": "/outputs/audio/speech.wav"}}
)
download_response = FakeResponse(content=b"wav-bytes", content_type="audio/wav")
with tempfile.TemporaryDirectory() as temp_dir:
reference_audio = Path(temp_dir) / "reference.wav"
output_file = Path(temp_dir) / "output.wav"
reference_audio.write_bytes(b"reference-wav")
with (
patch("app.services.voice.requests.post", return_value=generation_response) as post,
patch("app.services.voice.requests.get", return_value=download_response) as get,
patch("app.services.voice.get_audio_duration_from_file", return_value=1.25),
):
result = voice.indextts2_tts(
text=" 新版接口测试。 ",
voice_name=f"indextts2:{reference_audio}",
voice_file=str(output_file),
)
output_bytes = output_file.read_bytes() if output_file.exists() else b""
self.assertIsNotNone(result)
self.assertEqual(output_bytes, b"wav-bytes")
self.assertEqual(
post.call_args.args[0],
"http://127.0.0.1:7860/v1/audio/speech/upload",
)
self.assertEqual(
post.call_args.kwargs["data"],
{
"text": "新版接口测试。",
"emotion": "happy:0.7,calm:0.3",
"emo_alpha": 0.6,
"speed": 1.15,
"seed": 20260713,
"max_mel_tokens": 1500,
"max_text_tokens_per_segment": 120,
"interval_silence": 200,
"temperature": 0.8,
"top_p": 0.8,
"top_k": 30,
"repetition_penalty": 10.0,
"diffusion_steps": 25,
"cfg_rate": 0.7,
"segment_overlap_ms": 50,
},
)
self.assertIn("reference_audio", post.call_args.kwargs["files"])
self.assertEqual(
get.call_args.args[0],
"http://127.0.0.1:7860/outputs/audio/speech.wav",
)
def test_maps_legacy_vector_settings_to_pack_emotion_and_endpoint(self):
voice.config.indextts2.clear()
voice.config.indextts2.update(
{
"api_url": "http://127.0.0.1:7860/tts",
"emotion_mode": "vector",
"emotion_alpha": 0.65,
"vec_happy": 0.3,
"vec_calm": 0.7,
}
)
voice.config.proxy.clear()
generation_response = FakeResponse(content=b"wav-bytes", content_type="audio/wav")
with tempfile.TemporaryDirectory() as temp_dir:
reference_audio = Path(temp_dir) / "reference.wav"
output_file = Path(temp_dir) / "output.wav"
reference_audio.write_bytes(b"reference-wav")
with (
patch("app.services.voice.requests.post", return_value=generation_response) as post,
patch("app.services.voice.get_audio_duration_from_file", return_value=1.0),
):
result = voice.indextts2_tts(
text="旧配置兼容测试。",
voice_name=f"indextts2:{reference_audio}",
voice_file=str(output_file),
)
self.assertIsNotNone(result)
self.assertEqual(
post.call_args.args[0],
"http://127.0.0.1:7860/v1/audio/speech/upload",
)
self.assertEqual(post.call_args.kwargs["data"]["emotion"], "happy:0.3,calm:0.7")
self.assertEqual(post.call_args.kwargs["data"]["emo_alpha"], 0.65)
self.assertNotIn("emotion_mode", post.call_args.kwargs["data"])
self.assertNotIn("vec_happy", post.call_args.kwargs["data"])
def test_normalizes_saved_values_to_pack_request_ranges(self):
voice.config.indextts2.clear()
voice.config.indextts2.update(
{
"api_url": "http://127.0.0.1:7860",
"emo_alpha": 2.0,
"speed": 0.1,
"max_mel_tokens": 50,
"max_text_tokens_per_segment": 1,
"interval_silence": -10,
"temperature": 0.0,
"top_p": 0.0,
"top_k": 0,
"repetition_penalty": 0.1,
"diffusion_steps": 0,
"cfg_rate": -1.0,
"segment_overlap_ms": -5,
"seed": "not-an-integer",
}
)
voice.config.proxy.clear()
generation_response = FakeResponse(content=b"wav-bytes", content_type="audio/wav")
with tempfile.TemporaryDirectory() as temp_dir:
reference_audio = Path(temp_dir) / "reference.wav"
output_file = Path(temp_dir) / "output.wav"
reference_audio.write_bytes(b"reference-wav")
with (
patch("app.services.voice.requests.post", return_value=generation_response) as post,
patch("app.services.voice.get_audio_duration_from_file", return_value=1.0),
):
result = voice.indextts2_tts(
text="参数范围测试。",
voice_name=f"indextts2:{reference_audio}",
voice_file=str(output_file),
)
self.assertIsNotNone(result)
self.assertEqual(
post.call_args.kwargs["data"],
{
"text": "参数范围测试。",
"emo_alpha": 1.0,
"speed": 0.5,
"max_mel_tokens": 64,
"max_text_tokens_per_segment": 20,
"interval_silence": 0,
"temperature": 0.05,
"top_p": 0.05,
"top_k": 1,
"repetition_penalty": 1.0,
"diffusion_steps": 1,
"cfg_rate": 0.0,
"segment_overlap_ms": 0,
},
)
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,102 @@
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from app.services import voice
class FakeResponse:
def __init__(self, *, status_code=200, content=b"", payload=None, content_type="application/json"):
self.status_code = status_code
self.content = content
self._payload = payload or {}
self.headers = {"content-type": content_type}
self.text = "OK"
def json(self):
return self._payload
class IndexTTSMacOSTtsTests(unittest.TestCase):
def setUp(self):
self.original_config = dict(voice.config.indextts_macos)
self.original_proxy = dict(voice.config.proxy)
def tearDown(self):
voice.config.indextts_macos.clear()
voice.config.indextts_macos.update(self.original_config)
voice.config.proxy.clear()
voice.config.proxy.update(self.original_proxy)
def test_uploads_reference_audio_and_downloads_pack_output_url(self):
voice.config.indextts_macos.clear()
voice.config.indextts_macos.update(
{
"api_url": "http://127.0.0.1:7866",
"speed": 1.1,
"seed": 42,
"max_mel_tokens": 800,
"max_text_tokens_per_segment": 120,
"interval_silence": 200,
"temperature": 1.0,
"top_p": 0.8,
"top_k": 30,
"repetition_penalty": 10.0,
"segment_overlap_ms": 50,
}
)
voice.config.proxy.clear()
generation_response = FakeResponse(payload={"output_url": "/outputs/speech.wav"})
download_response = FakeResponse(content=b"wav-bytes", content_type="audio/wav")
with tempfile.TemporaryDirectory() as temp_dir:
reference_audio = Path(temp_dir) / "reference.wav"
output_file = Path(temp_dir) / "output.wav"
reference_audio.write_bytes(b"reference-wav")
with (
patch("app.services.voice.requests.post", return_value=generation_response) as post,
patch("app.services.voice.requests.get", return_value=download_response) as get,
patch("app.services.voice.get_audio_duration_from_file", return_value=1.25),
):
result = voice.indextts_macos_tts(
text=" macOS 接口测试。 ",
voice_name=f"indextts_macos:{reference_audio}",
voice_file=str(output_file),
)
output_bytes = output_file.read_bytes() if output_file.exists() else b""
self.assertIsNotNone(result)
self.assertEqual(output_bytes, b"wav-bytes")
self.assertEqual(
post.call_args.args[0],
"http://127.0.0.1:7866/v1/audio/speech/upload",
)
self.assertEqual(
post.call_args.kwargs["data"],
{
"text": "macOS 接口测试。",
"speed": 1.1,
"seed": 42,
"max_mel_tokens": 800,
"max_text_tokens_per_segment": 120,
"interval_silence": 200,
"temperature": 1.0,
"top_p": 0.8,
"top_k": 30,
"repetition_penalty": 10.0,
"segment_overlap_ms": 50,
},
)
self.assertIn("reference_audio", post.call_args.kwargs["files"])
self.assertEqual(
get.call_args.args[0],
"http://127.0.0.1:7866/outputs/speech.wav",
)
if __name__ == "__main__":
unittest.main()

View File

@ -27,6 +27,14 @@ class SubtitleCorrectorTests(unittest.TestCase):
}
with (
mock.patch.dict(
corrector.config.app,
{
"text_openai_model_name": "reasoning-model",
"text_openai_fast_model_name": "fast-subtitle-model",
},
clear=False,
),
mock.patch("app.services.subtitle_corrector._ensure_llm_providers_registered"),
mock.patch(
"app.services.subtitle_corrector._run_async_safely",
@ -49,6 +57,8 @@ class SubtitleCorrectorTests(unittest.TestCase):
self.assertEqual("openai", call_kwargs["provider"])
self.assertEqual("sk-test", call_kwargs["api_key"])
self.assertEqual("https://llm.example/v1", call_kwargs["api_base"])
self.assertEqual("fast-subtitle-model", call_kwargs["model"])
self.assertEqual("off", call_kwargs["thinking_level"])
self.assertEqual("json", call_kwargs["response_format"])
self.assertIn("多语言字幕校对员", call_kwargs["system_prompt"])
self.assertIn("保持原语言", call_kwargs["prompt"])

View File

@ -48,6 +48,14 @@ class SubtitleTranslatorTests(unittest.TestCase):
}
with (
mock.patch.dict(
translator.config.app,
{
"text_openai_model_name": "reasoning-model",
"text_openai_fast_model_name": "fast-subtitle-model",
},
clear=False,
),
mock.patch("app.services.subtitle_translator._ensure_llm_providers_registered"),
mock.patch(
"app.services.subtitle_translator._run_async_safely",
@ -71,6 +79,8 @@ class SubtitleTranslatorTests(unittest.TestCase):
self.assertEqual("openai", call_kwargs["provider"])
self.assertEqual("sk-test", call_kwargs["api_key"])
self.assertEqual("https://llm.example/v1", call_kwargs["api_base"])
self.assertEqual("fast-subtitle-model", call_kwargs["model"])
self.assertEqual("off", call_kwargs["thinking_level"])
self.assertEqual("json", call_kwargs["response_format"])
self.assertIn("专业字幕翻译员", call_kwargs["system_prompt"])
self.assertIn("翻译为中文", call_kwargs["prompt"])

View File

@ -0,0 +1,81 @@
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from app.services import voice
class FakeResponse:
def __init__(self, *, status_code=200, content=b"", payload=None):
self.status_code = status_code
self.content = content
self._payload = payload or {}
self.text = "OK"
def json(self):
return self._payload
class VoxCPM2TtsTests(unittest.TestCase):
def setUp(self):
self.original_config = dict(voice.config.voxcpm_2b)
self.original_proxy = dict(voice.config.proxy)
def tearDown(self):
voice.config.voxcpm_2b.clear()
voice.config.voxcpm_2b.update(self.original_config)
voice.config.proxy.clear()
voice.config.proxy.update(self.original_proxy)
def test_voice_design_sends_control_and_downloads_wav(self):
voice.config.voxcpm_2b.clear()
voice.config.voxcpm_2b.update({
"api_url": "http://127.0.0.1:7863/v1/audio/speech",
"mode": "design", "control": "温暖自然的年轻女声",
"cfg_value": 2.0, "inference_timesteps": 10,
"normalize": True, "denoise": False, "output_48k": True,
"context_aware": True, "streaming": False,
})
voice.config.proxy.clear()
with tempfile.TemporaryDirectory() as temp_dir:
output = Path(temp_dir) / "output.wav"
with (
patch("app.services.voice.requests.post", return_value=FakeResponse(payload={"downloads": {"wav": "/download/result.wav"}})) as post,
patch("app.services.voice.requests.get", return_value=FakeResponse(content=b"wav-2b")) as get,
patch("app.services.voice.get_audio_duration_from_file", return_value=2.0),
):
result = voice.voxcpm2_tts(" 高质量旁白。 ", "voxcpm_2b:design", str(output))
self.assertIsNotNone(result)
self.assertEqual(output.read_bytes(), b"wav-2b")
self.assertEqual(post.call_args.args[0], "http://127.0.0.1:7863/tts")
self.assertIsNone(post.call_args.kwargs["files"])
self.assertEqual(post.call_args.kwargs["data"]["control"], "温暖自然的年轻女声")
self.assertEqual(post.call_args.kwargs["data"]["output_48k"], "true")
self.assertEqual(get.call_args.args[0], "http://127.0.0.1:7863/download/result.wav")
def test_clone_uploads_reference_audio(self):
voice.config.voxcpm_2b.clear()
voice.config.voxcpm_2b.update({
"api_url": "http://127.0.0.1:7863/tts/batch",
"mode": "clone", "prompt_text": "参考音频文本",
})
voice.config.proxy.clear()
with tempfile.TemporaryDirectory() as temp_dir:
reference = Path(temp_dir) / "reference.wav"
output = Path(temp_dir) / "output.wav"
reference.write_bytes(b"reference")
with (
patch("app.services.voice.requests.post", return_value=FakeResponse(payload={"downloads": {"wav": "/download/result.wav"}})) as post,
patch("app.services.voice.requests.get", return_value=FakeResponse(content=b"wav")),
patch("app.services.voice.get_audio_duration_from_file", return_value=1.0),
):
result = voice.voxcpm2_tts("克隆测试", f"voxcpm_2b:{reference}", str(output))
self.assertIsNotNone(result)
self.assertEqual(post.call_args.args[0], "http://127.0.0.1:7863/tts")
self.assertIn("reference_audio", post.call_args.kwargs["files"])
self.assertEqual(post.call_args.kwargs["data"]["prompt_text"], "参考音频文本")
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,83 @@
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from app.services import voice
class FakeResponse:
def __init__(self, *, status_code=200, content=b"", payload=None):
self.status_code = status_code
self.content = content
self._payload = payload or {}
self.text = "OK"
def json(self):
return self._payload
class VoxCPMTtsTests(unittest.TestCase):
def setUp(self):
self.original_config = dict(voice.config.voxcpm_05b)
self.original_proxy = dict(voice.config.proxy)
def tearDown(self):
voice.config.voxcpm_05b.clear()
voice.config.voxcpm_05b.update(self.original_config)
voice.config.proxy.clear()
voice.config.proxy.update(self.original_proxy)
def test_uploads_prompt_audio_and_downloads_generated_wav(self):
voice.config.voxcpm_05b.clear()
voice.config.voxcpm_05b.update({
"api_url": "http://127.0.0.1:7864/v1/audio/speech",
"prompt_text": "参考文本",
"cfg_value": 2.1,
"inference_timesteps": 12,
"max_length": 2048,
"normalize": True,
"denoise": False,
})
voice.config.proxy.clear()
with tempfile.TemporaryDirectory() as temp_dir:
reference = Path(temp_dir) / "reference.wav"
output = Path(temp_dir) / "output.wav"
reference.write_bytes(b"reference")
with (
patch("app.services.voice.requests.post", return_value=FakeResponse(payload={"audio_url": "/outputs/result.wav"})) as post,
patch("app.services.voice.requests.get", return_value=FakeResponse(content=b"wav-bytes")) as get,
patch("app.services.voice.get_audio_duration_from_file", return_value=1.5),
):
result = voice.voxcpm_tts(" 测试文本。 ", f"voxcpm_05b:{reference}", str(output))
self.assertIsNotNone(result)
self.assertEqual(output.read_bytes(), b"wav-bytes")
self.assertEqual(post.call_args.args[0], "http://127.0.0.1:7864/tts")
self.assertEqual(post.call_args.kwargs["data"], {
"text": "测试文本。", "prompt_text": "参考文本", "cfg_value": 2.1,
"inference_timesteps": 12, "max_length": 2048,
"normalize": "true", "denoise": "false",
})
self.assertIn("prompt_audio", post.call_args.kwargs["files"])
self.assertEqual(get.call_args.args[0], "http://127.0.0.1:7864/outputs/result.wav")
def test_default_voice_does_not_upload_audio(self):
voice.config.voxcpm_05b.clear()
voice.config.voxcpm_05b.update({"api_url": "http://127.0.0.1:7864"})
voice.config.proxy.clear()
with tempfile.TemporaryDirectory() as temp_dir:
output = Path(temp_dir) / "output.wav"
with (
patch("app.services.voice.requests.post", return_value=FakeResponse(payload={"audio_url": "/outputs/result.wav"})) as post,
patch("app.services.voice.requests.get", return_value=FakeResponse(content=b"wav")),
patch("app.services.voice.get_audio_duration_from_file", return_value=1.0),
):
result = voice.voxcpm_tts("默认音色", "voxcpm_05b:default", str(output))
self.assertIsNotNone(result)
self.assertIsNone(post.call_args.kwargs["files"])
if __name__ == "__main__":
unittest.main()

View File

@ -1302,6 +1302,10 @@ def tts(
logger.info("分发到 IndexTTS-1.5")
return indextts_tts(text, voice_name, voice_file, speed=voice_rate)
if tts_engine == config.INDEXTTS_MACOS_ENGINE:
logger.info("分发到 IndexTTS-1.5-macOS")
return indextts_macos_tts(text, voice_name, voice_file)
if tts_engine == config.INDEXTTS2_ENGINE:
logger.info("分发到 IndexTTS-2")
return indextts2_tts(text, voice_name, voice_file)
@ -1309,6 +1313,14 @@ def tts(
if tts_engine == config.OMNIVOICE_ENGINE:
logger.info("分发到 OmniVoice")
return omnivoice_tts(text, voice_name, voice_file, speed=voice_rate)
if tts_engine == config.VOXCPM_ENGINE:
logger.info("分发到 VoxCPM-0.5B")
return voxcpm_tts(text, voice_name, voice_file)
if tts_engine == config.VOXCPM2_ENGINE:
logger.info("分发到 VoxCPM-2B")
return voxcpm2_tts(text, voice_name, voice_file)
if tts_engine == "doubaotts":
logger.info("分发到豆包语音 TTS")
@ -1796,8 +1808,11 @@ def tts_multiple(task_id: str, list_script: list, voice_name: str, voice_rate: f
tts_results = []
audio_extension = ".wav" if tts_engine in (
config.INDEXTTS_ENGINE,
config.INDEXTTS_MACOS_ENGINE,
config.INDEXTTS2_ENGINE,
config.OMNIVOICE_ENGINE,
config.VOXCPM_ENGINE,
config.VOXCPM2_ENGINE,
) else ".mp3"
for item in list_script:
@ -1828,7 +1843,14 @@ def tts_multiple(task_id: str, list_script: list, voice_name: str, voice_rate: f
if (
is_soulvoice_voice(voice_name)
or is_qwen_engine(tts_engine)
or tts_engine in (config.INDEXTTS_ENGINE, config.INDEXTTS2_ENGINE, config.OMNIVOICE_ENGINE)
or tts_engine in (
config.INDEXTTS_ENGINE,
config.INDEXTTS_MACOS_ENGINE,
config.INDEXTTS2_ENGINE,
config.OMNIVOICE_ENGINE,
config.VOXCPM_ENGINE,
config.VOXCPM2_ENGINE,
)
or tts_engine == "doubaotts"
):
# 获取实际音频文件的时长
@ -2271,6 +2293,13 @@ def parse_indextts2_voice(voice_name: str) -> str:
return voice_name
def parse_indextts_macos_voice(voice_name: str) -> str:
"""解析 IndexTTS-1.5-macOS 参考音频路径。"""
if isinstance(voice_name, str) and voice_name.startswith(config.INDEXTTS_MACOS_VOICE_PREFIX):
return voice_name[len(config.INDEXTTS_MACOS_VOICE_PREFIX):]
return voice_name
def parse_omnivoice_voice(voice_name: str) -> str:
"""
解析 OmniVoice 语音名称
@ -2282,6 +2311,20 @@ def parse_omnivoice_voice(voice_name: str) -> str:
return voice_name
def parse_voxcpm_voice(voice_name: str) -> str:
"""解析 VoxCPM-0.5B 的可选参考音频路径。"""
if isinstance(voice_name, str) and voice_name.startswith(config.VOXCPM_VOICE_PREFIX):
return voice_name[len(config.VOXCPM_VOICE_PREFIX):]
return voice_name
def parse_voxcpm2_voice(voice_name: str) -> str:
"""解析 VoxCPM-2B 的模式或参考音频路径。"""
if isinstance(voice_name, str) and voice_name.startswith(config.VOXCPM2_VOICE_PREFIX):
return voice_name[len(config.VOXCPM2_VOICE_PREFIX):]
return voice_name
def indextts_tts(text: str, voice_name: str, voice_file: str, speed: float = 1.0) -> Union[SubMaker, None]:
"""
使用 IndexTTS-1.5 API 进行零样本语音克隆
@ -2395,10 +2438,166 @@ def indextts_tts(text: str, voice_name: str, voice_file: str, speed: float = 1.0
def _normalize_indextts2_api_url(api_url: str) -> str:
api_url = (api_url or "http://192.168.3.6:7863/tts").strip()
if api_url.endswith("/tts"):
"""Return the IndexTTS-2 MLX Pack upload endpoint for a configured URL.
The Pack accepts a server root, the JSON speech endpoint, or the multipart
upload endpoint. Treat an old ``/tts`` value as a server root so existing
saved settings move to the new Pack route instead of continuing to 404.
"""
api_url = (api_url or "http://127.0.0.1:7860").strip().rstrip("/")
upload_path = "/v1/audio/speech/upload"
speech_path = "/v1/audio/speech"
if api_url.endswith(upload_path):
return api_url
return f"{api_url.rstrip('/')}/tts"
if api_url.endswith(speech_path):
return f"{api_url}/upload"
if api_url.endswith("/tts"):
api_url = api_url[: -len("/tts")]
return f"{api_url}{upload_path}"
def _normalize_indextts_macos_api_url(api_url: str) -> str:
"""Return the IndexTTS 1.5 MLX Pack multipart upload endpoint."""
api_url = (api_url or "http://127.0.0.1:7866").strip().rstrip("/")
upload_path = "/v1/audio/speech/upload"
speech_path = "/v1/audio/speech"
if api_url.endswith(upload_path):
return api_url
if api_url.endswith(speech_path):
return f"{api_url}/upload"
return f"{api_url}{upload_path}"
def _get_indextts_macos_number(
key: str,
default: float | int,
minimum: float | int,
maximum: float | int,
*,
integer: bool = False,
) -> float | int:
try:
raw_value = config.indextts_macos.get(key, default)
value = int(float(raw_value)) if integer else float(raw_value)
except (TypeError, ValueError):
value = default
return max(minimum, min(maximum, value))
def _get_indextts_macos_seed() -> int | None:
seed = config.indextts_macos.get("seed")
if seed in (None, ""):
return None
try:
return int(seed)
except (TypeError, ValueError):
logger.warning("IndexTTS-1.5-macOS 随机种子无效,将使用随机采样: {}", seed)
return None
def _download_indextts_macos_audio(
response: requests.Response,
api_url: str,
voice_file: str,
proxies: dict,
) -> bool:
try:
result = response.json()
except ValueError:
logger.error("IndexTTS-1.5-macOS API 返回了无效的 JSON 响应")
return False
download_url = result.get("output_url") if isinstance(result, dict) else ""
if not download_url:
logger.error(f"IndexTTS-1.5-macOS API 响应中没有音频下载地址: {result}")
return False
audio_response = requests.get(
urljoin(api_url, download_url),
proxies=proxies,
timeout=120,
)
if audio_response.status_code != 200:
logger.error(
f"IndexTTS-1.5-macOS 音频下载失败: "
f"{audio_response.status_code} - {audio_response.text}"
)
return False
with open(voice_file, "wb") as f:
f.write(audio_response.content)
return os.path.getsize(voice_file) > 0
def indextts_macos_tts(text: str, voice_name: str, voice_file: str) -> Union[SubMaker, None]:
"""使用 IndexTTS-MLX-1.5-Pack 的上传接口进行零样本语音克隆。"""
api_url = _normalize_indextts_macos_api_url(
config.indextts_macos.get("api_url", "http://127.0.0.1:7866")
)
reference_audio_path = parse_indextts_macos_voice(voice_name)
if not reference_audio_path or not os.path.exists(reference_audio_path):
logger.error(f"IndexTTS-1.5-macOS 参考音频文件不存在: {reference_audio_path}")
return None
data = {
"text": text.strip(),
"speed": _get_indextts_macos_number("speed", 1.0, 0.5, 2.0),
"max_mel_tokens": _get_indextts_macos_number(
"max_mel_tokens", 800, 64, 1600, integer=True
),
"max_text_tokens_per_segment": _get_indextts_macos_number(
"max_text_tokens_per_segment", 120, 20, 600, integer=True
),
"interval_silence": _get_indextts_macos_number(
"interval_silence", 200, 0, 2000, integer=True
),
"temperature": _get_indextts_macos_number("temperature", 1.0, 0.0, 2.0),
"top_p": _get_indextts_macos_number("top_p", 0.8, 0.05, 1.0),
"top_k": _get_indextts_macos_number("top_k", 30, 0, 200, integer=True),
"repetition_penalty": _get_indextts_macos_number(
"repetition_penalty", 10.0, 1.0, 20.0
),
"segment_overlap_ms": _get_indextts_macos_number(
"segment_overlap_ms", 50, 0, 500, integer=True
),
}
seed = _get_indextts_macos_seed()
if seed is not None:
data["seed"] = seed
proxies = _get_configured_proxies()
for attempt in range(3):
try:
with open(reference_audio_path, "rb") as reference_audio:
logger.info(f"{attempt + 1} 次调用 IndexTTS-1.5-macOS API: {api_url}")
response = requests.post(
api_url,
files={"reference_audio": reference_audio},
data=data,
proxies=proxies,
timeout=600,
)
if response.status_code == 200 and _download_indextts_macos_audio(
response, api_url, voice_file, proxies
):
sub_maker = new_sub_maker()
duration = get_audio_duration_from_file(voice_file)
duration_ms = int(duration * 1000) if duration > 0 else max(1000, int(len(text) * 200))
add_subtitle_event(sub_maker, 0, duration_ms * 10000, text)
return sub_maker
logger.error(
f"IndexTTS-1.5-macOS API 调用失败: {response.status_code} - {response.text}"
)
except requests.exceptions.Timeout:
logger.error(f"IndexTTS-1.5-macOS API 调用超时 (尝试 {attempt + 1}/3)")
except requests.exceptions.RequestException as e:
logger.error(f"IndexTTS-1.5-macOS API 网络错误: {str(e)} (尝试 {attempt + 1}/3)")
except Exception as e:
logger.error(f"IndexTTS-1.5-macOS TTS 处理错误: {str(e)} (尝试 {attempt + 1}/3)")
if attempt < 2:
time.sleep(2)
return None
def _get_configured_proxies() -> dict:
@ -2410,6 +2609,49 @@ def _get_configured_proxies() -> dict:
}
def _get_indextts2_number(
key: str,
default: float | int,
minimum: float | int,
maximum: float | int,
*,
integer: bool = False,
) -> float | int:
"""Read an IndexTTS-2 option and constrain it to the MLX Pack schema."""
try:
raw_value = config.indextts2.get(key, default)
value = int(float(raw_value)) if integer else float(raw_value)
except (TypeError, ValueError):
value = default
return max(minimum, min(maximum, value))
def _get_indextts2_seed() -> int | None:
"""Return the optional Pack seed, omitting invalid legacy text values."""
seed = config.indextts2.get("seed")
if seed in (None, ""):
return None
try:
return int(seed)
except (TypeError, ValueError):
logger.warning("IndexTTS-2 随机种子无效,将使用随机采样: {}", seed)
return None
def _get_indextts2_emotion() -> str:
"""Map current and legacy IndexTTS-2 emotion settings to the MLX Pack API."""
emotion = config.get_indextts2_pack_emotion(config.indextts2)
if emotion:
return emotion
emotion_mode = config.indextts2.get("emotion_mode", "speaker")
if emotion_mode == "audio" and config.indextts2.get("emotion_audio"):
logger.warning(
"IndexTTS-2 MLX Pack 不支持单独的情感参考音频,将使用音色参考音频的情感。"
)
return ""
def _download_indextts2_audio(response: requests.Response, api_url: str, voice_file: str, proxies: dict) -> bool:
content_type = response.headers.get("content-type", "").lower()
if "application/json" not in content_type:
@ -2417,9 +2659,14 @@ def _download_indextts2_audio(response: requests.Response, api_url: str, voice_f
f.write(response.content)
return os.path.getsize(voice_file) > 0
result = response.json()
downloads = result.get("downloads") if isinstance(result, dict) else {}
download_url = downloads.get("wav") if isinstance(downloads, dict) else ""
try:
result = response.json()
except ValueError:
logger.error("IndexTTS-2 API 返回了无效的 JSON 响应")
return False
output = result.get("output") if isinstance(result, dict) else {}
download_url = output.get("url") if isinstance(output, dict) else ""
if not download_url:
logger.error(f"IndexTTS-2 API 响应中没有音频下载地址: {result}")
return False
@ -2437,68 +2684,74 @@ def _download_indextts2_audio(response: requests.Response, api_url: str, voice_f
def indextts2_tts(text: str, voice_name: str, voice_file: str) -> Union[SubMaker, None]:
"""
使用 IndexTTS-2 API 进行零样本语音克隆
接口兼容 IndexTTS2-Pack POST /tts multipart form
使用 IndexTTS-2 MLX Pack API 进行零样本语音克隆
参考音频通过 ``POST /v1/audio/speech/upload`` 上传这样 Pack 即使
运行在另一台机器上也不需要访问 NarratoAI 的本地文件路径
"""
api_url = _normalize_indextts2_api_url(config.indextts2.get("api_url", "http://192.168.3.6:7863/tts"))
api_url = _normalize_indextts2_api_url(config.indextts2.get("api_url", "http://127.0.0.1:7860"))
reference_audio_path = parse_indextts2_voice(voice_name)
if not reference_audio_path or not os.path.exists(reference_audio_path):
logger.error(f"IndexTTS-2 参考音频文件不存在: {reference_audio_path}")
return None
emotion_mode = config.indextts2.get("emotion_mode", "speaker")
emotion_audio_path = config.indextts2.get("emotion_audio", "")
data = {
"text": text.strip(),
"emotion_mode": emotion_mode,
"emotion_alpha": config.indextts2.get("emotion_alpha", 0.65),
"emotion_text": config.indextts2.get("emotion_text", ""),
"use_random": str(bool(config.indextts2.get("use_random", False))).lower(),
"max_text_tokens_per_segment": config.indextts2.get("max_text_tokens_per_segment", 120),
"vec_happy": config.indextts2.get("vec_happy", 0.0),
"vec_angry": config.indextts2.get("vec_angry", 0.0),
"vec_sad": config.indextts2.get("vec_sad", 0.0),
"vec_afraid": config.indextts2.get("vec_afraid", 0.0),
"vec_disgusted": config.indextts2.get("vec_disgusted", 0.0),
"vec_melancholic": config.indextts2.get("vec_melancholic", 0.0),
"vec_surprised": config.indextts2.get("vec_surprised", 0.0),
"vec_calm": config.indextts2.get("vec_calm", 0.8),
"temperature": config.indextts2.get("temperature", 0.8),
"top_p": config.indextts2.get("top_p", 0.8),
"top_k": config.indextts2.get("top_k", 30),
"num_beams": config.indextts2.get("num_beams", 3),
"repetition_penalty": config.indextts2.get("repetition_penalty", 10.0),
"max_mel_tokens": config.indextts2.get("max_mel_tokens", 1500),
"emo_alpha": _get_indextts2_number(
"emo_alpha", config.indextts2.get("emotion_alpha", 0.6), 0.0, 1.0
),
"speed": _get_indextts2_number("speed", 1.0, 0.5, 2.0),
"max_mel_tokens": _get_indextts2_number(
"max_mel_tokens", 1500, 64, 1815, integer=True
),
"max_text_tokens_per_segment": _get_indextts2_number(
"max_text_tokens_per_segment", 120, 20, 600, integer=True
),
"interval_silence": _get_indextts2_number(
"interval_silence", 200, 0, 5000, integer=True
),
"temperature": _get_indextts2_number("temperature", 0.8, 0.05, 2.0),
"top_p": _get_indextts2_number("top_p", 0.8, 0.05, 1.0),
"top_k": _get_indextts2_number("top_k", 30, 1, 200, integer=True),
"repetition_penalty": _get_indextts2_number(
"repetition_penalty", 10.0, 1.0, 30.0
),
"diffusion_steps": _get_indextts2_number(
"diffusion_steps", 25, 1, 100, integer=True
),
"cfg_rate": _get_indextts2_number("cfg_rate", 0.7, 0.0, 2.0),
"segment_overlap_ms": _get_indextts2_number(
"segment_overlap_ms", 50, 0, 1000, integer=True
),
}
emotion = _get_indextts2_emotion()
if emotion:
data["emotion"] = emotion
seed = _get_indextts2_seed()
if seed is not None:
data["seed"] = seed
proxies = _get_configured_proxies()
for attempt in range(3):
files = {}
try:
files["speaker_audio"] = open(reference_audio_path, "rb")
if emotion_mode == "audio":
if not emotion_audio_path or not os.path.exists(emotion_audio_path):
logger.error(f"IndexTTS-2 情感参考音频文件不存在: {emotion_audio_path}")
return None
files["emotion_audio"] = open(emotion_audio_path, "rb")
with open(reference_audio_path, "rb") as reference_audio:
logger.info(f"{attempt + 1} 次调用 IndexTTS-2 API: {api_url}")
response = requests.post(
api_url,
files={"reference_audio": reference_audio},
data=data,
proxies=proxies,
timeout=180,
)
logger.info(f"{attempt + 1} 次调用 IndexTTS-2 API: {api_url}")
response = requests.post(
api_url,
files=files,
data=data,
proxies=proxies,
timeout=180,
)
if response.status_code == 200 and _download_indextts2_audio(response, api_url, voice_file, proxies):
logger.info(f"IndexTTS-2 成功生成音频: {voice_file}, 大小: {os.path.getsize(voice_file)} 字节")
sub_maker = new_sub_maker()
duration = get_audio_duration_from_file(voice_file)
duration_ms = int(duration * 1000) if duration > 0 else max(1000, int(len(text) * 200))
add_subtitle_event(sub_maker, 0, duration_ms * 10000, text)
return sub_maker
if response.status_code == 200 and _download_indextts2_audio(response, api_url, voice_file, proxies):
logger.info(f"IndexTTS-2 成功生成音频: {voice_file}, 大小: {os.path.getsize(voice_file)} 字节")
sub_maker = new_sub_maker()
duration = get_audio_duration_from_file(voice_file)
duration_ms = int(duration * 1000) if duration > 0 else max(1000, int(len(text) * 200))
add_subtitle_event(sub_maker, 0, duration_ms * 10000, text)
return sub_maker
logger.error(f"IndexTTS-2 API 调用失败: {response.status_code} - {response.text}")
except requests.exceptions.Timeout:
@ -2507,12 +2760,6 @@ def indextts2_tts(text: str, voice_name: str, voice_file: str) -> Union[SubMaker
logger.error(f"IndexTTS-2 API 网络错误: {str(e)} (尝试 {attempt + 1}/3)")
except Exception as e:
logger.error(f"IndexTTS-2 TTS 处理错误: {str(e)} (尝试 {attempt + 1}/3)")
finally:
for file_obj in files.values():
try:
file_obj.close()
except Exception:
pass
if attempt < 2:
time.sleep(2)
@ -2521,6 +2768,144 @@ def indextts2_tts(text: str, voice_name: str, voice_file: str) -> Union[SubMaker
return None
def _normalize_voxcpm_api_url(api_url: str) -> str:
api_url = (api_url or "http://127.0.0.1:7864").strip().rstrip("/")
if api_url.endswith("/v1/audio/speech"):
api_url = api_url[:-len("/v1/audio/speech")]
if api_url.endswith("/tts"):
return api_url
return f"{api_url}/tts"
def voxcpm_tts(text: str, voice_name: str, voice_file: str) -> Union[SubMaker, None]:
"""使用 VoxCPM-0.5B-Pack 的 multipart /tts 接口生成 WAV。"""
voxcpm_config = getattr(config, "voxcpm_05b", {}) or {}
api_url = _normalize_voxcpm_api_url(voxcpm_config.get("api_url", "http://127.0.0.1:7864"))
reference_audio = parse_voxcpm_voice(voice_name)
if reference_audio in ("", "default") or not os.path.isfile(reference_audio):
reference_audio = voxcpm_config.get("reference_audio", "") or ""
if reference_audio and not os.path.isfile(reference_audio):
logger.error(f"VoxCPM-0.5B 参考音频文件不存在: {reference_audio}")
return None
data = {"text": text.strip()}
optional_fields = {
"prompt_text": voxcpm_config.get("prompt_text"),
"cfg_value": voxcpm_config.get("cfg_value"),
"inference_timesteps": voxcpm_config.get("inference_timesteps"),
"max_length": voxcpm_config.get("max_length"),
}
for key, value in optional_fields.items():
if value not in (None, ""):
data[key] = value
for key in ("normalize", "denoise"):
if key in voxcpm_config:
data[key] = str(bool(voxcpm_config[key])).lower()
proxies = _get_configured_proxies()
for attempt in range(3):
files = {}
try:
if reference_audio:
files["prompt_audio"] = open(reference_audio, "rb")
response = requests.post(api_url, data=data, files=files or None, proxies=proxies, timeout=300)
if response.status_code == 200:
result = response.json()
audio_url = result.get("audio_url", "") if isinstance(result, dict) else ""
if audio_url:
audio_response = requests.get(urljoin(api_url, audio_url), proxies=proxies, timeout=180)
if audio_response.status_code == 200:
with open(voice_file, "wb") as output:
output.write(audio_response.content)
if os.path.getsize(voice_file) > 0:
sub_maker = new_sub_maker()
duration = get_audio_duration_from_file(voice_file)
duration_ms = int(duration * 1000) if duration > 0 else max(1000, len(text) * 200)
add_subtitle_event(sub_maker, 0, duration_ms * 10000, text)
return sub_maker
logger.error(f"VoxCPM-0.5B API 响应中没有有效音频地址: {result}")
else:
logger.error(f"VoxCPM-0.5B API 调用失败: {response.status_code} - {response.text}")
except (ValueError, requests.exceptions.RequestException) as exc:
logger.error(f"VoxCPM-0.5B TTS 请求失败 (尝试 {attempt + 1}/3): {exc}")
finally:
for file_obj in files.values():
file_obj.close()
if attempt < 2:
time.sleep(2)
return None
def _normalize_voxcpm2_api_url(api_url: str) -> str:
api_url = (api_url or "http://127.0.0.1:7863").strip().rstrip("/")
for suffix in ("/v1/audio/speech", "/tts/batch"):
if api_url.endswith(suffix):
api_url = api_url[:-len(suffix)]
return api_url if api_url.endswith("/tts") else f"{api_url}/tts"
def voxcpm2_tts(text: str, voice_name: str, voice_file: str) -> Union[SubMaker, None]:
"""使用 VoxCPM-2B-Pack 的 /tts 接口进行音色设计或参考音频克隆。"""
pack_config = getattr(config, "voxcpm_2b", {}) or {}
api_url = _normalize_voxcpm2_api_url(pack_config.get("api_url", "http://127.0.0.1:7863"))
mode = str(pack_config.get("mode", "design"))
parsed_voice = parse_voxcpm2_voice(voice_name)
reference_audio = ""
if mode == "clone":
reference_audio = parsed_voice if parsed_voice and os.path.isfile(parsed_voice) else pack_config.get("reference_audio", "")
if not reference_audio or not os.path.isfile(reference_audio):
logger.error(f"VoxCPM-2B 参考音频文件不存在: {reference_audio}")
return None
data = {"text": text.strip()}
optional_fields = {
"control": pack_config.get("control"),
"prompt_text": pack_config.get("prompt_text") if mode == "clone" else None,
"cfg_value": pack_config.get("cfg_value"),
"inference_timesteps": pack_config.get("inference_timesteps"),
}
for key, value in optional_fields.items():
if value not in (None, ""):
data[key] = value
for key in ("normalize", "denoise", "output_48k", "context_aware", "streaming"):
if key in pack_config:
data[key] = str(bool(pack_config[key])).lower()
proxies = _get_configured_proxies()
for attempt in range(3):
files = {}
try:
if reference_audio:
files["reference_audio"] = open(reference_audio, "rb")
response = requests.post(api_url, data=data, files=files or None, proxies=proxies, timeout=600)
if response.status_code == 200:
result = response.json()
downloads = result.get("downloads", {}) if isinstance(result, dict) else {}
audio_url = downloads.get("wav", "") if isinstance(downloads, dict) else ""
if audio_url:
audio_response = requests.get(urljoin(api_url, audio_url), proxies=proxies, timeout=180)
if audio_response.status_code == 200:
with open(voice_file, "wb") as output:
output.write(audio_response.content)
if os.path.getsize(voice_file) > 0:
sub_maker = new_sub_maker()
duration = get_audio_duration_from_file(voice_file)
duration_ms = int(duration * 1000) if duration > 0 else max(1000, len(text) * 200)
add_subtitle_event(sub_maker, 0, duration_ms * 10000, text)
return sub_maker
logger.error(f"VoxCPM-2B API 响应中没有有效 WAV 下载地址: {result}")
else:
logger.error(f"VoxCPM-2B API 调用失败: {response.status_code} - {response.text}")
except (ValueError, requests.exceptions.RequestException) as exc:
logger.error(f"VoxCPM-2B TTS 请求失败 (尝试 {attempt + 1}/3): {exc}")
finally:
for file_obj in files.values():
file_obj.close()
if attempt < 2:
time.sleep(2)
return None
def _normalize_omnivoice_api_url(api_url: str) -> str:
api_url = (api_url or "http://127.0.0.1:7866/tts").strip()
if api_url.endswith("/tts"):

View File

@ -53,7 +53,8 @@
# - Qwen: qwen/qwen-plus, qwen/qwen-turbo
# - SiliconFlow: siliconflow/deepseek-ai/DeepSeek-R1
# - Moonshot: moonshot/moonshot-v1-8k
text_openai_model_name = "Pro/zai-org/GLM-5"
text_openai_model_name = "Pro/zai-org/GLM-5" # 高推理模型:剧情分析、文案生成、脚本匹配
text_openai_fast_model_name = "" # 高效率模型:字幕翻译、字幕校准;留空时回退到高推理模型
text_openai_api_key = "" # 填入对应 provider 的 API key
text_openai_base_url = "https://api.siliconflow.cn/v1" # 可选:自定义 API base URL界面会提示 API key 将发送到对应端点
text_openai_temperature = 1.0
@ -67,6 +68,20 @@
tavily_search_depth = "basic" # basic / advanced / fast / ultra-fast
tavily_max_results = 5
# ===== 可选Sonilo AI 配乐BGM/ AI 音效SFX=====
# 配乐:在 WebUI 背景音乐来源中选择 "AI 生成配乐Sonilo" 即可启用(默认关闭,不影响现有 BGM 逻辑)。
# 启用后会将合成完成的视频(未加 BGM上传到 Sonilo API根据画面内容与剪辑节奏生成配乐
# 生成的音乐已获授权、可商用(以条款为准)。视频时长上限 6 分钟,生成失败时自动回退到随机背景音乐。
# 音效:在 WebUI 音频设置中勾选 "AI 音效Sonilo" 即可启用(默认关闭)。
# 启用后会将合成完成的视频上传到 Sonilo API根据画面内容生成音效并混在现有音轨之下解说不受影响
# 生成的音效为免版税素材。视频时长上限 3 分钟,生成失败时自动跳过音效。
sonilo_api_key = "" # 获取地址https://sonilo.com配乐与音效共用
# sonilo_base_url = "https://api.sonilo.com"
# sonilo_timeout_seconds = 600 # 生成接口读超时 / 音效任务等待上限(秒)
# sonilo_bgm_prompt = "" # 可选:配乐风格提示,留空则完全根据画面生成
# sonilo_sfx_prompt = "" # 可选:音效风格提示,留空则完全根据画面生成
# sonilo_sfx_volume = 0.6 # 音效混入原声之下的音量,范围 (0, 2],默认 0.6
# ===== API Keys 参考 =====
# 主流 LLM Providers API Key 获取地址:
#
@ -160,40 +175,51 @@
num_beams = 3
repetition_penalty = 10.0
[indextts_macos]
# IndexTTS-1.5 macOS MLX Pack 语音克隆配置(仅适用于 Apple Silicon
# 参考音频会通过 POST /v1/audio/speech/upload 上传
api_url = "http://127.0.0.1:7866"
reference_audio_source = "resource"
# reference_audio = "/path/to/reference_audio.wav"
speed = 1.0
# seed = 42
max_mel_tokens = 800
max_text_tokens_per_segment = 120
interval_silence = 200
temperature = 1.0
top_p = 0.8
top_k = 30
repetition_penalty = 10.0
segment_overlap_ms = 50
[indextts2]
# IndexTTS-2 语音克隆配置
# 支持 IndexTTS2-Pack FastAPI 接口POST /tts
api_url = "http://192.168.3.6:7863/tts"
# IndexTTS-2 MLX Pack 语音克隆配置
# 参考音频会通过 POST /v1/audio/speech/upload 上传;可填写服务根地址或完整接口地址
api_url = "http://127.0.0.1:7860"
# 默认参考音频(可选),音色列表复用 IndexTTS-1.5 的资源目录
reference_audio_source = "resource"
# reference_audio = "/path/to/reference_audio.wav"
# 情感控制speaker / audio / vector / text
emotion_mode = "speaker"
emotion_audio = ""
emotion_alpha = 0.65
emotion_text = ""
use_random = false
max_text_tokens_per_segment = 120
# 8 维情感向量顺序happy, angry, sad, afraid, disgusted, melancholic, surprised, calm
vec_happy = 0.0
vec_angry = 0.0
vec_sad = 0.0
vec_afraid = 0.0
vec_disgusted = 0.0
vec_melancholic = 0.0
vec_surprised = 0.0
vec_calm = 0.8
# 留空时保留参考音频情绪;支持单个情绪或混合权重,例如 happy:0.7,calm:0.3
emotion = ""
emo_alpha = 0.6
speed = 1.0
# 可选:设置固定随机种子以复现生成结果
# seed = 20260713
# 高级生成参数
max_text_tokens_per_segment = 120
interval_silence = 200
temperature = 0.8
top_p = 0.8
top_k = 30
num_beams = 3
repetition_penalty = 10.0
max_mel_tokens = 1500
diffusion_steps = 25
cfg_rate = 0.7
segment_overlap_ms = 50
[omnivoice]
# OmniVoice-Pack 语音合成配置
@ -219,6 +245,35 @@
postprocess_output = true
preprocess_prompt = true
[voxcpm_05b]
# VoxCPM-0.5B-Pack 本地语音合成/参考音频克隆配置
# 启动整合包后,可填写服务根地址或完整 /tts 地址
api_url = "http://127.0.0.1:7864"
reference_audio_source = "resource"
reference_audio = ""
prompt_text = ""
cfg_value = 2.0
inference_timesteps = 10
max_length = 4096
normalize = true
denoise = false
[voxcpm_2b]
# VoxCPM-2B-Pack 本地语音合成、音色设计与参考音频克隆
api_url = "http://127.0.0.1:7863"
mode = "design"
control = ""
reference_audio_source = "resource"
reference_audio = ""
prompt_text = ""
cfg_value = 2.0
inference_timesteps = 10
normalize = true
denoise = false
output_48k = true
context_aware = true
streaming = false
[doubaotts]
# 豆包语音 TTS 配置
# 新版配置优先填写 API Key旧版 appid/token 配置仍兼容

View File

@ -1 +1 @@
0.8.5
0.8.7

View File

@ -0,0 +1,163 @@
import base64
import json
import os
import tempfile
import unittest
from unittest import mock
from app.services import sonilo
def _ndjson_lines(*events):
return [json.dumps(event) for event in events]
class ConsumeNdjsonStreamTests(unittest.TestCase):
def test_returns_first_stream_audio_grouped_by_index(self):
lines = _ndjson_lines(
{"type": "audio_chunk", "stream_index": 0, "data": base64.b64encode(b"ab").decode()},
{"type": "audio_chunk", "stream_index": 1, "data": base64.b64encode(b"zz").decode()},
{"type": "audio_chunk", "stream_index": 0, "data": base64.b64encode(b"cd").decode()},
{"type": "title", "data": "some title"},
{"type": "complete"},
)
self.assertEqual(b"abcd", sonilo._consume_ndjson_stream(lines))
def test_ignores_blank_and_unparseable_lines(self):
lines = [
"",
" ",
"not json",
json.dumps({"type": "audio_chunk", "stream_index": 0, "data": base64.b64encode(b"ok").decode()}),
json.dumps({"type": "complete"}),
]
self.assertEqual(b"ok", sonilo._consume_ndjson_stream(lines))
def test_error_event_raises(self):
lines = _ndjson_lines(
{"type": "audio_chunk", "stream_index": 0, "data": base64.b64encode(b"ab").decode()},
{"type": "error", "message": "boom"},
)
with self.assertRaises(sonilo.SoniloError):
sonilo._consume_ndjson_stream(lines)
def test_missing_complete_event_raises(self):
lines = _ndjson_lines(
{"type": "audio_chunk", "stream_index": 0, "data": base64.b64encode(b"ab").decode()},
)
with self.assertRaises(sonilo.SoniloError):
sonilo._consume_ndjson_stream(lines)
def test_complete_without_audio_raises(self):
lines = _ndjson_lines({"type": "complete"})
with self.assertRaises(sonilo.SoniloError):
sonilo._consume_ndjson_stream(lines)
class GenerateBgmTests(unittest.TestCase):
def setUp(self):
self._tmp_dir = tempfile.TemporaryDirectory()
self.addCleanup(self._tmp_dir.cleanup)
self.video_path = os.path.join(self._tmp_dir.name, "combined.mp4")
with open(self.video_path, "wb") as f:
f.write(b"fake video")
self.save_path = os.path.join(self._tmp_dir.name, "sonilo_bgm.m4a")
def test_returns_empty_without_api_key(self):
with mock.patch.object(sonilo, "get_api_key", return_value=""):
self.assertEqual("", sonilo.generate_bgm(self.video_path, self.save_path))
def test_returns_empty_when_video_missing(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"):
missing = os.path.join(self._tmp_dir.name, "missing.mp4")
self.assertEqual("", sonilo.generate_bgm(missing, self.save_path))
def test_skips_upload_when_duration_exceeds_limit(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_probe_video_duration", return_value=361.0), \
mock.patch.object(sonilo, "_request_video_to_music") as request_mock:
self.assertEqual("", sonilo.generate_bgm(self.video_path, self.save_path))
request_mock.assert_not_called()
def test_saves_audio_on_success(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_probe_video_duration", return_value=60.0), \
mock.patch.object(sonilo, "_request_video_to_music", return_value=b"audio-bytes"):
self.assertEqual(self.save_path, sonilo.generate_bgm(self.video_path, self.save_path))
with open(self.save_path, "rb") as f:
self.assertEqual(b"audio-bytes", f.read())
def test_request_failure_degrades_to_empty(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_probe_video_duration", return_value=60.0), \
mock.patch.object(
sonilo,
"_request_video_to_music",
side_effect=sonilo.SoniloError("timeout"),
):
self.assertEqual("", sonilo.generate_bgm(self.video_path, self.save_path))
self.assertFalse(os.path.exists(self.save_path))
class ResolveBgmPathTests(unittest.TestCase):
"""任务层的 BGM 解析Sonilo 模式失败时回退到现有 BGM 逻辑。"""
def _make_params(self, bgm_type, bgm_file=""):
params = mock.Mock()
params.bgm_type = bgm_type
params.bgm_file = bgm_file
return params
def test_non_sonilo_type_uses_existing_logic(self):
from app.services import task
params = self._make_params("custom", "/tmp/some_bgm.mp3")
with mock.patch.object(task.utils, "get_bgm_file", return_value="/tmp/some_bgm.mp3") as get_bgm, \
mock.patch.object(task.sonilo, "generate_bgm") as generate_bgm:
result = task._resolve_bgm_path("task-id", params, "/tmp/combined.mp4")
self.assertEqual("/tmp/some_bgm.mp3", result)
get_bgm.assert_called_once_with(bgm_type="custom", bgm_file="/tmp/some_bgm.mp3")
generate_bgm.assert_not_called()
def test_sonilo_type_uses_generated_bgm(self):
from app.services import task
params = self._make_params("sonilo")
with mock.patch.object(task.utils, "task_dir", return_value="/tmp/task-id"), \
mock.patch.object(
task.sonilo, "generate_bgm", return_value="/tmp/task-id/sonilo_bgm.m4a"
) as generate_bgm, \
mock.patch.object(task.utils, "get_bgm_file") as get_bgm:
result = task._resolve_bgm_path("task-id", params, "/tmp/combined.mp4")
self.assertEqual("/tmp/task-id/sonilo_bgm.m4a", result)
generate_bgm.assert_called_once_with(
"/tmp/combined.mp4", os.path.join("/tmp/task-id", "sonilo_bgm.m4a")
)
get_bgm.assert_not_called()
def test_sonilo_failure_falls_back_to_random_bgm(self):
from app.services import task
params = self._make_params("sonilo")
with mock.patch.object(task.utils, "task_dir", return_value="/tmp/task-id"), \
mock.patch.object(task.sonilo, "generate_bgm", return_value=""), \
mock.patch.object(
task.utils, "get_bgm_file", return_value="/resource/songs/output000.mp3"
) as get_bgm:
result = task._resolve_bgm_path("task-id", params, "/tmp/combined.mp4")
self.assertEqual("/resource/songs/output000.mp3", result)
get_bgm.assert_called_once_with(bgm_type="random", bgm_file="")
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,362 @@
import os
import tempfile
import unittest
from unittest import mock
import requests
from app.services import sonilo
def _response(status_code=200, json_body=None, content=b"{}"):
resp = mock.Mock()
resp.status_code = status_code
resp.content = content
if json_body is None:
resp.json.side_effect = ValueError("no json")
else:
resp.json.return_value = json_body
return resp
class SubmitSfxTaskTests(unittest.TestCase):
def setUp(self):
self._tmp_dir = tempfile.TemporaryDirectory()
self.addCleanup(self._tmp_dir.cleanup)
self.video_path = os.path.join(self._tmp_dir.name, "combined.mp4")
with open(self.video_path, "wb") as f:
f.write(b"fake video")
def test_returns_task_id(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(
sonilo.requests,
"post",
return_value=_response(202, {"task_id": "task-123"}),
) as post_mock:
self.assertEqual("task-123", sonilo._submit_sfx_task(self.video_path))
args, kwargs = post_mock.call_args
self.assertTrue(args[0].endswith("/v1/video-to-sfx"))
self.assertEqual("Bearer sk-test", kwargs["headers"]["Authorization"])
def test_http_error_raises(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(
sonilo.requests,
"post",
return_value=_response(402, content=b'{"detail": "no credits"}'),
):
with self.assertRaises(sonilo.SoniloError):
sonilo._submit_sfx_task(self.video_path)
def test_missing_task_id_raises(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(
sonilo.requests, "post", return_value=_response(202, {})
):
with self.assertRaises(sonilo.SoniloError):
sonilo._submit_sfx_task(self.video_path)
class PollSfxTaskTests(unittest.TestCase):
def test_returns_body_when_succeeded(self):
responses = [
_response(200, {"status": "processing"}),
_response(
200,
{"status": "succeeded", "audio": {"url": "https://cdn/x.m4a"}},
),
]
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_sleep"), \
mock.patch.object(sonilo.requests, "get", side_effect=responses):
body = sonilo._poll_sfx_task("task-123")
self.assertEqual("succeeded", body["status"])
def test_failed_status_raises_with_task_id(self):
response = _response(
200,
{
"status": "failed",
"error": {"code": "GENERATION_FAILED", "message": "boom"},
"refunded": True,
},
)
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_sleep"), \
mock.patch.object(sonilo.requests, "get", return_value=response):
with self.assertRaises(sonilo.SoniloError) as ctx:
sonilo._poll_sfx_task("task-123")
self.assertIn("task-123", str(ctx.exception))
self.assertIn("boom", str(ctx.exception))
def test_transient_error_retries_until_succeeded(self):
responses = [
requests.exceptions.ConnectionError("blip"),
_response(
200,
{"status": "succeeded", "audio": {"url": "https://cdn/x.m4a"}},
),
]
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_sleep"), \
mock.patch.object(sonilo.requests, "get", side_effect=responses):
body = sonilo._poll_sfx_task("task-123")
self.assertEqual("succeeded", body["status"])
def test_non_recoverable_http_error_raises(self):
response = _response(401, content=b'{"detail": "bad key"}')
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_sleep"), \
mock.patch.object(sonilo.requests, "get", return_value=response):
with self.assertRaises(sonilo.SoniloError):
sonilo._poll_sfx_task("task-123")
def test_timeout_raises_with_task_id(self):
response = _response(200, {"status": "processing"})
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_sleep"), \
mock.patch.object(sonilo, "_get_timeout_seconds", return_value=0.0), \
mock.patch.object(sonilo.requests, "get", return_value=response):
with self.assertRaises(sonilo.SoniloError) as ctx:
sonilo._poll_sfx_task("task-123")
self.assertIn("task-123", str(ctx.exception))
class DownloadSfxAudioTests(unittest.TestCase):
def test_returns_content_without_auth_headers(self):
response = _response(200, content=b"audio-bytes")
with mock.patch.object(
sonilo.requests, "get", return_value=response
) as get_mock:
self.assertEqual(
b"audio-bytes", sonilo._download_sfx_audio("https://cdn/x.m4a")
)
# 预签名 URL 自带鉴权,绝不能把 API Key 发给存储域名。
_, kwargs = get_mock.call_args
self.assertNotIn("headers", kwargs)
def test_http_error_raises(self):
with mock.patch.object(
sonilo.requests, "get", return_value=_response(403, content=b"denied")
):
with self.assertRaises(sonilo.SoniloError):
sonilo._download_sfx_audio("https://cdn/x.m4a")
def test_empty_content_raises(self):
with mock.patch.object(
sonilo.requests, "get", return_value=_response(200, content=b"")
):
with self.assertRaises(sonilo.SoniloError):
sonilo._download_sfx_audio("https://cdn/x.m4a")
class ExtractSfxAudioUrlTests(unittest.TestCase):
def test_returns_audio_url(self):
body = {"status": "succeeded", "audio": {"url": "https://cdn/x.m4a"}}
self.assertEqual(
"https://cdn/x.m4a", sonilo._extract_sfx_audio_url(body, "task-123")
)
def test_missing_audio_raises(self):
with self.assertRaises(sonilo.SoniloError):
sonilo._extract_sfx_audio_url({"status": "succeeded"}, "task-123")
class GenerateSfxTests(unittest.TestCase):
def setUp(self):
self._tmp_dir = tempfile.TemporaryDirectory()
self.addCleanup(self._tmp_dir.cleanup)
self.video_path = os.path.join(self._tmp_dir.name, "combined.mp4")
with open(self.video_path, "wb") as f:
f.write(b"fake video")
self.save_path = os.path.join(self._tmp_dir.name, "sonilo_sfx.m4a")
def test_returns_empty_without_api_key(self):
with mock.patch.object(sonilo, "get_api_key", return_value=""):
self.assertEqual("", sonilo.generate_sfx(self.video_path, self.save_path))
def test_returns_empty_when_video_missing(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"):
missing = os.path.join(self._tmp_dir.name, "missing.mp4")
self.assertEqual("", sonilo.generate_sfx(missing, self.save_path))
def test_skips_upload_when_duration_exceeds_limit(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_probe_video_duration", return_value=181.0), \
mock.patch.object(sonilo, "_request_video_to_sfx") as request_mock:
self.assertEqual("", sonilo.generate_sfx(self.video_path, self.save_path))
request_mock.assert_not_called()
def test_saves_audio_on_success(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_probe_video_duration", return_value=60.0), \
mock.patch.object(sonilo, "_request_video_to_sfx", return_value=b"audio-bytes"):
self.assertEqual(self.save_path, sonilo.generate_sfx(self.video_path, self.save_path))
with open(self.save_path, "rb") as f:
self.assertEqual(b"audio-bytes", f.read())
def test_request_failure_degrades_to_empty(self):
with mock.patch.object(sonilo, "get_api_key", return_value="sk-test"), \
mock.patch.object(sonilo, "_probe_video_duration", return_value=60.0), \
mock.patch.object(
sonilo,
"_request_video_to_sfx",
side_effect=sonilo.SoniloError("timeout"),
):
self.assertEqual("", sonilo.generate_sfx(self.video_path, self.save_path))
self.assertFalse(os.path.exists(self.save_path))
class GetSfxVolumeTests(unittest.TestCase):
def test_default_when_unset(self):
with mock.patch.object(sonilo.config, "app", {}):
self.assertEqual(sonilo.DEFAULT_SFX_VOLUME, sonilo._get_sfx_volume())
def test_default_when_invalid(self):
with mock.patch.object(sonilo.config, "app", {"sonilo_sfx_volume": "abc"}):
self.assertEqual(sonilo.DEFAULT_SFX_VOLUME, sonilo._get_sfx_volume())
def test_default_when_non_positive(self):
with mock.patch.object(sonilo.config, "app", {"sonilo_sfx_volume": 0}):
self.assertEqual(sonilo.DEFAULT_SFX_VOLUME, sonilo._get_sfx_volume())
def test_clamped_to_upper_bound(self):
with mock.patch.object(sonilo.config, "app", {"sonilo_sfx_volume": 5}):
self.assertEqual(2.0, sonilo._get_sfx_volume())
def test_valid_value_passes_through(self):
with mock.patch.object(sonilo.config, "app", {"sonilo_sfx_volume": 0.8}):
self.assertEqual(0.8, sonilo._get_sfx_volume())
class MixSfxUnderOriginalTests(unittest.TestCase):
def _run(self, has_audio, returncode=0, run_side_effect=None):
run_mock = mock.Mock(return_value=mock.Mock(returncode=returncode, stderr=b"err"))
if run_side_effect is not None:
run_mock = mock.Mock(side_effect=run_side_effect)
with mock.patch.object(sonilo, "_get_ffmpeg_binary", return_value="ffmpeg"), \
mock.patch.object(sonilo, "_probe_has_audio_stream", return_value=has_audio), \
mock.patch.object(sonilo, "_get_sfx_volume", return_value=0.6), \
mock.patch.object(sonilo.subprocess, "run", run_mock):
result = sonilo._mix_sfx_under_original(
"/tmp/combined.mp4", "/tmp/sfx.m4a", "/tmp/merger_sfx.mp4"
)
return result, run_mock
def test_mixes_under_existing_audio_with_amix(self):
result, run_mock = self._run(has_audio=True)
self.assertEqual("/tmp/merger_sfx.mp4", result)
cmd = run_mock.call_args[0][0]
filter_complex = cmd[cmd.index("-filter_complex") + 1]
self.assertIn("volume=0.6", filter_complex)
self.assertIn("amix", filter_complex)
# 视频流直接复制,不重编码画面。
self.assertIn("copy", cmd[cmd.index("-c:v") + 1])
self.assertNotIn("-shortest", cmd)
def test_sfx_becomes_audio_track_when_video_has_no_audio(self):
result, run_mock = self._run(has_audio=False)
self.assertEqual("/tmp/merger_sfx.mp4", result)
cmd = run_mock.call_args[0][0]
filter_complex = cmd[cmd.index("-filter_complex") + 1]
self.assertNotIn("amix", filter_complex)
self.assertIn("-shortest", cmd)
def test_ffmpeg_failure_returns_empty(self):
result, _ = self._run(has_audio=True, returncode=1)
self.assertEqual("", result)
def test_ffmpeg_oserror_returns_empty(self):
result, _ = self._run(has_audio=True, run_side_effect=OSError("no ffmpeg"))
self.assertEqual("", result)
class ApplySfxTests(unittest.TestCase):
def test_success_returns_output_path(self):
with mock.patch.object(
sonilo, "generate_sfx", return_value="/tmp/merger_sfx.m4a"
) as generate_mock, mock.patch.object(
sonilo, "_mix_sfx_under_original", return_value="/tmp/merger_sfx.mp4"
) as mix_mock:
result = sonilo.apply_sfx("/tmp/combined.mp4", "/tmp/merger_sfx.mp4")
self.assertEqual("/tmp/merger_sfx.mp4", result)
generate_mock.assert_called_once_with("/tmp/combined.mp4", "/tmp/merger_sfx.m4a")
mix_mock.assert_called_once_with(
"/tmp/combined.mp4", "/tmp/merger_sfx.m4a", "/tmp/merger_sfx.mp4"
)
def test_generation_failure_skips_mixing(self):
with mock.patch.object(sonilo, "generate_sfx", return_value=""), \
mock.patch.object(sonilo, "_mix_sfx_under_original") as mix_mock:
self.assertEqual(
"", sonilo.apply_sfx("/tmp/combined.mp4", "/tmp/merger_sfx.mp4")
)
mix_mock.assert_not_called()
def test_mixing_failure_returns_empty(self):
with mock.patch.object(
sonilo, "generate_sfx", return_value="/tmp/merger_sfx.m4a"
), mock.patch.object(sonilo, "_mix_sfx_under_original", return_value=""):
self.assertEqual(
"", sonilo.apply_sfx("/tmp/combined.mp4", "/tmp/merger_sfx.mp4")
)
class ApplySoniloSfxTaskTests(unittest.TestCase):
"""任务层的音效挂载:默认关闭,失败时沿用原视频,绝不中断成片任务。"""
def _make_params(self, sfx_enabled):
params = mock.Mock()
params.sonilo_sfx_enabled = sfx_enabled
return params
def test_disabled_returns_original_path_without_calling_sonilo(self):
from app.services import task
params = self._make_params(False)
with mock.patch.object(task.sonilo, "apply_sfx") as apply_mock:
result = task._apply_sonilo_sfx("task-id", params, "/tmp/merger.mp4")
self.assertEqual("/tmp/merger.mp4", result)
apply_mock.assert_not_called()
def test_enabled_returns_sfx_video_path(self):
from app.services import task
params = self._make_params(True)
with mock.patch.object(task.utils, "task_dir", return_value="/tmp/task-id"), \
mock.patch.object(
task.sonilo, "apply_sfx", return_value="/tmp/task-id/merger_sfx.mp4"
) as apply_mock:
result = task._apply_sonilo_sfx("task-id", params, "/tmp/merger.mp4")
self.assertEqual("/tmp/task-id/merger_sfx.mp4", result)
apply_mock.assert_called_once_with(
"/tmp/merger.mp4", os.path.join("/tmp/task-id", "merger_sfx.mp4")
)
def test_failure_falls_back_to_original_path(self):
from app.services import task
params = self._make_params(True)
with mock.patch.object(task.utils, "task_dir", return_value="/tmp/task-id"), \
mock.patch.object(task.sonilo, "apply_sfx", return_value=""):
result = task._apply_sonilo_sfx("task-id", params, "/tmp/merger.mp4")
self.assertEqual("/tmp/merger.mp4", result)
if __name__ == "__main__":
unittest.main()

View File

@ -3,8 +3,9 @@ import os
import shutil
import json
from uuid import uuid4
from loguru import logger
from app.config import config
from app.services import voice
from app.services import sonilo, voice
from app.models.schema import AudioVolumeDefaults
from app.utils import utils
@ -42,8 +43,11 @@ BGM_UPLOAD_SUBDIR = "uploaded_bgms"
BGM_AUDIO_EXTENSIONS = (".mp3", ".wav", ".flac", ".m4a", ".aac", ".ogg")
LOCAL_TTS_ENGINES = {
config.INDEXTTS_ENGINE,
config.INDEXTTS_MACOS_ENGINE,
config.INDEXTTS2_ENGINE,
config.OMNIVOICE_ENGINE,
config.VOXCPM_ENGINE,
config.VOXCPM2_ENGINE,
}
@ -74,8 +78,11 @@ def get_tts_engine_options(tr=lambda key: key):
"""获取TTS引擎选项"""
engine_options = {
config.INDEXTTS_ENGINE: config.INDEXTTS_DISPLAY_NAME,
config.INDEXTTS_MACOS_ENGINE: config.INDEXTTS_MACOS_DISPLAY_NAME,
config.INDEXTTS2_ENGINE: config.INDEXTTS2_DISPLAY_NAME,
config.OMNIVOICE_ENGINE: config.OMNIVOICE_DISPLAY_NAME,
config.VOXCPM_ENGINE: config.VOXCPM_DISPLAY_NAME,
config.VOXCPM2_ENGINE: config.VOXCPM2_DISPLAY_NAME,
"edge_tts": "Edge TTS",
"qwen3_tts": tr("Tongyi Qwen3 TTS"),
"tencent_tts": tr("Tencent Cloud TTS"),
@ -136,6 +143,12 @@ def get_tts_engine_descriptions(tr=lambda key: key):
"use_case": tr("IndexTTS use case"),
"registration": None
},
config.INDEXTTS_MACOS_ENGINE: {
"title": config.INDEXTTS_MACOS_DISPLAY_NAME,
"features": tr("IndexTTS macOS features"),
"use_case": tr("IndexTTS macOS use case"),
"registration": None
},
config.INDEXTTS2_ENGINE: {
"title": config.INDEXTTS2_DISPLAY_NAME,
"features": tr("IndexTTS2 features"),
@ -148,6 +161,18 @@ def get_tts_engine_descriptions(tr=lambda key: key):
"use_case": tr("OmniVoice use case"),
"registration": None
},
config.VOXCPM_ENGINE: {
"title": config.VOXCPM_DISPLAY_NAME,
"features": tr("VoxCPM features"),
"use_case": tr("VoxCPM use case"),
"registration": None,
},
config.VOXCPM2_ENGINE: {
"title": config.VOXCPM2_DISPLAY_NAME,
"features": tr("VoxCPM2 features"),
"use_case": tr("VoxCPM2 use case"),
"registration": None,
},
"doubaotts": {
"title": tr("Doubao TTS"),
"features": tr("Doubao TTS features"),
@ -524,6 +549,9 @@ def render_audio_panel(tr):
# 背景音乐独立成框,放在音频设置下方
render_bgm_panel(tr)
# AI 音效独立成框(可选功能,默认关闭)
render_sonilo_sfx_panel(tr)
def render_bgm_panel(tr):
"""渲染背景音乐设置面板"""
@ -531,6 +559,12 @@ def render_bgm_panel(tr):
render_bgm_settings(tr)
def render_sonilo_sfx_panel(tr):
"""渲染 Sonilo AI 音效设置面板(可选功能,默认关闭)"""
with st.container(border=True):
render_sonilo_sfx_settings(tr)
def render_tts_settings(tr):
"""渲染TTS(文本转语音)设置"""
@ -588,10 +622,16 @@ def render_tts_settings(tr):
render_qwen3_tts_settings(tr)
elif selected_engine == config.INDEXTTS_ENGINE:
render_indextts_tts_settings(tr)
elif selected_engine == config.INDEXTTS_MACOS_ENGINE:
render_indextts_macos_tts_settings(tr)
elif selected_engine == config.INDEXTTS2_ENGINE:
render_indextts2_tts_settings(tr)
elif selected_engine == config.OMNIVOICE_ENGINE:
render_omnivoice_tts_settings(tr)
elif selected_engine == config.VOXCPM_ENGINE:
render_voxcpm_tts_settings(tr)
elif selected_engine == config.VOXCPM2_ENGINE:
render_voxcpm2_tts_settings(tr)
elif selected_engine == "doubaotts":
render_doubaotts_settings(tr)
@ -1122,11 +1162,115 @@ def render_indextts_tts_settings(tr):
config.ui["voice_name"] = f"{config.INDEXTTS_VOICE_PREFIX}{reference_audio}"
def render_indextts2_tts_settings(tr):
"""渲染 IndexTTS-2 TTS 设置"""
def render_indextts_macos_tts_settings(tr):
"""渲染 IndexTTS-1.5 macOS MLX Pack 设置。"""
tts_config = config.indextts_macos
def bounded_value(key, default, min_value, max_value):
try:
value = float(tts_config.get(key, default))
except (TypeError, ValueError):
value = default
return max(min_value, min(max_value, value))
api_url = st.text_input(
tr("API URL"),
value=config.indextts2.get("api_url", "http://192.168.3.6:7863/tts"),
value=tts_config.get("api_url", "http://127.0.0.1:7866"),
help=tr("IndexTTS macOS API URL Help"),
)
reference_audio_source, reference_audio = render_indextts_reference_audio_selector(
tr,
tts_config,
"indextts_macos",
)
speed = st.slider(
tr("IndexTTS2 Speed"),
min_value=0.5,
max_value=2.0,
value=bounded_value("speed", 1.0, 0.5, 2.0),
step=0.05,
help=tr("IndexTTS2 Speed Help"),
)
seed = st.text_input(
tr("IndexTTS2 Seed"),
value=str(tts_config.get("seed", "") or ""),
help=tr("IndexTTS2 Seed Help"),
placeholder=tr("IndexTTS2 Seed Placeholder"),
)
with st.expander(tr("Advanced Parameters"), expanded=False):
col1, col2 = st.columns(2)
with col1:
temperature = st.slider(
tr("Sampling Temperature"), 0.0, 2.0,
bounded_value("temperature", 1.0, 0.0, 2.0), 0.05,
)
top_p = st.slider(
"Top P", 0.05, 1.0,
bounded_value("top_p", 0.8, 0.05, 1.0), 0.05,
)
top_k = st.slider(
"Top K", 0, 200,
int(bounded_value("top_k", 30, 0, 200)), 1,
)
max_text_tokens_per_segment = st.slider(
tr("Max Text Tokens Per Segment"), 20, 600,
int(bounded_value("max_text_tokens_per_segment", 120, 20, 600)), 10,
)
with col2:
repetition_penalty = st.slider(
tr("Repetition Penalty"), 1.0, 20.0,
bounded_value("repetition_penalty", 10.0, 1.0, 20.0), 0.1,
)
max_mel_tokens = st.slider(
tr("Max Mel Tokens"), 64, 1600,
int(bounded_value("max_mel_tokens", 800, 64, 1600)), 1,
)
interval_silence = st.slider(
tr("Interval Silence"), 0, 2000,
int(bounded_value("interval_silence", 200, 0, 2000)), 50,
)
segment_overlap_ms = st.slider(
tr("Segment Overlap"), 0, 500,
int(bounded_value("segment_overlap_ms", 50, 0, 500)), 10,
)
with st.expander(tr("IndexTTS macOS Usage Instructions Title"), expanded=False):
st.markdown(tr("IndexTTS macOS Usage Instructions"))
tts_config["api_url"] = api_url
tts_config["reference_audio_source"] = reference_audio_source
tts_config["reference_audio"] = reference_audio
tts_config["speed"] = speed
tts_config["seed"] = seed.strip()
tts_config["temperature"] = temperature
tts_config["top_p"] = top_p
tts_config["top_k"] = top_k
tts_config["max_text_tokens_per_segment"] = max_text_tokens_per_segment
tts_config["repetition_penalty"] = repetition_penalty
tts_config["max_mel_tokens"] = max_mel_tokens
tts_config["interval_silence"] = interval_silence
tts_config["segment_overlap_ms"] = segment_overlap_ms
if reference_audio:
config.ui["voice_name"] = f"{config.INDEXTTS_MACOS_VOICE_PREFIX}{reference_audio}"
st.session_state["voice_rate"] = 1.0
st.session_state["voice_pitch"] = 1.0
def render_indextts2_tts_settings(tr):
"""渲染 IndexTTS-2 MLX Pack TTS 设置"""
def bounded_value(key, default, min_value, max_value):
try:
value = float(config.indextts2.get(key, default))
except (TypeError, ValueError):
value = default
return max(min_value, min(max_value, value))
api_url = st.text_input(
tr("API URL"),
value=config.indextts2.get("api_url", "http://127.0.0.1:7860"),
help=tr("IndexTTS2 API URL Help")
)
@ -1135,108 +1279,43 @@ def render_indextts2_tts_settings(tr):
config.indextts2,
"indextts2",
)
emotion_mode_options = [
("speaker", tr("Emotion Mode Speaker")),
("audio", tr("Emotion Mode Audio")),
("vector", tr("Emotion Mode Vector")),
("text", tr("Emotion Mode Text")),
]
saved_emotion_mode = config.indextts2.get("emotion_mode", "speaker")
emotion_mode_values = [item[0] for item in emotion_mode_options]
if saved_emotion_mode not in emotion_mode_values:
saved_emotion_mode = "speaker"
initial_emotion = config.get_indextts2_pack_emotion(config.indextts2)
legacy_emotion_audio = (
config.indextts2.get("emotion_mode") == "audio"
and bool(config.indextts2.get("emotion_audio"))
)
with st.expander(tr("IndexTTS2 Emotion Parameters"), expanded=False):
emotion_mode = emotion_mode_options[st.selectbox(
tr("Emotion Mode"),
options=range(len(emotion_mode_options)),
index=emotion_mode_values.index(saved_emotion_mode),
format_func=lambda x: emotion_mode_options[x][1],
help=tr("Emotion Mode Help"),
)][0]
emotion_alpha = st.slider(
emotion = st.text_input(
tr("IndexTTS2 Emotion"),
value=initial_emotion,
help=tr("IndexTTS2 Emotion Help"),
placeholder=tr("IndexTTS2 Emotion Placeholder"),
)
if legacy_emotion_audio and not emotion.strip():
st.warning(tr("IndexTTS2 Emotion Audio Unsupported"))
emo_alpha = st.slider(
tr("Emotion Alpha"),
min_value=0.0,
max_value=1.0,
value=float(config.indextts2.get("emotion_alpha", 0.65)),
value=bounded_value("emo_alpha", config.indextts2.get("emotion_alpha", 0.6), 0.0, 1.0),
step=0.05,
help=tr("Emotion Alpha Help"),
)
emotion_audio = config.indextts2.get("emotion_audio", "")
emotion_text = config.indextts2.get("emotion_text", "")
if emotion_mode == "audio":
emotion_audio_col, emotion_preview_col = st.columns([5, 1])
with emotion_audio_col:
emotion_audio = st.text_input(
tr("Emotion Reference Audio Path"),
value=emotion_audio,
help=tr("Emotion Reference Audio Path Help"),
)
with emotion_preview_col:
render_reference_audio_preview_button(
emotion_audio,
"indextts2_emotion_audio_preview",
tr,
preview_state_key="indextts2_emotion_audio_preview_path",
)
preview_audio_path = st.session_state.get("indextts2_emotion_audio_preview_path", "")
if preview_audio_path == emotion_audio and os.path.isfile(preview_audio_path):
with open(preview_audio_path, "rb") as audio_file:
st.audio(audio_file.read(), format=get_audio_mime_type(preview_audio_path))
elif emotion_mode == "text":
emotion_text = st.text_input(
tr("Emotion Text"),
value=emotion_text,
help=tr("Emotion Text Help"),
placeholder=tr("Emotion Text Placeholder"),
)
use_random = st.checkbox(
tr("Use Random Emotion"),
value=bool(config.indextts2.get("use_random", False)),
help=tr("Use Random Emotion Help"),
speed = st.slider(
tr("IndexTTS2 Speed"),
min_value=0.5,
max_value=2.0,
value=bounded_value("speed", 1.0, 0.5, 2.0),
step=0.05,
help=tr("IndexTTS2 Speed Help"),
)
seed = st.text_input(
tr("IndexTTS2 Seed"),
value=str(config.indextts2.get("seed", "") or ""),
help=tr("IndexTTS2 Seed Help"),
placeholder=tr("IndexTTS2 Seed Placeholder"),
)
emotion_vector_defaults = {
"vec_happy": 0.0,
"vec_angry": 0.0,
"vec_sad": 0.0,
"vec_afraid": 0.0,
"vec_disgusted": 0.0,
"vec_melancholic": 0.0,
"vec_surprised": 0.0,
"vec_calm": 0.8,
}
emotion_vector_labels = {
"vec_happy": tr("Emotion Happy"),
"vec_angry": tr("Emotion Angry"),
"vec_sad": tr("Emotion Sad"),
"vec_afraid": tr("Emotion Afraid"),
"vec_disgusted": tr("Emotion Disgusted"),
"vec_melancholic": tr("Emotion Melancholic"),
"vec_surprised": tr("Emotion Surprised"),
"vec_calm": tr("Emotion Calm"),
}
emotion_vector_values = {}
if emotion_mode == "vector":
vec_cols = st.columns(2)
for index, (field, default_value) in enumerate(emotion_vector_defaults.items()):
with vec_cols[index % 2]:
emotion_vector_values[field] = st.slider(
emotion_vector_labels[field],
min_value=0.0,
max_value=1.0,
value=float(config.indextts2.get(field, default_value)),
step=0.05,
)
else:
emotion_vector_values = {
field: float(config.indextts2.get(field, default_value))
for field, default_value in emotion_vector_defaults.items()
}
with st.expander(tr("Advanced Parameters"), expanded=False):
col1, col2 = st.columns(2)
@ -1244,65 +1323,92 @@ def render_indextts2_tts_settings(tr):
with col1:
temperature = st.slider(
tr("Sampling Temperature"),
min_value=0.1,
min_value=0.05,
max_value=2.0,
value=float(config.indextts2.get("temperature", 0.8)),
step=0.1,
value=bounded_value("temperature", 0.8, 0.05, 2.0),
step=0.05,
help=tr("Sampling Temperature Help")
)
top_p = st.slider(
"Top P",
min_value=0.0,
min_value=0.05,
max_value=1.0,
value=float(config.indextts2.get("top_p", 0.8)),
value=bounded_value("top_p", 0.8, 0.05, 1.0),
step=0.05,
help=tr("Top P Help")
)
top_k = st.slider(
"Top K",
min_value=0,
max_value=100,
value=int(config.indextts2.get("top_k", 30)),
step=5,
help=tr("Top K Help")
min_value=1,
max_value=200,
value=int(bounded_value("top_k", 30, 1, 200)),
step=1,
help=tr("IndexTTS2 Top K Help")
)
max_text_tokens_per_segment = st.slider(
tr("Max Text Tokens Per Segment"),
min_value=20,
max_value=600,
value=int(config.indextts2.get("max_text_tokens_per_segment", 120)),
value=int(bounded_value("max_text_tokens_per_segment", 120, 20, 600)),
step=10,
help=tr("Max Text Tokens Per Segment Help")
)
interval_silence = st.slider(
tr("Interval Silence"),
min_value=0,
max_value=5000,
value=int(bounded_value("interval_silence", 200, 0, 5000)),
step=50,
help=tr("Interval Silence Help"),
)
segment_overlap_ms = st.slider(
tr("Segment Overlap"),
min_value=0,
max_value=1000,
value=int(bounded_value("segment_overlap_ms", 50, 0, 1000)),
step=10,
help=tr("Segment Overlap Help"),
)
with col2:
num_beams = st.slider(
tr("Num Beams"),
diffusion_steps = st.slider(
tr("Diffusion Steps"),
min_value=1,
max_value=10,
value=int(config.indextts2.get("num_beams", 3)),
max_value=100,
value=int(bounded_value("diffusion_steps", 25, 1, 100)),
step=1,
help=tr("Num Beams Help")
help=tr("Diffusion Steps Help")
)
cfg_rate = st.slider(
tr("CFG Rate"),
min_value=0.0,
max_value=2.0,
value=bounded_value("cfg_rate", 0.7, 0.0, 2.0),
step=0.05,
help=tr("CFG Rate Help"),
)
repetition_penalty = st.slider(
tr("Repetition Penalty"),
min_value=0.1,
max_value=20.0,
value=float(config.indextts2.get("repetition_penalty", 10.0)),
min_value=1.0,
max_value=30.0,
value=bounded_value("repetition_penalty", 10.0, 1.0, 30.0),
step=0.1,
help=tr("Repetition Penalty Help")
)
max_mel_tokens = st.slider(
tr("Max Mel Tokens"),
min_value=50,
min_value=64,
max_value=1815,
value=int(config.indextts2.get("max_mel_tokens", 1500)),
step=10,
value=int(bounded_value("max_mel_tokens", 1500, 64, 1815)),
step=1,
help=tr("Max Mel Tokens Help")
)
@ -1312,20 +1418,32 @@ def render_indextts2_tts_settings(tr):
config.indextts2["api_url"] = api_url
config.indextts2["reference_audio_source"] = reference_audio_source
config.indextts2["reference_audio"] = reference_audio
config.indextts2["emotion_mode"] = emotion_mode
config.indextts2["emotion_audio"] = emotion_audio
config.indextts2["emotion_alpha"] = emotion_alpha
config.indextts2["emotion_text"] = emotion_text
config.indextts2["use_random"] = use_random
config.indextts2["emotion"] = emotion
config.indextts2["emo_alpha"] = emo_alpha
config.indextts2["speed"] = speed
config.indextts2["seed"] = seed.strip()
config.indextts2["max_text_tokens_per_segment"] = max_text_tokens_per_segment
for field, value in emotion_vector_values.items():
config.indextts2[field] = value
config.indextts2["interval_silence"] = interval_silence
config.indextts2["segment_overlap_ms"] = segment_overlap_ms
config.indextts2["temperature"] = temperature
config.indextts2["top_p"] = top_p
config.indextts2["top_k"] = top_k
config.indextts2["num_beams"] = num_beams
config.indextts2["repetition_penalty"] = repetition_penalty
config.indextts2["max_mel_tokens"] = max_mel_tokens
config.indextts2["diffusion_steps"] = diffusion_steps
config.indextts2["cfg_rate"] = cfg_rate
legacy_fields = (
"emotion_mode", "emotion_audio", "emotion_alpha", "emotion_text", "use_random",
"num_beams", "vec_happy", "vec_angry", "vec_sad", "vec_afraid",
"vec_disgusted", "vec_melancholic", "vec_surprised", "vec_calm",
)
if legacy_emotion_audio and not emotion.strip():
legacy_fields = tuple(
field for field in legacy_fields if field not in {"emotion_mode", "emotion_audio"}
)
for field in legacy_fields:
config.indextts2.pop(field, None)
if reference_audio:
config.ui["voice_name"] = f"{config.INDEXTTS2_VOICE_PREFIX}{reference_audio}"
@ -1475,6 +1593,109 @@ def render_omnivoice_tts_settings(tr):
st.session_state["voice_pitch"] = 1.0
def render_voxcpm_tts_settings(tr):
"""渲染 VoxCPM-0.5B-Pack 设置。"""
pack_config = config.voxcpm_05b
api_url = st.text_input(
tr("API URL"),
value=pack_config.get("api_url", "http://127.0.0.1:7864"),
help=tr("VoxCPM API URL Help"),
)
use_reference = st.checkbox(
tr("VoxCPM Use Reference Audio"),
value=bool(pack_config.get("reference_audio", "")),
help=tr("VoxCPM Use Reference Audio Help"),
)
source = pack_config.get("reference_audio_source", "resource")
reference_audio = pack_config.get("reference_audio", "")
prompt_text = pack_config.get("prompt_text", "")
if use_reference:
source, reference_audio = render_indextts_reference_audio_selector(tr, pack_config, "voxcpm_05b")
prompt_text = st.text_area(
tr("VoxCPM Prompt Text"), value=prompt_text,
help=tr("VoxCPM Prompt Text Help"), height=90,
)
else:
reference_audio = ""
with st.expander(tr("Advanced Parameters"), expanded=False):
cfg_value = st.slider("CFG Value", 1.0, 3.0, float(pack_config.get("cfg_value", 2.0)), 0.1)
inference_timesteps = st.slider("Inference Timesteps", 1, 50, int(pack_config.get("inference_timesteps", 10)), 1)
max_length = st.number_input("Max Length", 128, 8192, int(pack_config.get("max_length", 4096)), 128)
normalize = st.checkbox(tr("VoxCPM Normalize"), value=bool(pack_config.get("normalize", True)))
denoise = st.checkbox(tr("VoxCPM Denoise"), value=bool(pack_config.get("denoise", False)))
with st.expander(tr("VoxCPM Usage Instructions Title"), expanded=False):
st.markdown(tr("VoxCPM Usage Instructions"))
pack_config.update({
"api_url": api_url, "reference_audio_source": source,
"reference_audio": reference_audio, "prompt_text": prompt_text,
"cfg_value": cfg_value, "inference_timesteps": inference_timesteps,
"max_length": max_length, "normalize": normalize, "denoise": denoise,
})
config.ui["voice_name"] = f"{config.VOXCPM_VOICE_PREFIX}{reference_audio or 'default'}"
st.session_state["voice_rate"] = 1.0
st.session_state["voice_pitch"] = 1.0
def render_voxcpm2_tts_settings(tr):
"""渲染 VoxCPM-2B-Pack 设置。"""
pack_config = config.voxcpm_2b
api_url = st.text_input(
tr("API URL"), value=pack_config.get("api_url", "http://127.0.0.1:7863"),
help=tr("VoxCPM2 API URL Help"),
)
mode_options = [("design", tr("VoxCPM2 Mode Design")), ("clone", tr("VoxCPM2 Mode Clone"))]
mode_values = [item[0] for item in mode_options]
saved_mode = pack_config.get("mode", "design")
if saved_mode not in mode_values:
saved_mode = "design"
mode = mode_options[st.selectbox(
tr("VoxCPM2 Generation Mode"), options=range(len(mode_options)),
index=mode_values.index(saved_mode), format_func=lambda index: mode_options[index][1],
help=tr("VoxCPM2 Generation Mode Help"),
)][0]
control = st.text_area(
tr("VoxCPM2 Voice Control"), value=pack_config.get("control", ""),
help=tr("VoxCPM2 Voice Control Help"), height=80,
)
source = pack_config.get("reference_audio_source", "resource")
reference_audio = pack_config.get("reference_audio", "")
prompt_text = pack_config.get("prompt_text", "")
if mode == "clone":
source, reference_audio = render_indextts_reference_audio_selector(tr, pack_config, "voxcpm_2b")
prompt_text = st.text_area(
tr("VoxCPM Prompt Text"), value=prompt_text,
help=tr("VoxCPM Prompt Text Help"), height=90,
)
else:
reference_audio = ""
with st.expander(tr("Advanced Parameters"), expanded=False):
cfg_value = st.slider("CFG Value", 1.0, 3.0, float(pack_config.get("cfg_value", 2.0)), 0.1, key="voxcpm2_cfg")
inference_timesteps = st.slider("Inference Timesteps", 1, 50, int(pack_config.get("inference_timesteps", 10)), 1, key="voxcpm2_steps")
normalize = st.checkbox(tr("VoxCPM Normalize"), value=bool(pack_config.get("normalize", True)), key="voxcpm2_normalize")
denoise = st.checkbox(tr("VoxCPM Denoise"), value=bool(pack_config.get("denoise", False)), key="voxcpm2_denoise")
output_48k = st.checkbox(tr("VoxCPM2 Output 48k"), value=bool(pack_config.get("output_48k", True)))
context_aware = st.checkbox(tr("VoxCPM2 Context Aware"), value=bool(pack_config.get("context_aware", True)))
streaming = st.checkbox(tr("VoxCPM2 Streaming"), value=bool(pack_config.get("streaming", False)))
with st.expander(tr("VoxCPM2 Usage Instructions Title"), expanded=False):
st.markdown(tr("VoxCPM2 Usage Instructions"))
pack_config.update({
"api_url": api_url, "mode": mode, "control": control,
"reference_audio_source": source, "reference_audio": reference_audio,
"prompt_text": prompt_text, "cfg_value": cfg_value,
"inference_timesteps": inference_timesteps, "normalize": normalize,
"denoise": denoise, "output_48k": output_48k,
"context_aware": context_aware, "streaming": streaming,
})
config.ui["voice_name"] = f"{config.VOXCPM2_VOICE_PREFIX}{reference_audio if mode == 'clone' else mode}"
st.session_state["voice_rate"] = 1.0
st.session_state["voice_pitch"] = 1.0
def render_doubaotts_settings(tr):
"""渲染豆包语音 TTS 设置"""
api_key = st.text_input(
@ -1766,6 +1987,12 @@ def render_voice_preview_new(tr, selected_engine):
voice_name = f"{config.INDEXTTS_VOICE_PREFIX}{reference_audio}"
voice_rate = 1.0 # IndexTTS-1.5 不支持速度调节
voice_pitch = 1.0 # IndexTTS-1.5 不支持音调调节
elif selected_engine == config.INDEXTTS_MACOS_ENGINE:
reference_audio = config.indextts_macos.get("reference_audio", "")
if reference_audio:
voice_name = f"{config.INDEXTTS_MACOS_VOICE_PREFIX}{reference_audio}"
voice_rate = 1.0 # 语速由 macOS Pack 配置传递
voice_pitch = 1.0
elif selected_engine == config.INDEXTTS2_ENGINE:
reference_audio = config.indextts2.get("reference_audio", "")
if reference_audio:
@ -1781,6 +2008,17 @@ def render_voice_preview_new(tr, selected_engine):
voice_name = f"{config.OMNIVOICE_VOICE_PREFIX}{mode}"
voice_rate = config.omnivoice.get("speed", 1.0)
voice_pitch = 1.0
elif selected_engine == config.VOXCPM_ENGINE:
reference_audio = config.voxcpm_05b.get("reference_audio", "")
voice_name = f"{config.VOXCPM_VOICE_PREFIX}{reference_audio or 'default'}"
voice_rate = 1.0
voice_pitch = 1.0
elif selected_engine == config.VOXCPM2_ENGINE:
mode = config.voxcpm_2b.get("mode", "design")
reference_audio = config.voxcpm_2b.get("reference_audio", "")
voice_name = f"{config.VOXCPM2_VOICE_PREFIX}{reference_audio if mode == 'clone' else mode}"
voice_rate = 1.0
voice_pitch = 1.0
elif selected_engine == "doubaotts":
voice_type = config.ui.get("doubaotts_voice_type", "BV700_streaming")
voice_name = voice_type
@ -1795,8 +2033,11 @@ def render_voice_preview_new(tr, selected_engine):
temp_dir = utils.storage_dir("temp", create=True)
audio_format = "audio/wav" if selected_engine in (
config.INDEXTTS_ENGINE,
config.INDEXTTS_MACOS_ENGINE,
config.INDEXTTS2_ENGINE,
config.OMNIVOICE_ENGINE,
config.VOXCPM_ENGINE,
config.VOXCPM2_ENGINE,
) else "audio/mp3"
audio_extension = ".wav" if audio_format == "audio/wav" else ".mp3"
audio_file = os.path.join(temp_dir, f"tmp-voice-{str(uuid4())}{audio_extension}")
@ -1937,16 +2178,107 @@ def render_voice_preview(tr, voice_name):
st.error(tr("Voice synthesis failed"))
def render_sonilo_bgm_settings(tr):
"""渲染 Sonilo AI 配乐设置(可选功能,默认关闭)"""
# 避免在本模块顶层引入 basic_settings 的重依赖链,按需导入。
from webui.components.basic_settings import update_app_config_if_changed
st.info(tr("Sonilo BGM Notice"))
sonilo_api_key = st.text_input(
tr("Sonilo API Key"),
value=config.app.get("sonilo_api_key", ""),
type="password",
help=tr("Sonilo API Key Help"),
key="sonilo_api_key_input",
)
sonilo_bgm_prompt = st.text_input(
tr("Sonilo BGM Prompt"),
value=config.app.get("sonilo_bgm_prompt", ""),
help=tr("Sonilo BGM Prompt Help"),
key="sonilo_bgm_prompt_input",
)
api_key_changed = update_app_config_if_changed(
"sonilo_api_key", str(sonilo_api_key or "").strip()
)
prompt_changed = update_app_config_if_changed(
"sonilo_bgm_prompt", str(sonilo_bgm_prompt or "").strip()
)
if api_key_changed or prompt_changed:
try:
config.save_config()
st.success(tr("Sonilo config saved"))
except Exception as e:
st.error(f"{tr('Failed to save config')}: {str(e)}")
logger.error(f"保存 Sonilo 配置失败: {str(e)}")
if not sonilo.is_enabled():
st.warning(tr("Sonilo API Key Required"))
def render_sonilo_sfx_settings(tr):
"""渲染 Sonilo AI 音效设置(可选功能,默认关闭)"""
# 避免在本模块顶层引入 basic_settings 的重依赖链,按需导入。
from webui.components.basic_settings import update_app_config_if_changed
sfx_enabled = st.checkbox(
tr("Sonilo AI Sound Effects"),
value=bool(st.session_state.get("sonilo_sfx_enabled", False)),
help=tr("Sonilo SFX Help"),
key="sonilo_sfx_enabled_checkbox",
)
st.session_state["sonilo_sfx_enabled"] = bool(sfx_enabled)
if not sfx_enabled:
return
st.info(tr("Sonilo SFX Notice"))
sonilo_api_key = st.text_input(
tr("Sonilo API Key"),
value=config.app.get("sonilo_api_key", ""),
type="password",
help=tr("Sonilo API Key Help"),
key="sonilo_sfx_api_key_input",
)
sonilo_sfx_prompt = st.text_input(
tr("Sonilo SFX Prompt"),
value=config.app.get("sonilo_sfx_prompt", ""),
help=tr("Sonilo SFX Prompt Help"),
key="sonilo_sfx_prompt_input",
)
api_key_changed = update_app_config_if_changed(
"sonilo_api_key", str(sonilo_api_key or "").strip()
)
prompt_changed = update_app_config_if_changed(
"sonilo_sfx_prompt", str(sonilo_sfx_prompt or "").strip()
)
if api_key_changed or prompt_changed:
try:
config.save_config()
st.success(tr("Sonilo config saved"))
except Exception as e:
st.error(f"{tr('Failed to save config')}: {str(e)}")
logger.error(f"保存 Sonilo 配置失败: {str(e)}")
if not sonilo.is_enabled():
st.warning(tr("Sonilo SFX API Key Required"))
def render_bgm_settings(tr):
"""渲染背景音乐设置"""
saved_bgm_file = st.session_state.get('bgm_file', '')
saved_bgm_source = st.session_state.get('bgm_source', 'resource')
if st.session_state.get('bgm_type') == "":
saved_bgm_source = "none"
elif st.session_state.get('bgm_type') == "sonilo":
saved_bgm_source = "sonilo"
bgm_source_labels = {
"resource": "Select from Resource Directory",
"upload": "Upload Background Music",
"sonilo": "Sonilo AI Background Music",
"none": "No Background Music",
}
if saved_bgm_source not in bgm_source_labels:
@ -2029,12 +2361,18 @@ def render_bgm_settings(tr):
tr,
)
if bgm_source == "sonilo":
render_sonilo_bgm_settings(tr)
preview_bgm_path = st.session_state.get("bgm_preview_path", "")
if preview_bgm_path == bgm_file and os.path.isfile(preview_bgm_path):
with open(preview_bgm_path, "rb") as audio_file:
st.audio(audio_file.read(), format=get_audio_mime_type(preview_bgm_path))
bgm_type = "" if bgm_source == "none" or not bgm_file else "custom"
if bgm_source == "sonilo":
bgm_type = "sonilo"
else:
bgm_type = "" if bgm_source == "none" or not bgm_file else "custom"
st.session_state['bgm_source'] = bgm_source
st.session_state['bgm_type'] = bgm_type
st.session_state['bgm_file'] = bgm_file if bgm_type else ""
@ -2063,5 +2401,6 @@ def get_audio_params():
'bgm_type': st.session_state.get('bgm_type', 'random'),
'bgm_file': st.session_state.get('bgm_file', ''),
'bgm_volume': st.session_state.get('bgm_volume', AudioVolumeDefaults.BGM_VOLUME),
'sonilo_sfx_enabled': bool(st.session_state.get('sonilo_sfx_enabled', False)),
'tts_engine': st.session_state.get('tts_engine', config.INDEXTTS_ENGINE),
}

View File

@ -9,6 +9,7 @@ from app.config.defaults import (
DEFAULT_OPENAI_COMPATIBLE_BASE_URL,
DEFAULT_OPENAI_COMPATIBLE_PROVIDER,
DEFAULT_TEXT_LLM_PROVIDER,
DEFAULT_TEXT_OPENAI_FAST_MODEL_NAME,
DEFAULT_TEXT_OPENAI_MODEL_NAME,
DEFAULT_VISION_LLM_PROVIDER,
DEFAULT_VISION_OPENAI_MODEL_NAME,
@ -876,6 +877,10 @@ def render_text_llm_settings(tr):
# 获取已保存的配置
full_text_model_name = config.app.get("text_openai_model_name") or DEFAULT_TEXT_OPENAI_MODEL_NAME
full_fast_model_name = (
config.app.get("text_openai_fast_model_name")
or DEFAULT_TEXT_OPENAI_FAST_MODEL_NAME
)
text_api_key = config.app.get("text_openai_api_key", "")
text_base_url = config.app.get("text_openai_base_url", DEFAULT_OPENAI_COMPATIBLE_BASE_URL)
@ -885,10 +890,14 @@ def render_text_llm_settings(tr):
DEFAULT_TEXT_OPENAI_MODEL_NAME,
provider=DEFAULT_TEXT_LLM_PROVIDER,
)
current_fast_model = normalize_openai_compatible_model_id(
full_fast_model_name,
provider=DEFAULT_TEXT_LLM_PROVIDER,
)
selected_provider = DEFAULT_TEXT_LLM_PROVIDER
# 渲染配置输入框
col1, col2 = st.columns([1, 2])
col1, col2, col3 = st.columns([1, 2, 2])
with col1:
render_openai_compatible_protocol_field(
tr,
@ -897,11 +906,13 @@ def render_text_llm_settings(tr):
)
with col2:
model_name_input = st.text_input(
tr("Text Model Name"),
reasoning_model_name_input = st.text_input(
tr("High Reasoning Model Name"),
value=current_model,
help=(
tr("Model Name Input Help")
tr("High Reasoning Model Help")
+ "\n\n"
+ tr("Model Name Input Help")
+ "\n\n"
+ "• Pro/zai-org/GLM-5\n"
+ "• deepseek/deepseek-chat\n"
@ -912,8 +923,24 @@ def render_text_llm_settings(tr):
key="text_model_input"
)
with col3:
fast_model_name_input = st.text_input(
tr("High Efficiency Model Name"),
value=current_fast_model,
help=(
tr("High Efficiency Model Help")
+ "\n\n"
+ "• Qwen/Qwen3.5-32B\n"
+ "• gpt-4o-mini\n"
+ "• gemini-2.5-flash\n"
+ "• deepseek/deepseek-chat"
),
key="text_fast_model_input",
)
# 组合完整的模型名称
st_text_model_name = normalize_openai_compatible_model_name(model_name_input)
st_text_model_name = normalize_openai_compatible_model_name(reasoning_model_name_input)
st_text_fast_model_name = normalize_openai_compatible_model_name(fast_model_name_input)
st_text_api_key = st.text_input(
tr("Text API Key"),
@ -952,7 +979,7 @@ def render_text_llm_settings(tr):
test_errors = []
if not st_text_api_key:
test_errors.append(tr("Please enter API key"))
if not model_name_input:
if not reasoning_model_name_input:
test_errors.append(tr("Please enter model name"))
if test_errors:
@ -961,17 +988,25 @@ def render_text_llm_settings(tr):
else:
with st.spinner(tr("Testing connection...")):
try:
success, message = test_openai_compatible_text_model(
api_key=st_text_api_key,
base_url=st_text_base_url,
model_name=st_text_model_name,
tr=tr
)
if success:
st.success(message)
else:
st.error(message)
test_targets = [
(tr("High Reasoning Model Name"), st_text_model_name),
]
if st_text_fast_model_name:
test_targets.append((
tr("High Efficiency Model Name"),
st_text_fast_model_name,
))
for label, target_model in test_targets:
success, message = test_openai_compatible_text_model(
api_key=st_text_api_key,
base_url=st_text_base_url,
model_name=target_model,
tr=tr,
)
if success:
st.success(f"{label}: {message}")
else:
st.error(f"{label}: {message}")
except Exception as e:
st.error(f"{tr('Connection test error')}: {str(e)}")
logger.error(f"OpenAI 兼容 文案生成模型连接测试失败: {str(e)}")
@ -992,6 +1027,25 @@ def render_text_llm_settings(tr):
else:
text_validation_errors.append(error_msg)
if st_text_fast_model_name:
is_valid, error_msg = validate_openai_compatible_model_name(
st_text_fast_model_name,
"高效率文案生成",
)
if is_valid:
text_config_changed |= update_app_config_if_changed(
"text_openai_fast_model_name",
st_text_fast_model_name,
)
st.session_state["text_openai_fast_model_name"] = st_text_fast_model_name
else:
text_validation_errors.append(error_msg)
else:
text_config_changed |= update_app_config_if_changed(
"text_openai_fast_model_name",
"",
)
# 验证 API 密钥
if st_text_api_key:
is_valid, error_msg = validate_api_key(st_text_api_key, "文案生成")
@ -1027,7 +1081,7 @@ def render_text_llm_settings(tr):
config.save_config()
# 清除缓存,确保下次使用新配置
UnifiedLLMService.clear_cache()
if st_text_api_key or st_text_base_url or st_text_model_name:
if st_text_api_key or st_text_base_url or st_text_model_name or st_text_fast_model_name:
st.success(tr("Text model config saved"))
except Exception as e:
st.error(f"{tr('Failed to save config')}: {str(e)}")

View File

@ -97,6 +97,9 @@ FILM_TV_TYPE_VALUES = {
"horror_thriller": "恐怖/惊悚",
}
SHORT_DRAMA_ORIGINAL_SOUND_RATIO_OPTIONS = list(range(0, 100, 10))
DEFAULT_NARRATION_WORD_COUNT = 500
MIN_NARRATION_WORD_COUNT = 100
MAX_NARRATION_WORD_COUNT = 5000
SUMMARY_MODE_CONFIGS = {
MODE_FILM_SUMMARY: {
"mode_label_key": "Film TV Narration",
@ -1196,6 +1199,8 @@ def render_fun_asr_transcription(tr):
)
backend = backend_options[backend_label]
st.markdown(tr("Subtitle transcription package downloads"))
if backend == "upload":
render_subtitle_upload(tr)
elif backend == "local":
@ -1572,6 +1577,7 @@ def render_script_buttons(tr, params):
type_option_key = _summary_state_key(summary_config, "type_option")
custom_type_key = _summary_state_key(summary_config, "custom_type")
original_sound_ratio_key = _summary_state_key(summary_config, "original_sound_ratio")
narration_word_count_key = _summary_state_key(summary_config, "narration_word_count")
language_option_key = _summary_state_key(summary_config, "narration_language_option")
custom_language_key = _summary_state_key(summary_config, "custom_narration_language")
narration_copy_key = _summary_state_key(summary_config, "narration_copy")
@ -1590,7 +1596,7 @@ def render_script_buttons(tr, params):
config_col_widths = [1.15]
if show_custom_type:
config_col_widths.append(1.15)
config_col_widths.extend([0.9, 1.15])
config_col_widths.extend([0.9, 0.9, 1.15])
if show_custom_language:
config_col_widths.append(1.15)
@ -1621,6 +1627,16 @@ def render_script_buttons(tr, params):
key=original_sound_ratio_key,
)
config_col_index += 1
with config_cols[config_col_index]:
st.number_input(
tr("文案字数"),
min_value=MIN_NARRATION_WORD_COUNT,
max_value=MAX_NARRATION_WORD_COUNT,
value=DEFAULT_NARRATION_WORD_COUNT,
step=50,
key=narration_word_count_key,
)
config_col_index += 1
with config_cols[config_col_index]:
st.selectbox(
tr("解说语言"),
@ -1661,6 +1677,7 @@ def render_script_buttons(tr, params):
type_option_key = _summary_state_key(summary_config, "type_option")
custom_type_key = _summary_state_key(summary_config, "custom_type")
original_sound_ratio_key = _summary_state_key(summary_config, "original_sound_ratio")
narration_word_count_key = _summary_state_key(summary_config, "narration_word_count")
language_option_key = _summary_state_key(summary_config, "narration_language_option")
custom_language_key = _summary_state_key(summary_config, "custom_narration_language")
narration_copy_key = _summary_state_key(summary_config, "narration_copy")
@ -1672,6 +1689,9 @@ def render_script_buttons(tr, params):
narration_language = _resolve_summary_narration_language(summary_config)
drama_genre = _resolve_summary_type(summary_config)
original_sound_ratio = int(st.session_state.get(original_sound_ratio_key, 30))
narration_word_count = int(
st.session_state.get(narration_word_count_key, DEFAULT_NARRATION_WORD_COUNT)
)
if (
st.session_state.get(type_option_key) == "custom"
and not str(st.session_state.get(custom_type_key, '') or '').strip()
@ -1718,6 +1738,7 @@ def render_script_buttons(tr, params):
video_paths=_selected_video_paths(),
narration_language=narration_language,
drama_genre=drama_genre,
narration_word_count=narration_word_count,
prompt_category=summary_config["prompt_category"],
search_keywords=summary_config["search_keywords"],
empty_title_message_key=summary_config["empty_title_message_key"],

View File

@ -1,6 +1,9 @@
import unittest
from webui.components.audio_settings import _normalize_source_pills_value
from webui.components.audio_settings import (
_normalize_source_pills_value,
get_tts_engine_options,
)
def zh_tr(key):
@ -12,6 +15,17 @@ def zh_tr(key):
class AudioSettingsSourcePillsTests(unittest.TestCase):
def test_tts_engine_options_include_indextts_15_macos_as_local_engine(self):
options = get_tts_engine_options(lambda key: {
"Local Deployment": "本地部署",
"Cloud Service": "云服务",
}.get(key, key))
self.assertEqual(
"IndexTTS-1.5-macOS [本地部署]",
options["indextts_macos"],
)
def test_normalize_source_pills_value_keeps_canonical_value(self):
options = {
"resource": "Select from Resource Directory",

View File

@ -54,7 +54,21 @@
"Custom Background Music": "Custom Background Music",
"Custom Background Music File": "Please enter the file path of the custom background music",
"Background Music Source": "Background Music Source",
"Background Music Source Help": "Choose background music from the resource directory, upload a new file, or disable background music.",
"Background Music Source Help": "Choose background music from the resource directory, upload a new file, let Sonilo generate music from the visuals, or disable background music.",
"Sonilo AI Background Music": "AI-Generated Background Music (Sonilo)",
"Sonilo BGM Notice": "When enabled, the assembled video (without background music) is uploaded to the Sonilo API to generate background music that follows the visuals and editing pace; the narration keeps its existing volume settings and stays clearly audible. Generated music is licensed for commercial use (terms apply). Videos longer than 6 minutes are not supported; if generation fails, the task falls back to random background music.",
"Sonilo API Key": "Sonilo API Key",
"Sonilo API Key Help": "Get an API key at https://sonilo.com. Only used when Sonilo background music or sound effects are enabled.",
"Sonilo API Key Required": "Please enter a Sonilo API Key first, otherwise the task will fall back to random background music.",
"Sonilo BGM Prompt": "Music Style Hint (Optional)",
"Sonilo BGM Prompt Help": "Optional: describe the desired music style, e.g. \"calm piano\" or \"tense suspense\". Leave empty to generate purely from the visuals.",
"Sonilo config saved": "Sonilo configuration saved",
"Sonilo AI Sound Effects": "AI Sound Effects (Sonilo)",
"Sonilo SFX Help": "Automatically generate sound effects that match the visuals of the assembled video (optional, disabled by default).",
"Sonilo SFX Notice": "When enabled, the assembled video is uploaded to the Sonilo API to generate sound effects based on the visuals; the effects are mixed underneath the existing audio track, and the narration keeps its existing volume settings and stays clearly audible. Generated sound effects are royalty-free. Videos longer than 3 minutes are not supported; if generation fails, the sound-effects step is skipped and the video is produced as usual.",
"Sonilo SFX Prompt": "Sound Effects Hint (Optional)",
"Sonilo SFX Prompt Help": "Optional: describe the desired sound effects, e.g. \"rain with distant thunder\" or \"metal clanking\". Leave empty to generate purely from the visuals.",
"Sonilo SFX API Key Required": "Please enter a Sonilo API Key first, otherwise the sound-effects step will be skipped.",
"Upload Background Music": "Upload Background Music",
"Background Music Path Help": "Choose the background music used for video synthesis.",
"No Background Music Resources Found": "No background music resources found. Please upload a background music file.",
@ -197,6 +211,10 @@
"Text API Key": "Text API Key",
"Text Base URL": "Text Base URL",
"Text Model Name": "Text Model Name",
"High Reasoning Model Name": "High-Reasoning Model Name",
"High Efficiency Model Name": "High-Efficiency Model Name",
"High Reasoning Model Help": "Used for plot analysis, copy generation, and script generation and matching.",
"High Efficiency Model Help": "Used for subtitle translation and calibration; falls back to the high-reasoning model when empty.",
"Top P": "Top P",
"Top K": "Top K",
"Max Output Tokens": "Max Output Tokens",
@ -333,6 +351,7 @@
"影视类型": "Film/TV Type",
"自定义影视类型": "Custom Film/TV Type",
"原片占比": "Original Footage Ratio",
"文案字数": "Copy Length",
"例如:豪门虐恋": "For example: billionaire angst romance",
"例如:悬疑犯罪": "For example: suspense crime",
"请输入自定义短剧类型": "Please enter a custom short drama type",
@ -373,13 +392,19 @@
"Tencent Cloud TTS use case": "Personal and enterprise users who need stable Chinese speech synthesis",
"Tongyi Qwen3 TTS features": "Alibaba Cloud Tongyi Qwen speech synthesis with high-quality voices and multiple voice options.",
"High-quality Chinese speech synthesis use case": "Users who need high-quality Chinese speech synthesis",
"IndexTTS features": "A locally or privately deployed IndexTTS-1.5 voice-cloning engine. Choose a resource audio file or upload a reference audio file, then synthesize narration in that voice.",
"IndexTTS use case": "Best for fixed narrator voices, character dubbing, or generating multiple videos with the same voice. Start the IndexTTS-1.5 API service before use. Deployment package: https://pan.quark.cn/s/0767c9bcefd5",
"IndexTTS download link": "Download link: https://pan.quark.cn/s/0767c9bcefd5",
"IndexTTS2 features": "A locally or privately deployed IndexTTS-2 voice-cloning engine with emotion control and fuller generation parameters.",
"IndexTTS2 use case": "Best for fixed voices, emotional narration, and local speech synthesis workflows that need finer sampling controls. Start the IndexTTS-2 API service before use.",
"IndexTTS features": "A locally or privately deployed IndexTTS-1.5 voice-cloning engine for Windows. Choose or upload reference audio to synthesize narration in that voice.",
"IndexTTS use case": "Best for fixed narrator voices, character dubbing, and batch generation on Windows. Start the IndexTTS-1.5 API service before use. [Download the deployment package](https://cutagent.online/resources/indextts15-windows).",
"IndexTTS download link": "Download packages:\n\n- **macOS**: [Download](https://cutagent.online/resources/indextts15-mlx-macos)\n- **Windows**: [Download](https://cutagent.online/resources/indextts15-windows)",
"IndexTTS macOS features": "A local IndexTTS-1.5 MLX voice-cloning engine for Apple Silicon that uploads reference audio to synthesize narration.",
"IndexTTS macOS use case": "Best for local fixed narrator voices and character dubbing on Apple Silicon Macs. Start IndexTTS-MLX-1.5-Pack before use. [Download the deployment package](https://cutagent.online/resources/indextts15-mlx-macos).",
"IndexTTS2 features": "A locally deployed IndexTTS-2 MLX Pack voice-cloning engine with emotion control and advanced generation parameters.",
"IndexTTS2 use case": "Best for fixed voices, emotional narration, and local speech synthesis workflows that need finer sampling controls. Start the IndexTTS-2 MLX Pack service before use. Deployment packages:\n\n- **macOS**: [Download](https://cutagent.online/resources/indextts2-full-macos)\n- **Windows**: Coming soon",
"OmniVoice features": "A locally or privately deployed OmniVoice-Pack multilingual TTS engine with automatic voice generation, voice design, and reference-audio cloning.",
"OmniVoice use case": "Best for local controllable multilingual narration, voice design, or reference-audio cloning. Start the OmniVoice-Pack API service before use.",
"OmniVoice use case": "Best for local controllable multilingual narration, voice design, or reference-audio cloning. Start the OmniVoice-Pack API service before use. Deployment packages:\n\n- **macOS**: [Download](https://cutagent.online/resources/omnivoice-macos)\n- **Windows**: [Download](https://cutagent.online/resources/omnivoice-windows)",
"VoxCPM features": "A locally deployed VoxCPM-0.5B speech engine supporting its default voice and zero-shot cloning from reference audio.",
"VoxCPM use case": "Best for private local Chinese narration and reusable voices on Apple Silicon Macs. Start VoxCPM-0.5B-Pack before use; [download the deployment package](https://cutagent.online/resources/voxcpm-05b-macos).",
"VoxCPM2 features": "A locally deployed high-quality VoxCPM-2B engine with voice-description design, reference-audio cloning, 48k output, and context awareness.",
"VoxCPM2 use case": "Best for higher-quality local narration, describable voices, style control, and cloned voices on Apple Silicon Macs. Start VoxCPM-2B-Pack before use; [download the deployment package](https://cutagent.online/resources/voxcpm-2b-macos).",
"Doubao TTS features": "Volcengine Doubao speech synthesis with multiple voices and emotions, plus fast access in mainland China.",
"Local Deployment": "Local Deployment",
"Cloud Service": "Cloud Service",
@ -472,6 +497,7 @@
"Characters": "characters",
"Ali Bailian Fun-ASR Subtitle Transcription": "Subtitle Processing",
"Subtitle Processing Method": "Subtitle Processing Method",
"Subtitle transcription package downloads": "**Subtitle transcription package downloads**\n\n- **Fun-ASR-Nano**: [macOS](https://cutagent.online/resources/fun-asr-nano-macos) · [Windows](https://cutagent.online/resources/fun-asr-nano-windows)\n- **FunASR v1.1.0**: [macOS](https://cutagent.online/resources/funasr-v110-macos) · [Windows](https://cutagent.online/resources/funasr-v110-windows)\n- **FireRedASR2-AED**: [macOS](https://cutagent.online/resources/fireredasr2-aed-macos) · [Windows](https://cutagent.online/resources/fireredasr2-aed-windows)",
"Fun-ASR Backend": "Fun-ASR Backend",
"Local FunASR-Pack API": "FunASR (Local)",
"Local FireRedASR API": "FireRedASR2 (Local)",
@ -564,8 +590,30 @@
"Select Qwen3 TTS Voice": "Select a Qwen3 TTS voice",
"API URL": "API URL",
"IndexTTS API URL Help": "IndexTTS-1.5 API service URL",
"IndexTTS2 API URL Help": "IndexTTS-2 API service URL. You can enter the service root or the full /tts endpoint.",
"IndexTTS macOS API URL Help": "IndexTTS-MLX-1.5-Pack service URL. Enter the service root or the full /v1/audio/speech/upload endpoint.",
"IndexTTS2 API URL Help": "IndexTTS-2 MLX Pack service URL. Enter the service root or the full /v1/audio/speech/upload endpoint.",
"OmniVoice API URL Help": "OmniVoice-Pack API service URL. You can enter the service root or the full /tts endpoint.",
"VoxCPM API URL Help": "VoxCPM-0.5B-Pack service URL. Enter the service root, /tts, or /v1/audio/speech endpoint.",
"VoxCPM Use Reference Audio": "Clone from Reference Audio",
"VoxCPM Use Reference Audio Help": "Uploads prompt_audio through /tts when enabled; otherwise uses the model's default voice.",
"VoxCPM Prompt Text": "Reference Audio Text",
"VoxCPM Prompt Text Help": "Optional transcript of the speech in the reference audio.",
"VoxCPM Normalize": "Normalize Text",
"VoxCPM Denoise": "Denoise Reference Audio",
"VoxCPM Usage Instructions Title": "VoxCPM-0.5B Usage Instructions",
"VoxCPM Usage Instructions": "1. Start VoxCPM-0.5B-Pack (double-click start.command on macOS).\n2. The default service URL is http://127.0.0.1:7864.\n3. Use the default voice without reference audio, or enable reference audio to clone a voice.\n4. The first request may take longer while the model loads.",
"VoxCPM2 API URL Help": "VoxCPM-2B-Pack service URL. Enter the service root, /tts, /tts/batch, or /v1/audio/speech endpoint.",
"VoxCPM2 Generation Mode": "Generation Mode",
"VoxCPM2 Generation Mode Help": "Voice design generates a voice from a text description; cloning uploads reference_audio.",
"VoxCPM2 Mode Design": "Voice Design",
"VoxCPM2 Mode Clone": "Reference Audio Clone",
"VoxCPM2 Voice Control": "Voice Description / Style Control",
"VoxCPM2 Voice Control Help": "Describe voice, emotion, pace, and expression, for example: warm, natural, slightly fast young female voice.",
"VoxCPM2 Output 48k": "Output 48kHz Audio",
"VoxCPM2 Context Aware": "Enable Context Awareness",
"VoxCPM2 Streaming": "Enable Streaming Inference",
"VoxCPM2 Usage Instructions Title": "VoxCPM-2B Usage Instructions",
"VoxCPM2 Usage Instructions": "1. Start VoxCPM-2B-Pack (double-click start.command on macOS).\n2. The default service URL is http://127.0.0.1:7863.\n3. Describe the desired voice in design mode, or choose reference audio in clone mode.\n4. Initial model loading and 2B generation may take longer.",
"OmniVoice Language Code": "Synthesis Language",
"OmniVoice Language Code Help": "The language parameter sent to OmniVoice-Pack, such as zh or en.",
"OmniVoice Generation Mode": "Generation Mode",
@ -618,7 +666,18 @@
"Enable Sampling Help": "Enable sampling for more natural speech.",
"IndexTTS Usage Instructions Title": "IndexTTS-1.5 Usage Instructions",
"IndexTTS Usage Instructions": "**Zero-shot voice cloning**\n\n1. **Prepare reference audio**: upload or specify a clear audio file (3-10 seconds recommended)\n2. **Set API URL**: make sure the IndexTTS-1.5 service is running\n3. **Start synthesis**: the system will use the reference voice to synthesize new speech\n\n**Notes**:\n- Reference audio quality directly affects synthesis quality\n- Use clean audio without background noise when possible\n- Keep text length within a reasonable range\n- The first synthesis may take longer",
"IndexTTS macOS Usage Instructions Title": "IndexTTS-1.5-macOS Usage Instructions",
"IndexTTS macOS Usage Instructions": "**Local voice cloning on Apple Silicon**\n\n1. Double-click `start.command` in the Pack to start the service\n2. Keep the default API URL `http://127.0.0.1:7866`, or enter the full upload endpoint\n3. Select clean reference audio and start synthesis\n\nThe first request usually takes longer while the MLX model loads. This engine requires an Apple Silicon Mac.",
"IndexTTS2 Emotion Parameters": "Emotion Parameters",
"IndexTTS2 Emotion": "Emotion Override",
"IndexTTS2 Emotion Help": "Leave blank to preserve the reference audio emotion. Use a label such as happy or a weighted mix such as happy:0.7,calm:0.3.",
"IndexTTS2 Emotion Placeholder": "e.g. happy:0.7,calm:0.3",
"IndexTTS2 Emotion Audio Unsupported": "The IndexTTS-2 MLX Pack does not support a separate emotion reference audio. The speaker reference audio emotion will be used.",
"IndexTTS2 Speed": "Speech Speed",
"IndexTTS2 Speed Help": "Speech speed sent to the IndexTTS-2 MLX Pack (0.5x2.0x).",
"IndexTTS2 Seed": "Seed",
"IndexTTS2 Seed Help": "Optional integer seed for reproducible generation. Leave blank for random sampling.",
"IndexTTS2 Seed Placeholder": "Random",
"Emotion Mode": "Emotion Mode",
"Emotion Mode Help": "Choose the emotion control source for IndexTTS-2.",
"Emotion Mode Speaker": "Same as speaker reference",
@ -642,12 +701,21 @@
"Emotion Melancholic": "Melancholic",
"Emotion Surprised": "Surprised",
"Emotion Calm": "Calm",
"Diffusion Steps": "Diffusion Steps",
"Diffusion Steps Help": "Number of diffusion steps. Higher values can improve quality but take longer.",
"CFG Rate": "CFG Rate",
"CFG Rate Help": "Classifier-free guidance strength for the diffusion stage.",
"Interval Silence": "Inter-segment Silence (ms)",
"Interval Silence Help": "Silence inserted between text segments by the Pack.",
"Segment Overlap": "Segment Overlap (ms)",
"Segment Overlap Help": "Crossfade overlap between generated text segments.",
"IndexTTS2 Top K Help": "Top-k sampling value for IndexTTS-2 MLX Pack. It must be at least 1.",
"Max Text Tokens Per Segment": "Max Text Tokens Per Segment",
"Max Text Tokens Per Segment Help": "Maximum text tokens per segment for IndexTTS-2 inference.",
"Max Mel Tokens": "Max Mel Tokens",
"Max Mel Tokens Help": "Controls the maximum mel tokens generated in one request. Higher values can produce longer audio.",
"IndexTTS2 Usage Instructions Title": "IndexTTS-2 Usage Instructions",
"IndexTTS2 Usage Instructions": "**IndexTTS-2 voice cloning**\n\n1. **Choose a voice**: reuse IndexTTS-1.5 resource audio or upload a reference audio file\n2. **Set API URL**: for example http://192.168.3.6:7863/tts, or enter the service root\n3. **Tune emotion**: speaker is the default; switch to audio, vector, or text when needed\n4. **Tune generation**: temperature, top_p, top_k, num_beams, repetition_penalty, and max_mel_tokens are sent directly to the IndexTTS-2 API\n\n**Notes**:\n- Reference audio quality directly affects cloning quality\n- The first request may load the model and take longer\n- CPU deployments are much slower than GPU deployments",
"IndexTTS2 Usage Instructions": "**IndexTTS-2 MLX Pack voice cloning**\n\n1. **Choose a voice**: reuse IndexTTS-1.5 resource audio or upload a reference audio file\n2. **Set API URL**: the default Pack service is http://127.0.0.1:7860; NarratoAI calls /v1/audio/speech/upload and uploads the reference audio\n3. **Tune emotion**: leave it blank to preserve the reference audio emotion, or enter a label/mix such as happy:0.7,calm:0.3\n4. **Tune generation**: speed, temperature, top_p, top_k, repetition_penalty, diffusion_steps, cfg_rate, and max_mel_tokens are sent to the Pack\n\n**Notes**:\n- Reference audio quality directly affects cloning quality\n- The first request may load the model and take longer\n- The Pack binds to localhost by default",
"OmniVoice Usage Instructions Title": "OmniVoice Usage Instructions",
"OmniVoice Usage Instructions": "**OmniVoice-Pack speech synthesis**\n\n1. **Automatic voice**: set the API URL and language, then synthesize directly.\n2. **Voice design**: fill instruct with the desired gender, pitch, accent, or style.\n3. **Reference-audio clone**: upload or choose reference audio and fill its matching transcript.\n\n**Notes**:\n- The default service URL is http://127.0.0.1:7866/tts\n- Reference-audio cloning requires reference text when the service has no ASR model loaded\n- OmniVoice returns WAV audio, and NarratoAI estimates subtitle segment timing from the audio duration",
"Volcengine Access Key Help": "Volcengine Access Key",

View File

@ -42,7 +42,21 @@
"Custom Background Music": "自定义背景音乐",
"Custom Background Music File": "请输入自定义背景音乐的文件路径",
"Background Music Source": "背景音乐来源",
"Background Music Source Help": "选择资源目录中的背景音乐、上传新的背景音乐,或关闭背景音乐",
"Background Music Source Help": "选择资源目录中的背景音乐、上传新的背景音乐、使用 Sonilo 根据画面生成配乐,或关闭背景音乐",
"Sonilo AI Background Music": "AI 生成配乐Sonilo",
"Sonilo BGM Notice": "启用后,合成完成的视频(未加背景音乐)将上传至 Sonilo API根据画面内容与剪辑节奏生成配乐解说配音仍按现有音量设置保持清晰。生成的音乐已获授权、可商用以条款为准。视频时长上限 6 分钟;生成失败时自动回退到随机背景音乐,不影响成片。",
"Sonilo API Key": "Sonilo API Key",
"Sonilo API Key Help": "获取地址https://sonilo.com仅在启用 Sonilo 配乐或音效时使用",
"Sonilo API Key Required": "请先填写 Sonilo API Key否则生成时将回退到随机背景音乐",
"Sonilo BGM Prompt": "配乐风格提示(可选)",
"Sonilo BGM Prompt Help": "可选:描述期望的配乐风格,例如“舒缓钢琴”“紧张悬疑”,留空则完全根据画面生成",
"Sonilo config saved": "Sonilo 配置已保存",
"Sonilo AI Sound Effects": "AI 音效Sonilo",
"Sonilo SFX Help": "为成片自动生成贴合画面的音效(可选功能,默认关闭)",
"Sonilo SFX Notice": "启用后,合成完成的视频将上传至 Sonilo API根据画面内容生成音效并混在现有音轨之下解说配音仍按现有音量设置保持清晰。生成的音效为免版税素材。视频时长上限 3 分钟;生成失败时自动跳过音效,不影响成片。",
"Sonilo SFX Prompt": "音效风格提示(可选)",
"Sonilo SFX Prompt Help": "可选:描述期望的音效,例如“雨声和远处雷声”“金属碰撞”,留空则完全根据画面生成",
"Sonilo SFX API Key Required": "请先填写 Sonilo API Key否则生成时将跳过音效",
"Upload Background Music": "上传背景音乐",
"Background Music Path Help": "选择用于视频合成的背景音乐",
"No Background Music Resources Found": "未找到资源目录中的背景音乐,请上传背景音乐文件",
@ -186,6 +200,10 @@
"Text API Key": "文案生成 API 密钥",
"Text Base URL": "文案生成接口地址",
"Text Model Name": "文案生成模型名称",
"High Reasoning Model Name": "高推理模型名称",
"High Efficiency Model Name": "高效率模型名称",
"High Reasoning Model Help": "用于剧情分析、文案生成、脚本生成与匹配等复杂任务。",
"High Efficiency Model Help": "用于字幕翻译、字幕校准等批量任务;留空时自动使用高推理模型。",
"Top P": "Top P",
"Top K": "Top K",
"Max Output Tokens": "最大输出 Token",
@ -311,13 +329,19 @@
"Tencent Cloud TTS use case": "个人和企业用户,需要稳定的中文语音合成",
"Tongyi Qwen3 TTS features": "阿里云通义千问语音合成,音质优秀,支持多种音色",
"High-quality Chinese speech synthesis use case": "需要高质量中文语音合成的用户",
"IndexTTS features": "本地/私有部署的 IndexTTS-1.5 语音克隆引擎。选择资源目录音频或上传参考音频后,可按该音色合成旁白。",
"IndexTTS use case": "适合需要固定旁白音色、角色配音或批量生成同一音色视频的场景。使用前请先启动 IndexTTS-1.5 API 服务部署包下载https://pan.quark.cn/s/0767c9bcefd5",
"IndexTTS download link": "下载地址https://pan.quark.cn/s/0767c9bcefd5",
"IndexTTS2 features": "本地/私有部署的 IndexTTS-2 语音克隆引擎,支持情感控制和更完整的生成参数。",
"IndexTTS2 use case": "适合需要固定音色、情绪化旁白或更细致采样控制的本地语音合成场景。使用前请先启动 IndexTTS-2 API 服务。",
"IndexTTS features": "Windows 本地/私有部署的 IndexTTS-1.5 语音克隆引擎。选择资源目录音频或上传参考音频后,可按该音色合成旁白。",
"IndexTTS use case": "适合在 Windows 上生成固定旁白音色、角色配音或批量生成同一音色视频。使用前请先启动 IndexTTS-1.5 API 服务;[下载部署包](https://cutagent.online/resources/indextts15-windows)。",
"IndexTTS download link": "下载地址:\n\n- **macOS**[下载地址](https://cutagent.online/resources/indextts15-mlx-macos)\n- **Windows**[下载地址](https://cutagent.online/resources/indextts15-windows)",
"IndexTTS macOS features": "面向 Apple Silicon 的 IndexTTS-1.5 MLX 本地语音克隆引擎,通过上传参考音频合成旁白。",
"IndexTTS macOS use case": "适合在 Apple Silicon Mac 上本地生成固定旁白音色和角色配音。使用前请启动 IndexTTS-MLX-1.5-Pack[下载部署包](https://cutagent.online/resources/indextts15-mlx-macos)。",
"IndexTTS2 features": "本地部署的 IndexTTS-2 MLX Pack 语音克隆引擎,支持情感控制和更完整的生成参数。",
"IndexTTS2 use case": "适合需要固定音色、情绪化旁白或更细致采样控制的本地语音合成场景。使用前请先启动 IndexTTS-2 MLX Pack 服务;部署包下载:\n\n- **macOS**[下载地址](https://cutagent.online/resources/indextts2-full-macos)\n- **Windows**:待更新",
"OmniVoice features": "本地/私有部署的 OmniVoice-Pack 多语种语音合成引擎,支持自动音色、指令音色和参考音频克隆。",
"OmniVoice use case": "适合需要本地可控、多语言旁白、音色设计或参考音频克隆的场景。使用前请先启动 OmniVoice-Pack API 服务。",
"OmniVoice use case": "适合需要本地可控、多语言旁白、音色设计或参考音频克隆的场景。使用前请先启动 OmniVoice-Pack API 服务;部署包下载:\n\n- **macOS**[下载地址](https://cutagent.online/resources/omnivoice-macos)\n- **Windows**[下载地址](https://cutagent.online/resources/omnivoice-windows)",
"VoxCPM features": "本地部署的 VoxCPM-0.5B 语音合成引擎,支持默认音色和参考音频零样本音色克隆。",
"VoxCPM use case": "适合 Apple Silicon Mac 上的本地中文旁白、固定音色与隐私敏感型语音合成。使用前请启动 VoxCPM-0.5B-Pack[下载部署包](https://cutagent.online/resources/voxcpm-05b-macos)。",
"VoxCPM2 features": "本地部署的 VoxCPM-2B 高质量语音引擎支持音色描述设计、参考音频克隆、48k 输出和上下文感知。",
"VoxCPM2 use case": "适合 Apple Silicon Mac 上需要更高质量、可描述音色、情绪风格控制或固定克隆音色的本地旁白。使用前请启动 VoxCPM-2B-Pack[下载部署包](https://cutagent.online/resources/voxcpm-2b-macos)。",
"Doubao TTS features": "火山引擎豆包语音合成,支持多种音色和情感,国内访问速度快",
"Local Deployment": "本地部署",
"Cloud Service": "云端服务",
@ -411,6 +435,7 @@
"Characters": "字符",
"Ali Bailian Fun-ASR Subtitle Transcription": "字幕处理",
"Subtitle Processing Method": "字幕处理方式",
"Subtitle transcription package downloads": "**字幕转录整合包下载**\n\n- **Fun-ASR-Nano**[macOS](https://cutagent.online/resources/fun-asr-nano-macos) · [Windows](https://cutagent.online/resources/fun-asr-nano-windows)\n- **FunASR v1.1.0**[macOS](https://cutagent.online/resources/funasr-v110-macos) · [Windows](https://cutagent.online/resources/funasr-v110-windows)\n- **FireRedASR2-AED**[macOS](https://cutagent.online/resources/fireredasr2-aed-macos) · [Windows](https://cutagent.online/resources/fireredasr2-aed-windows)",
"Fun-ASR Backend": "Fun-ASR 后端",
"Local FunASR-Pack API": "FunASR(本地部署)",
"Local FireRedASR API": "FireRedASR2(本地部署)",
@ -503,8 +528,30 @@
"Select Qwen3 TTS Voice": "选择 Qwen3 TTS 音色",
"API URL": "API 地址",
"IndexTTS API URL Help": "IndexTTS-1.5 API 服务地址",
"IndexTTS2 API URL Help": "IndexTTS-2 API 服务地址,可填写服务根地址或完整 /tts 地址",
"IndexTTS macOS API URL Help": "IndexTTS-MLX-1.5-Pack 服务地址,可填写服务根地址或完整 /v1/audio/speech/upload 地址",
"IndexTTS2 API URL Help": "IndexTTS-2 MLX Pack 服务地址,可填写服务根地址或完整 /v1/audio/speech/upload 地址",
"OmniVoice API URL Help": "OmniVoice-Pack API 服务地址,可填写服务根地址或完整 /tts 地址",
"VoxCPM API URL Help": "VoxCPM-0.5B-Pack 服务地址,可填写服务根地址、/tts 或 /v1/audio/speech 地址。",
"VoxCPM Use Reference Audio": "使用参考音频克隆音色",
"VoxCPM Use Reference Audio Help": "启用后通过 /tts 上传 prompt_audio关闭时使用模型默认音色。",
"VoxCPM Prompt Text": "参考音频文本",
"VoxCPM Prompt Text Help": "可选,填写参考音频中实际朗读的文字。",
"VoxCPM Normalize": "文本规范化",
"VoxCPM Denoise": "参考音频降噪",
"VoxCPM Usage Instructions Title": "VoxCPM-0.5B 使用说明",
"VoxCPM Usage Instructions": "1. 启动 VoxCPM-0.5B-PackmacOS 双击 start.command。\n2. 默认服务地址为 http://127.0.0.1:7864。\n3. 不选择参考音频时使用默认音色;启用参考音频后可克隆音色。\n4. 首次合成需要加载模型,可能耗时较长。",
"VoxCPM2 API URL Help": "VoxCPM-2B-Pack 服务地址,可填写服务根地址、/tts、/tts/batch 或 /v1/audio/speech 地址。",
"VoxCPM2 Generation Mode": "生成模式",
"VoxCPM2 Generation Mode Help": "音色设计使用文字描述生成声音;参考音频克隆会上传 reference_audio。",
"VoxCPM2 Mode Design": "音色设计",
"VoxCPM2 Mode Clone": "参考音频克隆",
"VoxCPM2 Voice Control": "音色描述 / 风格控制",
"VoxCPM2 Voice Control Help": "描述音色、情绪、语速和表达,例如:温暖、自然、稍快的年轻女声。",
"VoxCPM2 Output 48k": "输出 48kHz 音频",
"VoxCPM2 Context Aware": "启用上下文感知",
"VoxCPM2 Streaming": "启用流式推理",
"VoxCPM2 Usage Instructions Title": "VoxCPM-2B 使用说明",
"VoxCPM2 Usage Instructions": "1. 启动 VoxCPM-2B-PackmacOS 双击 start.command。\n2. 默认服务地址为 http://127.0.0.1:7863。\n3. 音色设计模式填写声音与风格描述;克隆模式选择参考音频,可补充其文本。\n4. 2B 模型首次加载和生成可能耗时较长。",
"OmniVoice Language Code": "合成语言",
"OmniVoice Language Code Help": "传给 OmniVoice-Pack 的 language 参数,例如 zh、en。",
"OmniVoice Generation Mode": "生成模式",
@ -557,7 +604,18 @@
"Enable Sampling Help": "启用采样可以获得更自然的语音",
"IndexTTS Usage Instructions Title": "IndexTTS-1.5 使用说明",
"IndexTTS Usage Instructions": "**零样本语音克隆**\n\n1. **准备参考音频**:上传或指定一段清晰的音频文件(建议 3-10 秒)\n2. **设置 API 地址**:确保 IndexTTS-1.5 服务正常运行\n3. **开始合成**:系统会自动使用参考音频的音色合成新语音\n\n**注意事项**\n- 参考音频质量直接影响合成效果\n- 建议使用无背景噪音的清晰音频\n- 文本长度建议控制在合理范围内\n- 首次合成可能需要较长时间",
"IndexTTS macOS Usage Instructions Title": "IndexTTS-1.5-macOS 使用说明",
"IndexTTS macOS Usage Instructions": "**Apple Silicon 本地语音克隆**\n\n1. 双击 Pack 中的 `start.command` 启动服务\n2. 保持默认 API 地址 `http://127.0.0.1:7866`,或填写完整上传接口\n3. 选择清晰的参考音频后开始合成\n\n首次请求需要加载 MLX 模型,耗时通常更长。该引擎仅适用于 Apple Silicon Mac。",
"IndexTTS2 Emotion Parameters": "情感参数",
"IndexTTS2 Emotion": "情感覆盖",
"IndexTTS2 Emotion Help": "留空时保留参考音频的情感;可填写单个情感,如 happy或权重混合如 happy:0.7,calm:0.3。",
"IndexTTS2 Emotion Placeholder": "例如happy:0.7,calm:0.3",
"IndexTTS2 Emotion Audio Unsupported": "IndexTTS-2 MLX Pack 不支持单独的情感参考音频,将使用音色参考音频的情感。",
"IndexTTS2 Speed": "语速",
"IndexTTS2 Speed Help": "传给 IndexTTS-2 MLX Pack 的语速0.5x2.0x)。",
"IndexTTS2 Seed": "随机种子",
"IndexTTS2 Seed Help": "可选整数,用于复现生成结果;留空则随机采样。",
"IndexTTS2 Seed Placeholder": "随机",
"Emotion Mode": "情感控制方式",
"Emotion Mode Help": "选择 IndexTTS-2 的情感控制来源",
"Emotion Mode Speaker": "与音色参考相同",
@ -581,12 +639,21 @@
"Emotion Melancholic": "忧郁",
"Emotion Surprised": "惊讶",
"Emotion Calm": "平静",
"Diffusion Steps": "扩散步数",
"Diffusion Steps Help": "扩散生成的步数;值越大可能提升质量,但生成更慢。",
"CFG Rate": "CFG 引导强度",
"CFG Rate Help": "扩散阶段的 classifier-free guidance 强度。",
"Interval Silence": "分段静音(毫秒)",
"Interval Silence Help": "Pack 在文本分段之间插入的静音时长。",
"Segment Overlap": "分段重叠(毫秒)",
"Segment Overlap Help": "生成的文本片段之间用于交叉淡化的重叠时长。",
"IndexTTS2 Top K Help": "IndexTTS-2 MLX Pack 的 top-k 采样值,必须不小于 1。",
"Max Text Tokens Per Segment": "单段最大文本 Token",
"Max Text Tokens Per Segment Help": "IndexTTS-2 分段推理的最大文本 token 数",
"Max Mel Tokens": "最大 Mel Tokens",
"Max Mel Tokens Help": "控制单次生成的最大 mel token 数,值越大可生成更长音频",
"IndexTTS2 Usage Instructions Title": "IndexTTS-2 使用说明",
"IndexTTS2 Usage Instructions": "**IndexTTS-2 语音克隆**\n\n1. **选择音色**:复用 IndexTTS-1.5 的资源音频或上传参考音频\n2. **设置 API 地址**:例如 http://192.168.3.6:7863/tts也可以填写服务根地址\n3. **调整情感参数**:默认使用 speaker可按需切换到 audio、vector 或 text\n4. **调整生成参数**temperature、top_p、top_k、num_beams、repetition_penalty 和 max_mel_tokens 会直接传给 IndexTTS-2 接口\n\n**注意事项**\n- 参考音频质量会直接影响克隆效果\n- 首次请求可能需要加载模型,耗时更长\n- CPU 部署生成速度会明显慢于 GPU",
"IndexTTS2 Usage Instructions": "**IndexTTS-2 MLX Pack 语音克隆**\n\n1. **选择音色**:复用 IndexTTS-1.5 的资源音频或上传参考音频\n2. **设置 API 地址**默认 Pack 服务为 http://127.0.0.1:7860NarratoAI 会调用 /v1/audio/speech/upload 并上传参考音频\n3. **调整情感参数**:留空时保留参考音频情感;也可输入 happy:0.7,calm:0.3 这类情感或权重混合\n4. **调整生成参数**speed、temperature、top_p、top_k、repetition_penalty、diffusion_steps、cfg_rate 和 max_mel_tokens 会传给 Pack\n\n**注意事项**\n- 参考音频质量会直接影响克隆效果\n- 首次请求可能需要加载模型,耗时更长\n- Pack 默认仅监听本机地址",
"OmniVoice Usage Instructions Title": "OmniVoice 使用说明",
"OmniVoice Usage Instructions": "**OmniVoice-Pack 语音合成**\n\n1. **自动音色**:只需要设置 API 地址和语言,可直接合成。\n2. **指令音色**:填写 instruct 描述想要的性别、音高、口音或风格。\n3. **参考音频克隆**:上传或选择参考音频,并填写该音频对应文本。\n\n**注意事项**\n- 当前默认服务地址为 http://127.0.0.1:7866/tts\n- 参考音频克隆在服务未加载 ASR 模型时必须填写参考文本\n- OmniVoice 返回 WAV 音频,系统会按音频时长估算字幕段落",
"Volcengine Access Key Help": "火山引擎 Access Key",
@ -646,6 +713,7 @@
"影视类型": "影视类型",
"自定义影视类型": "自定义影视类型",
"原片占比": "原片占比",
"文案字数": "文案字数",
"例如:豪门虐恋": "例如:豪门虐恋",
"例如:悬疑犯罪": "例如:悬疑犯罪",
"请输入自定义短剧类型": "请输入自定义短剧类型",

View File

@ -363,6 +363,7 @@ def generate_short_drama_narration_copy(
search_keywords: str = SHORT_DRAMA_SEARCH_KEYWORDS,
empty_title_message_key: str = "Please enter short drama name before web search",
web_search_context_description: str = "短剧名称、人物关系、剧情背景和公开剧情梗概",
narration_word_count: int = 500,
):
"""生成可由用户审核修改的短剧解说正文,不绑定时间戳。"""
subtitle_paths = _normalize_paths(subtitle_path)
@ -422,6 +423,7 @@ def generate_short_drama_narration_copy(
temperature=temperature,
narration_language=narration_language,
drama_genre=drama_genre,
narration_word_count=narration_word_count,
)
except Exception as e:
logger.warning(f"使用新LLM服务生成文案失败回退到旧实现: {str(e)}")
@ -436,6 +438,7 @@ def generate_short_drama_narration_copy(
provider=text_provider,
narration_language=narration_language,
drama_genre=drama_genre,
narration_word_count=narration_word_count,
prompt_category=prompt_category,
)