mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-13 23:48:53 +00:00
feat(skills): support OpenAI-compatible image generation (#5389)
* feat(skills): support OpenAI-compatible image generation * fix(skills): address image provider review feedback
This commit is contained in:
parent
cf556fa9d4
commit
a22c6169b3
@ -37,6 +37,12 @@ SOFYA_API_KEY=your-sofya-api-key
|
||||
# DEEPSEEK_API_KEY=your-deepseek-api-key
|
||||
# NOVITA_API_KEY=your-novita-api-key # OpenAI-compatible, see https://novita.ai
|
||||
# MINIMAX_API_KEY=your-minimax-api-key # OpenAI-compatible, see https://platform.minimax.io
|
||||
# OpenAI-compatible Images API (used by the image-generation skill)
|
||||
# IMAGE_GENERATION_PROVIDER=openai # openai-compatible is also accepted
|
||||
# IMAGE_GENERATION_API_KEY=your-image-generation-api-key
|
||||
# IMAGE_GENERATION_BASE_URL=https://api.openai.com/v1
|
||||
# IMAGE_GENERATION_MODEL=gpt-image-2.5-flare
|
||||
# IMAGE_GENERATION_SIZE=1536x1024
|
||||
# STEPFUN_API_KEY=your-stepfun-api-key # OpenAI-compatible, see https://platform.stepfun.com
|
||||
# VLLM_API_KEY=your-vllm-api-key # OpenAI-compatible
|
||||
|
||||
|
||||
@ -262,6 +262,9 @@ This section accumulates work toward the **2.1.0** milestone
|
||||
|
||||
#### Skills
|
||||
|
||||
- **skills:** The built-in image-generation skill can use OpenAI-compatible
|
||||
Images APIs for generation and reference-image editing, with configurable
|
||||
endpoint, model, size, and output format.
|
||||
- **skills:** Native SkillScan (phase 1) statically analyzes skill packages at
|
||||
load, and `describe_skill` enables deferred discovery so the model fetches a
|
||||
skill's schema on demand instead of loading all skills up front. ([#3033],
|
||||
|
||||
@ -1194,6 +1194,14 @@ Web UI chat links percent-encode custom thread identifiers before placing them i
|
||||
└── lark-cli/lark-doc/SKILL.md ← managed, read-only
|
||||
```
|
||||
|
||||
The built-in `image-generation` skill supports Gemini, MiniMax, and
|
||||
OpenAI-compatible Images APIs. Select the latter with
|
||||
`IMAGE_GENERATION_PROVIDER=openai`, then configure
|
||||
`IMAGE_GENERATION_API_KEY`, `IMAGE_GENERATION_BASE_URL`, and
|
||||
`IMAGE_GENERATION_MODEL`. For a containerized sandbox, expose these variables
|
||||
through `sandbox.environment`; sandbox commands intentionally do not inherit
|
||||
API keys from the Gateway process.
|
||||
|
||||
#### Exporting Custom Skills
|
||||
|
||||
Administrators can export their own custom skills from **Settings → Skills → Custom → Export**. Review the file list and declared environment requirements, then choose **Download .skill**. The archive contains the currently saved skill, including supporting files and empty directories; disabled skills can also be exported. If the skill changes after preview, refresh the file list before downloading. Import the archive on another DeerFlow instance with **Install .skill**; existing-name conflicts and normal installation security checks still apply.
|
||||
|
||||
@ -1499,6 +1499,12 @@ sandbox:
|
||||
# # DEBUG: "false"
|
||||
# # API_KEY: $MY_API_KEY # Reads from host's MY_API_KEY env var
|
||||
# # DATABASE_URL: $DATABASE_URL # Reads from host's DATABASE_URL env var
|
||||
# # # Required when using the OpenAI-compatible image-generation provider:
|
||||
# # IMAGE_GENERATION_PROVIDER: $IMAGE_GENERATION_PROVIDER
|
||||
# # IMAGE_GENERATION_API_KEY: $IMAGE_GENERATION_API_KEY
|
||||
# # IMAGE_GENERATION_BASE_URL: $IMAGE_GENERATION_BASE_URL
|
||||
# # IMAGE_GENERATION_MODEL: $IMAGE_GENERATION_MODEL
|
||||
# # IMAGE_GENERATION_SIZE: $IMAGE_GENERATION_SIZE # Optional fixed-size override
|
||||
#
|
||||
# # Optional: Cross-instance container ownership (issue #4206).
|
||||
# #
|
||||
|
||||
@ -178,13 +178,33 @@ For scenarios where visual accuracy is critical, **use the `image_search` tool f
|
||||
|
||||
This approach significantly improves generation quality by providing the model with concrete visual guidance rather than relying solely on text descriptions.
|
||||
|
||||
## Providers (Gemini / MiniMax)
|
||||
## Providers (Gemini / MiniMax / OpenAI-compatible)
|
||||
|
||||
This skill auto-selects the provider by environment variables (no CLI change):
|
||||
|
||||
- `GEMINI_API_KEY` set → use Gemini (default, unchanged).
|
||||
- Only `MINIMAX_API_KEY` set → use MiniMax (`/v1/image_generation`, model `image-01`).
|
||||
- Force one explicitly with `IMAGE_GENERATION_PROVIDER=gemini|minimax`.
|
||||
- Otherwise, `MINIMAX_API_KEY` set → use MiniMax (`/v1/image_generation`, model `image-01`).
|
||||
- Otherwise, `IMAGE_GENERATION_API_KEY` set → use an OpenAI-compatible Images API.
|
||||
- Force one explicitly with `IMAGE_GENERATION_PROVIDER=gemini|minimax|openai`.
|
||||
`openai-compatible` is also accepted as an alias for `openai`.
|
||||
|
||||
OpenAI-compatible settings:
|
||||
|
||||
- `IMAGE_GENERATION_API_KEY` (required)
|
||||
- `IMAGE_GENERATION_BASE_URL` (default `https://api.openai.com/v1`)
|
||||
- `IMAGE_GENERATION_MODEL` (default `gpt-image-2.5-flare`)
|
||||
- `IMAGE_GENERATION_SIZE` (optional fixed size override)
|
||||
|
||||
Text-to-image calls use `POST {base_url}/images/generations`. Reference-image calls
|
||||
use multipart `POST {base_url}/images/edits`; a relay may support generation without
|
||||
supporting edits. Responses may contain base64 image data, a data URL, or a downloadable
|
||||
URL. Aspect ratios map to `1024x1024`, `1536x1024`, or `1024x1536` unless
|
||||
`IMAGE_GENERATION_SIZE` is set. The output extension selects the API `output_format`:
|
||||
`.jpg`/`.jpeg` uses `jpeg`, `.webp` uses `webp`, and all other extensions use `png`.
|
||||
When `dall-e-2` or `dall-e-3` is configured instead, the request uses the model's
|
||||
supported dimensions and `response_format=b64_json`; DALL-E output files must use a
|
||||
`.png` extension. Reference-image editing with DALL-E models is not supported by this
|
||||
skill; use the default GPT Image model for edits.
|
||||
|
||||
MiniMax optional overrides: `MINIMAX_API_HOST` (default `https://api.minimaxi.com`),
|
||||
`MINIMAX_IMAGE_MODEL` (default `image-01`). Reference images are sent as the MiniMax
|
||||
|
||||
@ -1,10 +1,13 @@
|
||||
import base64
|
||||
from contextlib import ExitStack
|
||||
import json
|
||||
import os
|
||||
|
||||
import requests
|
||||
|
||||
MINIMAX_DEFAULT_HOST = "https://api.minimaxi.com"
|
||||
OPENAI_DEFAULT_BASE_URL = "https://api.openai.com/v1"
|
||||
OPENAI_DEFAULT_MODEL = "gpt-image-2.5-flare"
|
||||
# MiniMax image-01 caps the prompt at 1500 characters and rejects longer requests
|
||||
# with a generic "invalid params" error, so validate before calling the API.
|
||||
MINIMAX_PROMPT_MAX_CHARS = 1500
|
||||
@ -25,12 +28,14 @@ def validate_image(image_path: str) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _resolve_provider(override_env: str, existing_provider: str, has_existing_creds: bool) -> str:
|
||||
def _resolve_provider(
|
||||
override_env: str, existing_provider: str, has_existing_creds: bool
|
||||
) -> str:
|
||||
"""Pick the generation provider.
|
||||
|
||||
1. Explicit <SKILL>_PROVIDER override wins.
|
||||
2. Otherwise prefer the existing provider when its credentials are present.
|
||||
3. Otherwise fall back to MiniMax when MINIMAX_API_KEY is set.
|
||||
3. Otherwise fall back to MiniMax, then the OpenAI-compatible provider.
|
||||
"""
|
||||
override = os.getenv(override_env)
|
||||
if override:
|
||||
@ -39,9 +44,12 @@ def _resolve_provider(override_env: str, existing_provider: str, has_existing_cr
|
||||
return existing_provider
|
||||
if os.getenv("MINIMAX_API_KEY"):
|
||||
return "minimax"
|
||||
if os.getenv("IMAGE_GENERATION_API_KEY"):
|
||||
return "openai"
|
||||
raise ValueError(
|
||||
f"No credentials found. Set GEMINI_API_KEY for {existing_provider}, "
|
||||
f"or MINIMAX_API_KEY for minimax (optionally force with {override_env})."
|
||||
"MINIMAX_API_KEY for minimax, or IMAGE_GENERATION_API_KEY for openai "
|
||||
f"(optionally force with {override_env})."
|
||||
)
|
||||
|
||||
|
||||
@ -49,6 +57,48 @@ def _minimax_host() -> str:
|
||||
return os.getenv("MINIMAX_API_HOST", MINIMAX_DEFAULT_HOST).rstrip("/")
|
||||
|
||||
|
||||
def _openai_base_url() -> str:
|
||||
return os.getenv("IMAGE_GENERATION_BASE_URL", OPENAI_DEFAULT_BASE_URL).rstrip("/")
|
||||
|
||||
|
||||
def _openai_size(aspect_ratio: str, model: str) -> str:
|
||||
override = os.getenv("IMAGE_GENERATION_SIZE")
|
||||
if override:
|
||||
allowed_dall_e_sizes = {
|
||||
"dall-e-2": {"256x256", "512x512", "1024x1024"},
|
||||
"dall-e-3": {"1024x1024", "1792x1024", "1024x1792"},
|
||||
}
|
||||
allowed = allowed_dall_e_sizes.get(model)
|
||||
if allowed is not None and override not in allowed:
|
||||
supported = ", ".join(sorted(allowed))
|
||||
raise ValueError(f"{model} size must be one of: {supported}")
|
||||
return override
|
||||
portrait = aspect_ratio in {"9:16", "2:3", "3:4"}
|
||||
landscape = aspect_ratio in {"16:9", "3:2", "4:3"}
|
||||
if model == "dall-e-2":
|
||||
return "1024x1024"
|
||||
if model == "dall-e-3":
|
||||
if portrait:
|
||||
return "1024x1792"
|
||||
if landscape:
|
||||
return "1792x1024"
|
||||
return "1024x1024"
|
||||
if portrait:
|
||||
return "1024x1536"
|
||||
if landscape:
|
||||
return "1536x1024"
|
||||
return "1024x1024"
|
||||
|
||||
|
||||
def _openai_output_format(output_file: str) -> str:
|
||||
extension = os.path.splitext(output_file)[1].lower()
|
||||
if extension in {".jpg", ".jpeg"}:
|
||||
return "jpeg"
|
||||
if extension == ".webp":
|
||||
return "webp"
|
||||
return "png"
|
||||
|
||||
|
||||
def _check_base_resp(payload: dict) -> None:
|
||||
base = payload.get("base_resp") or {}
|
||||
if base.get("status_code", 0) != 0:
|
||||
@ -106,8 +156,8 @@ def _minimax_prompt(raw: str) -> str:
|
||||
def _generate_image_minimax(
|
||||
prompt: str, reference_images: list[str], output_file: str, aspect_ratio: str
|
||||
) -> str:
|
||||
api_key = os.getenv("MINIMAX_API_KEY")
|
||||
if not api_key:
|
||||
bearer_value = os.getenv("MINIMAX_API_KEY")
|
||||
if not bearer_value:
|
||||
return "MINIMAX_API_KEY is not set"
|
||||
prompt = _minimax_prompt(prompt)
|
||||
if len(prompt) > MINIMAX_PROMPT_MAX_CHARS:
|
||||
@ -128,11 +178,15 @@ def _generate_image_minimax(
|
||||
# Reference images are passed as character subjects as-is; unlike the Gemini
|
||||
# path we do not pre-validate them — invalid files surface as a MiniMax API error.
|
||||
body["subject_reference"] = [
|
||||
{"type": "character", "image_file": _to_data_url(p)} for p in reference_images
|
||||
{"type": "character", "image_file": _to_data_url(p)}
|
||||
for p in reference_images
|
||||
]
|
||||
response = requests.post(
|
||||
f"{_minimax_host()}/v1/image_generation",
|
||||
headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
|
||||
headers={
|
||||
"Authorization": f"Bearer {bearer_value}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json=body,
|
||||
timeout=60,
|
||||
)
|
||||
@ -160,19 +214,21 @@ def _generate_image_gemini(
|
||||
print(f"Skipping invalid reference image: {ref_img}")
|
||||
if len(valid_reference_images) < len(reference_images):
|
||||
skipped = len(reference_images) - len(valid_reference_images)
|
||||
print(f"Note: {skipped} reference image(s) were skipped due to validation failure.")
|
||||
print(
|
||||
f"Note: {skipped} reference image(s) were skipped due to validation failure."
|
||||
)
|
||||
|
||||
for reference_image in valid_reference_images:
|
||||
with open(reference_image, "rb") as f:
|
||||
image_b64 = base64.b64encode(f.read()).decode("utf-8")
|
||||
parts.append({"inlineData": {"mimeType": "image/jpeg", "data": image_b64}})
|
||||
|
||||
api_key = os.getenv("GEMINI_API_KEY")
|
||||
if not api_key:
|
||||
bearer_value = os.getenv("GEMINI_API_KEY")
|
||||
if not bearer_value:
|
||||
return "GEMINI_API_KEY is not set"
|
||||
response = requests.post(
|
||||
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3-pro-image-preview:generateContent",
|
||||
headers={"x-goog-api-key": api_key, "Content-Type": "application/json"},
|
||||
headers={"x-goog-api-key": bearer_value, "Content-Type": "application/json"},
|
||||
json={
|
||||
"generationConfig": {"imageConfig": {"aspectRatio": aspect_ratio}},
|
||||
"contents": [{"parts": [*parts, {"text": prompt}]}],
|
||||
@ -191,6 +247,91 @@ def _generate_image_gemini(
|
||||
raise Exception("Failed to generate image")
|
||||
|
||||
|
||||
def _write_openai_image(payload: dict, output_file: str) -> str:
|
||||
images = payload.get("data") or []
|
||||
if not images or not isinstance(images[0], dict):
|
||||
raise Exception("OpenAI-compatible provider returned no image data")
|
||||
|
||||
item = images[0]
|
||||
encoded = item.get("b64_json") or item.get("image_base64") or item.get("base64")
|
||||
if encoded:
|
||||
image_bytes = base64.b64decode(encoded)
|
||||
else:
|
||||
image_url = item.get("url")
|
||||
if not image_url:
|
||||
raise Exception(
|
||||
"OpenAI-compatible provider returned neither base64 data nor a URL"
|
||||
)
|
||||
if image_url.startswith("data:"):
|
||||
_, encoded = image_url.split(",", 1)
|
||||
image_bytes = base64.b64decode(encoded)
|
||||
else:
|
||||
download = requests.get(image_url, timeout=120)
|
||||
download.raise_for_status()
|
||||
image_bytes = download.content
|
||||
|
||||
_ensure_output_dir(output_file)
|
||||
with open(output_file, "wb") as f:
|
||||
f.write(image_bytes)
|
||||
return f"Successfully generated image to {output_file}"
|
||||
|
||||
|
||||
def _generate_image_openai(
|
||||
prompt: str, reference_images: list[str], output_file: str, aspect_ratio: str
|
||||
) -> str:
|
||||
bearer_value = os.getenv("IMAGE_GENERATION_API_KEY")
|
||||
if not bearer_value:
|
||||
return "IMAGE_GENERATION_API_KEY is not set"
|
||||
|
||||
url = f"{_openai_base_url()}/images/generations"
|
||||
headers = {"Authorization": f"Bearer {bearer_value}"}
|
||||
model = os.getenv("IMAGE_GENERATION_MODEL", OPENAI_DEFAULT_MODEL)
|
||||
is_dall_e = model in {"dall-e-2", "dall-e-3"}
|
||||
if is_dall_e and os.path.splitext(output_file)[1].lower() != ".png":
|
||||
raise ValueError("DALL-E output files must use a .png extension")
|
||||
if is_dall_e and reference_images:
|
||||
raise ValueError(
|
||||
f"{model} reference-image editing is not supported by this skill"
|
||||
)
|
||||
fields = {
|
||||
"model": model,
|
||||
"prompt": prompt,
|
||||
"n": 1,
|
||||
"size": _openai_size(aspect_ratio, model),
|
||||
}
|
||||
if is_dall_e:
|
||||
fields["response_format"] = "b64_json"
|
||||
else:
|
||||
fields["output_format"] = _openai_output_format(output_file)
|
||||
|
||||
if reference_images:
|
||||
url = f"{_openai_base_url()}/images/edits"
|
||||
with ExitStack() as stack:
|
||||
files = [
|
||||
(
|
||||
"image[]",
|
||||
(
|
||||
os.path.basename(path),
|
||||
stack.enter_context(open(path, "rb")),
|
||||
_guess_mime(path),
|
||||
),
|
||||
)
|
||||
for path in reference_images
|
||||
]
|
||||
response = requests.post(
|
||||
url, headers=headers, data=fields, files=files, timeout=180
|
||||
)
|
||||
else:
|
||||
response = requests.post(
|
||||
url,
|
||||
headers={**headers, "Content-Type": "application/json"},
|
||||
json=fields,
|
||||
timeout=180,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return _write_openai_image(response.json(), output_file)
|
||||
|
||||
|
||||
def generate_image(
|
||||
prompt_file: str,
|
||||
reference_images: list[str],
|
||||
@ -202,27 +343,58 @@ def generate_image(
|
||||
provider = _resolve_provider(
|
||||
"IMAGE_GENERATION_PROVIDER", "gemini", bool(os.getenv("GEMINI_API_KEY"))
|
||||
)
|
||||
if provider in ("openai", "openai-compatible"):
|
||||
return _generate_image_openai(
|
||||
prompt, reference_images, output_file, aspect_ratio
|
||||
)
|
||||
if provider == "minimax":
|
||||
return _generate_image_minimax(prompt, reference_images, output_file, aspect_ratio)
|
||||
return _generate_image_minimax(
|
||||
prompt, reference_images, output_file, aspect_ratio
|
||||
)
|
||||
if provider in ("gemini", "google"):
|
||||
return _generate_image_gemini(prompt, reference_images, output_file, aspect_ratio)
|
||||
raise ValueError(f"Unknown image provider: {provider!r} (use 'gemini' or 'minimax')")
|
||||
return _generate_image_gemini(
|
||||
prompt, reference_images, output_file, aspect_ratio
|
||||
)
|
||||
raise ValueError(
|
||||
f"Unknown image provider: {provider!r} "
|
||||
"(use 'gemini', 'minimax', 'openai', or 'openai-compatible')"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description="Generate images using Gemini or MiniMax API")
|
||||
parser.add_argument("--prompt-file", required=True, help="Absolute path to JSON prompt file")
|
||||
parser.add_argument("--reference-images", nargs="*", default=[],
|
||||
help="Absolute paths to reference images (space-separated)")
|
||||
parser.add_argument("--output-file", required=True, help="Output path for generated image")
|
||||
parser.add_argument("--aspect-ratio", required=False, default="16:9",
|
||||
help="Aspect ratio of the generated image")
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Generate images using Gemini, MiniMax, or an OpenAI-compatible API"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--prompt-file", required=True, help="Absolute path to JSON prompt file"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--reference-images",
|
||||
nargs="*",
|
||||
default=[],
|
||||
help="Absolute paths to reference images (space-separated)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-file", required=True, help="Output path for generated image"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--aspect-ratio",
|
||||
required=False,
|
||||
default="16:9",
|
||||
help="Aspect ratio of the generated image",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
print(generate_image(args.prompt_file, args.reference_images,
|
||||
args.output_file, args.aspect_ratio))
|
||||
print(
|
||||
generate_image(
|
||||
args.prompt_file,
|
||||
args.reference_images,
|
||||
args.output_file,
|
||||
args.aspect_ratio,
|
||||
)
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"Error while generating image: {e}")
|
||||
|
||||
@ -12,30 +12,61 @@ img = load("image-generation")
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_env(monkeypatch):
|
||||
for k in ["GEMINI_API_KEY", "MINIMAX_API_KEY", "IMAGE_GENERATION_PROVIDER",
|
||||
"MINIMAX_API_HOST", "MINIMAX_IMAGE_MODEL"]:
|
||||
for k in [
|
||||
"GEMINI_API_KEY",
|
||||
"MINIMAX_API_KEY",
|
||||
"IMAGE_GENERATION_PROVIDER",
|
||||
"MINIMAX_API_HOST",
|
||||
"MINIMAX_IMAGE_MODEL",
|
||||
"IMAGE_GENERATION_API_KEY",
|
||||
"IMAGE_GENERATION_BASE_URL",
|
||||
"IMAGE_GENERATION_MODEL",
|
||||
"IMAGE_GENERATION_SIZE",
|
||||
]:
|
||||
monkeypatch.delenv(k, raising=False)
|
||||
|
||||
|
||||
def test_resolve_prefers_gemini(monkeypatch):
|
||||
monkeypatch.setenv("GEMINI_API_KEY", "g")
|
||||
monkeypatch.setenv("MINIMAX_API_KEY", "m")
|
||||
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "gemini"
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
assert (
|
||||
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "gemini"
|
||||
)
|
||||
|
||||
|
||||
def test_resolve_falls_back_to_minimax(monkeypatch):
|
||||
monkeypatch.setenv("MINIMAX_API_KEY", "m")
|
||||
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "minimax"
|
||||
assert (
|
||||
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "minimax"
|
||||
)
|
||||
|
||||
|
||||
def test_resolve_falls_back_to_openai_after_minimax(monkeypatch):
|
||||
monkeypatch.setenv("MINIMAX_API_KEY", "m")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
assert (
|
||||
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "minimax"
|
||||
)
|
||||
|
||||
|
||||
def test_resolve_falls_back_to_openai(monkeypatch):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
assert (
|
||||
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "openai"
|
||||
)
|
||||
|
||||
|
||||
def test_resolve_override_wins(monkeypatch):
|
||||
monkeypatch.setenv("GEMINI_API_KEY", "g")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "MiniMax")
|
||||
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "minimax"
|
||||
assert (
|
||||
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "minimax"
|
||||
)
|
||||
|
||||
|
||||
def test_resolve_errors_when_none(monkeypatch):
|
||||
with pytest.raises(ValueError):
|
||||
with pytest.raises(ValueError, match="IMAGE_GENERATION_API_KEY"):
|
||||
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False)
|
||||
|
||||
|
||||
@ -48,8 +79,12 @@ def test_minimax_builds_payload_and_writes(monkeypatch, tmp_path):
|
||||
captured["url"] = url
|
||||
captured["headers"] = headers
|
||||
captured["json"] = json
|
||||
return FakeResp({"data": {"image_base64": [base64.b64encode(raw).decode()]},
|
||||
"base_resp": {"status_code": 0, "status_msg": "success"}})
|
||||
return FakeResp(
|
||||
{
|
||||
"data": {"image_base64": [base64.b64encode(raw).decode()]},
|
||||
"base_resp": {"status_code": 0, "status_msg": "success"},
|
||||
}
|
||||
)
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
out = tmp_path / "o.jpg"
|
||||
@ -74,8 +109,12 @@ def test_minimax_reference_image_as_data_url(monkeypatch, tmp_path):
|
||||
|
||||
def fake_post(url, headers=None, json=None, **kw):
|
||||
captured["json"] = json
|
||||
return FakeResp({"data": {"image_base64": [base64.b64encode(b"x").decode()]},
|
||||
"base_resp": {"status_code": 0}})
|
||||
return FakeResp(
|
||||
{
|
||||
"data": {"image_base64": [base64.b64encode(b"x").decode()]},
|
||||
"base_resp": {"status_code": 0},
|
||||
}
|
||||
)
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
ref = tmp_path / "ref.jpg"
|
||||
@ -88,6 +127,7 @@ def test_minimax_reference_image_as_data_url(monkeypatch, tmp_path):
|
||||
assert subj[0]["type"] == "character"
|
||||
assert subj[0]["image_file"].startswith("data:image/jpeg;base64,")
|
||||
import base64 as _b64
|
||||
|
||||
encoded = subj[0]["image_file"].split(",", 1)[1]
|
||||
assert _b64.b64decode(encoded) == b"\xff\xd8refbytes"
|
||||
|
||||
@ -96,7 +136,9 @@ def test_minimax_raises_on_base_resp_error(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("MINIMAX_API_KEY", "m")
|
||||
|
||||
def fake_post(url, headers=None, json=None, **kw):
|
||||
return FakeResp({"base_resp": {"status_code": 1004, "status_msg": "auth failed"}})
|
||||
return FakeResp(
|
||||
{"base_resp": {"status_code": 1004, "status_msg": "auth failed"}}
|
||||
)
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
prompt_file = tmp_path / "p.json"
|
||||
@ -112,8 +154,12 @@ def test_minimax_extracts_json_prompt_field(monkeypatch, tmp_path):
|
||||
|
||||
def fake_post(url, headers=None, json=None, **kw):
|
||||
captured["json"] = json
|
||||
return FakeResp({"data": {"image_base64": [base64.b64encode(b"x").decode()]},
|
||||
"base_resp": {"status_code": 0}})
|
||||
return FakeResp(
|
||||
{
|
||||
"data": {"image_base64": [base64.b64encode(b"x").decode()]},
|
||||
"base_resp": {"status_code": 0},
|
||||
}
|
||||
)
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
prompt_file = tmp_path / "p.json"
|
||||
@ -135,8 +181,12 @@ def test_minimax_plaintext_prompt_passes_through(monkeypatch, tmp_path):
|
||||
|
||||
def fake_post(url, headers=None, json=None, **kw):
|
||||
captured["json"] = json
|
||||
return FakeResp({"data": {"image_base64": [base64.b64encode(b"x").decode()]},
|
||||
"base_resp": {"status_code": 0}})
|
||||
return FakeResp(
|
||||
{
|
||||
"data": {"image_base64": [base64.b64encode(b"x").decode()]},
|
||||
"base_resp": {"status_code": 0},
|
||||
}
|
||||
)
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
prompt_file = tmp_path / "p.txt"
|
||||
@ -167,8 +217,12 @@ def test_minimax_creates_nested_output_dir(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("MINIMAX_API_KEY", "m")
|
||||
|
||||
def fake_post(url, headers=None, json=None, **kw):
|
||||
return FakeResp({"data": {"image_base64": [base64.b64encode(b"img").decode()]},
|
||||
"base_resp": {"status_code": 0}})
|
||||
return FakeResp(
|
||||
{
|
||||
"data": {"image_base64": [base64.b64encode(b"img").decode()]},
|
||||
"base_resp": {"status_code": 0},
|
||||
}
|
||||
)
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
prompt_file = tmp_path / "p.txt"
|
||||
@ -180,7 +234,7 @@ def test_minimax_creates_nested_output_dir(monkeypatch, tmp_path):
|
||||
|
||||
|
||||
def test_unknown_provider_raises(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "unknown")
|
||||
monkeypatch.setenv("GEMINI_API_KEY", "g")
|
||||
pf = tmp_path / "p.json"
|
||||
pf.write_text("x", encoding="utf-8")
|
||||
@ -188,6 +242,183 @@ def test_unknown_provider_raises(monkeypatch, tmp_path):
|
||||
img.generate_image(str(pf), [], str(tmp_path / "o.jpg"), "1:1")
|
||||
|
||||
|
||||
def test_openai_compatible_generation_writes_base64(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_BASE_URL", "https://models.example/v1/")
|
||||
captured = {}
|
||||
|
||||
def fake_post(url, headers=None, json=None, **kwargs):
|
||||
captured.update(url=url, headers=headers, json=json, kwargs=kwargs)
|
||||
return FakeResp({"data": [{"b64_json": base64.b64encode(b"image").decode()}]})
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
prompt_file = tmp_path / "prompt.json"
|
||||
prompt_file.write_text('{"prompt": "a red deer"}', encoding="utf-8")
|
||||
output_file = tmp_path / "nested" / "image.png"
|
||||
|
||||
result = img.generate_image(str(prompt_file), [], str(output_file), "16:9")
|
||||
|
||||
assert captured["url"] == "https://models.example/v1/images/generations"
|
||||
assert captured["headers"]["Authorization"] == "Bearer image-key"
|
||||
assert captured["json"]["model"] == "gpt-image-2.5-flare"
|
||||
assert captured["json"]["size"] == "1536x1024"
|
||||
assert captured["json"]["output_format"] == "png"
|
||||
assert "response_format" not in captured["json"]
|
||||
assert output_file.read_bytes() == b"image"
|
||||
assert "Successfully generated image" in result
|
||||
|
||||
|
||||
def test_openai_compatible_generation_downloads_url(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai-compatible")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
|
||||
monkeypatch.setattr(
|
||||
img.requests,
|
||||
"post",
|
||||
lambda *args, **kwargs: FakeResp(
|
||||
{"data": [{"url": "https://cdn.example/image.png"}]}
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
img.requests,
|
||||
"get",
|
||||
lambda *args, **kwargs: FakeResp(content=b"downloaded-image"),
|
||||
)
|
||||
prompt_file = tmp_path / "prompt.txt"
|
||||
prompt_file.write_text("a red deer", encoding="utf-8")
|
||||
output_file = tmp_path / "image.png"
|
||||
|
||||
img.generate_image(str(prompt_file), [], str(output_file), "1:1")
|
||||
|
||||
assert output_file.read_bytes() == b"downloaded-image"
|
||||
|
||||
|
||||
def test_openai_compatible_dall_e_uses_response_format(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_MODEL", "dall-e-3")
|
||||
captured = {}
|
||||
|
||||
def fake_post(url, headers=None, json=None, **kwargs):
|
||||
captured["json"] = json
|
||||
return FakeResp({"data": [{"b64_json": base64.b64encode(b"image").decode()}]})
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
prompt_file = tmp_path / "prompt.txt"
|
||||
prompt_file.write_text("a red deer", encoding="utf-8")
|
||||
|
||||
img.generate_image(str(prompt_file), [], str(tmp_path / "image.png"), "1:1")
|
||||
|
||||
assert captured["json"]["response_format"] == "b64_json"
|
||||
assert "output_format" not in captured["json"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("model", "aspect_ratio", "expected"),
|
||||
[
|
||||
("gpt-image-2.5-flare", "16:9", "1536x1024"),
|
||||
("gpt-image-2.5-flare", "9:16", "1024x1536"),
|
||||
("dall-e-3", "16:9", "1792x1024"),
|
||||
("dall-e-3", "9:16", "1024x1792"),
|
||||
("dall-e-2", "16:9", "1024x1024"),
|
||||
],
|
||||
)
|
||||
def test_openai_size_matches_model(model, aspect_ratio, expected):
|
||||
assert img._openai_size(aspect_ratio, model) == expected
|
||||
|
||||
|
||||
def test_openai_size_override_wins(monkeypatch):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_SIZE", "2048x1024")
|
||||
|
||||
assert img._openai_size("1:1", "gpt-image-2.5-flare") == "2048x1024"
|
||||
|
||||
|
||||
def test_openai_dall_e_rejects_invalid_size_override(monkeypatch):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_SIZE", "1536x1024")
|
||||
|
||||
with pytest.raises(ValueError, match="dall-e-3 size must be one of"):
|
||||
img._openai_size("16:9", "dall-e-3")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("extension", [".jpg", ".webp", ".unknown"])
|
||||
def test_openai_dall_e_rejects_non_png_output(monkeypatch, tmp_path, extension):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_MODEL", "dall-e-3")
|
||||
monkeypatch.setattr(
|
||||
img.requests,
|
||||
"post",
|
||||
lambda *args, **kwargs: pytest.fail("request must not be sent"),
|
||||
)
|
||||
prompt_file = tmp_path / "prompt.txt"
|
||||
prompt_file.write_text("a red deer", encoding="utf-8")
|
||||
|
||||
with pytest.raises(ValueError, match=r"\.png extension"):
|
||||
img.generate_image(
|
||||
str(prompt_file), [], str(tmp_path / f"image{extension}"), "1:1"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("model", ["dall-e-2", "dall-e-3"])
|
||||
def test_openai_dall_e_rejects_reference_images(monkeypatch, tmp_path, model):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_MODEL", model)
|
||||
monkeypatch.setattr(
|
||||
img.requests,
|
||||
"post",
|
||||
lambda *args, **kwargs: pytest.fail("request must not be sent"),
|
||||
)
|
||||
prompt_file = tmp_path / "prompt.txt"
|
||||
prompt_file.write_text("add snow", encoding="utf-8")
|
||||
reference = tmp_path / "reference.png"
|
||||
reference.write_bytes(b"png")
|
||||
|
||||
with pytest.raises(ValueError, match="reference-image editing is not supported"):
|
||||
img.generate_image(
|
||||
str(prompt_file), [str(reference)], str(tmp_path / "image.png"), "1:1"
|
||||
)
|
||||
|
||||
|
||||
def test_openai_compatible_reference_images_use_edits(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
|
||||
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
|
||||
captured = {}
|
||||
|
||||
def fake_post(url, headers=None, data=None, files=None, **kwargs):
|
||||
captured.update(url=url, data=data, files=files)
|
||||
return FakeResp({"data": [{"b64_json": base64.b64encode(b"edited").decode()}]})
|
||||
|
||||
monkeypatch.setattr(img.requests, "post", fake_post)
|
||||
prompt_file = tmp_path / "prompt.txt"
|
||||
prompt_file.write_text("add snow", encoding="utf-8")
|
||||
reference = tmp_path / "reference.png"
|
||||
reference.write_bytes(b"png")
|
||||
output_file = tmp_path / "image.png"
|
||||
|
||||
img.generate_image(str(prompt_file), [str(reference)], str(output_file), "9:16")
|
||||
|
||||
assert captured["url"].endswith("/images/edits")
|
||||
assert captured["data"]["size"] == "1024x1536"
|
||||
assert captured["files"][0][0] == "image[]"
|
||||
assert output_file.read_bytes() == b"edited"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("filename", "expected"),
|
||||
[
|
||||
("image.png", "png"),
|
||||
("image.jpg", "jpeg"),
|
||||
("image.jpeg", "jpeg"),
|
||||
("image.webp", "webp"),
|
||||
("image.unknown", "png"),
|
||||
],
|
||||
)
|
||||
def test_openai_output_format_matches_filename(filename, expected):
|
||||
assert img._openai_output_format(filename) == expected
|
||||
|
||||
|
||||
def test_guess_mime_by_extension():
|
||||
assert img._guess_mime("/a/b.png") == "image/png"
|
||||
assert img._guess_mime("/a/b.webp") == "image/webp"
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user