feat(skills): support OpenAI-compatible image generation (#5389)

* feat(skills): support OpenAI-compatible image generation

* fix(skills): address image provider review feedback
This commit is contained in:
cybersentia 2026-09-13 18:07:58 +08:00 committed by GitHub
parent cf556fa9d4
commit a22c6169b3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 490 additions and 44 deletions

View File

@ -37,6 +37,12 @@ SOFYA_API_KEY=your-sofya-api-key
# DEEPSEEK_API_KEY=your-deepseek-api-key
# NOVITA_API_KEY=your-novita-api-key # OpenAI-compatible, see https://novita.ai
# MINIMAX_API_KEY=your-minimax-api-key # OpenAI-compatible, see https://platform.minimax.io
# OpenAI-compatible Images API (used by the image-generation skill)
# IMAGE_GENERATION_PROVIDER=openai # openai-compatible is also accepted
# IMAGE_GENERATION_API_KEY=your-image-generation-api-key
# IMAGE_GENERATION_BASE_URL=https://api.openai.com/v1
# IMAGE_GENERATION_MODEL=gpt-image-2.5-flare
# IMAGE_GENERATION_SIZE=1536x1024
# STEPFUN_API_KEY=your-stepfun-api-key # OpenAI-compatible, see https://platform.stepfun.com
# VLLM_API_KEY=your-vllm-api-key # OpenAI-compatible

View File

@ -262,6 +262,9 @@ This section accumulates work toward the **2.1.0** milestone
#### Skills
- **skills:** The built-in image-generation skill can use OpenAI-compatible
Images APIs for generation and reference-image editing, with configurable
endpoint, model, size, and output format.
- **skills:** Native SkillScan (phase 1) statically analyzes skill packages at
load, and `describe_skill` enables deferred discovery so the model fetches a
skill's schema on demand instead of loading all skills up front. ([#3033],

View File

@ -1194,6 +1194,14 @@ Web UI chat links percent-encode custom thread identifiers before placing them i
└── lark-cli/lark-doc/SKILL.md ← managed, read-only
```
The built-in `image-generation` skill supports Gemini, MiniMax, and
OpenAI-compatible Images APIs. Select the latter with
`IMAGE_GENERATION_PROVIDER=openai`, then configure
`IMAGE_GENERATION_API_KEY`, `IMAGE_GENERATION_BASE_URL`, and
`IMAGE_GENERATION_MODEL`. For a containerized sandbox, expose these variables
through `sandbox.environment`; sandbox commands intentionally do not inherit
API keys from the Gateway process.
#### Exporting Custom Skills
Administrators can export their own custom skills from **Settings → Skills → Custom → Export**. Review the file list and declared environment requirements, then choose **Download .skill**. The archive contains the currently saved skill, including supporting files and empty directories; disabled skills can also be exported. If the skill changes after preview, refresh the file list before downloading. Import the archive on another DeerFlow instance with **Install .skill**; existing-name conflicts and normal installation security checks still apply.

View File

@ -1499,6 +1499,12 @@ sandbox:
# # DEBUG: "false"
# # API_KEY: $MY_API_KEY # Reads from host's MY_API_KEY env var
# # DATABASE_URL: $DATABASE_URL # Reads from host's DATABASE_URL env var
# # # Required when using the OpenAI-compatible image-generation provider:
# # IMAGE_GENERATION_PROVIDER: $IMAGE_GENERATION_PROVIDER
# # IMAGE_GENERATION_API_KEY: $IMAGE_GENERATION_API_KEY
# # IMAGE_GENERATION_BASE_URL: $IMAGE_GENERATION_BASE_URL
# # IMAGE_GENERATION_MODEL: $IMAGE_GENERATION_MODEL
# # IMAGE_GENERATION_SIZE: $IMAGE_GENERATION_SIZE # Optional fixed-size override
#
# # Optional: Cross-instance container ownership (issue #4206).
# #

View File

@ -178,13 +178,33 @@ For scenarios where visual accuracy is critical, **use the `image_search` tool f
This approach significantly improves generation quality by providing the model with concrete visual guidance rather than relying solely on text descriptions.
## Providers (Gemini / MiniMax)
## Providers (Gemini / MiniMax / OpenAI-compatible)
This skill auto-selects the provider by environment variables (no CLI change):
- `GEMINI_API_KEY` set → use Gemini (default, unchanged).
- Only `MINIMAX_API_KEY` set → use MiniMax (`/v1/image_generation`, model `image-01`).
- Force one explicitly with `IMAGE_GENERATION_PROVIDER=gemini|minimax`.
- Otherwise, `MINIMAX_API_KEY` set → use MiniMax (`/v1/image_generation`, model `image-01`).
- Otherwise, `IMAGE_GENERATION_API_KEY` set → use an OpenAI-compatible Images API.
- Force one explicitly with `IMAGE_GENERATION_PROVIDER=gemini|minimax|openai`.
`openai-compatible` is also accepted as an alias for `openai`.
OpenAI-compatible settings:
- `IMAGE_GENERATION_API_KEY` (required)
- `IMAGE_GENERATION_BASE_URL` (default `https://api.openai.com/v1`)
- `IMAGE_GENERATION_MODEL` (default `gpt-image-2.5-flare`)
- `IMAGE_GENERATION_SIZE` (optional fixed size override)
Text-to-image calls use `POST {base_url}/images/generations`. Reference-image calls
use multipart `POST {base_url}/images/edits`; a relay may support generation without
supporting edits. Responses may contain base64 image data, a data URL, or a downloadable
URL. Aspect ratios map to `1024x1024`, `1536x1024`, or `1024x1536` unless
`IMAGE_GENERATION_SIZE` is set. The output extension selects the API `output_format`:
`.jpg`/`.jpeg` uses `jpeg`, `.webp` uses `webp`, and all other extensions use `png`.
When `dall-e-2` or `dall-e-3` is configured instead, the request uses the model's
supported dimensions and `response_format=b64_json`; DALL-E output files must use a
`.png` extension. Reference-image editing with DALL-E models is not supported by this
skill; use the default GPT Image model for edits.
MiniMax optional overrides: `MINIMAX_API_HOST` (default `https://api.minimaxi.com`),
`MINIMAX_IMAGE_MODEL` (default `image-01`). Reference images are sent as the MiniMax

View File

@ -1,10 +1,13 @@
import base64
from contextlib import ExitStack
import json
import os
import requests
MINIMAX_DEFAULT_HOST = "https://api.minimaxi.com"
OPENAI_DEFAULT_BASE_URL = "https://api.openai.com/v1"
OPENAI_DEFAULT_MODEL = "gpt-image-2.5-flare"
# MiniMax image-01 caps the prompt at 1500 characters and rejects longer requests
# with a generic "invalid params" error, so validate before calling the API.
MINIMAX_PROMPT_MAX_CHARS = 1500
@ -25,12 +28,14 @@ def validate_image(image_path: str) -> bool:
return False
def _resolve_provider(override_env: str, existing_provider: str, has_existing_creds: bool) -> str:
def _resolve_provider(
override_env: str, existing_provider: str, has_existing_creds: bool
) -> str:
"""Pick the generation provider.
1. Explicit <SKILL>_PROVIDER override wins.
2. Otherwise prefer the existing provider when its credentials are present.
3. Otherwise fall back to MiniMax when MINIMAX_API_KEY is set.
3. Otherwise fall back to MiniMax, then the OpenAI-compatible provider.
"""
override = os.getenv(override_env)
if override:
@ -39,9 +44,12 @@ def _resolve_provider(override_env: str, existing_provider: str, has_existing_cr
return existing_provider
if os.getenv("MINIMAX_API_KEY"):
return "minimax"
if os.getenv("IMAGE_GENERATION_API_KEY"):
return "openai"
raise ValueError(
f"No credentials found. Set GEMINI_API_KEY for {existing_provider}, "
f"or MINIMAX_API_KEY for minimax (optionally force with {override_env})."
"MINIMAX_API_KEY for minimax, or IMAGE_GENERATION_API_KEY for openai "
f"(optionally force with {override_env})."
)
@ -49,6 +57,48 @@ def _minimax_host() -> str:
return os.getenv("MINIMAX_API_HOST", MINIMAX_DEFAULT_HOST).rstrip("/")
def _openai_base_url() -> str:
return os.getenv("IMAGE_GENERATION_BASE_URL", OPENAI_DEFAULT_BASE_URL).rstrip("/")
def _openai_size(aspect_ratio: str, model: str) -> str:
override = os.getenv("IMAGE_GENERATION_SIZE")
if override:
allowed_dall_e_sizes = {
"dall-e-2": {"256x256", "512x512", "1024x1024"},
"dall-e-3": {"1024x1024", "1792x1024", "1024x1792"},
}
allowed = allowed_dall_e_sizes.get(model)
if allowed is not None and override not in allowed:
supported = ", ".join(sorted(allowed))
raise ValueError(f"{model} size must be one of: {supported}")
return override
portrait = aspect_ratio in {"9:16", "2:3", "3:4"}
landscape = aspect_ratio in {"16:9", "3:2", "4:3"}
if model == "dall-e-2":
return "1024x1024"
if model == "dall-e-3":
if portrait:
return "1024x1792"
if landscape:
return "1792x1024"
return "1024x1024"
if portrait:
return "1024x1536"
if landscape:
return "1536x1024"
return "1024x1024"
def _openai_output_format(output_file: str) -> str:
extension = os.path.splitext(output_file)[1].lower()
if extension in {".jpg", ".jpeg"}:
return "jpeg"
if extension == ".webp":
return "webp"
return "png"
def _check_base_resp(payload: dict) -> None:
base = payload.get("base_resp") or {}
if base.get("status_code", 0) != 0:
@ -106,8 +156,8 @@ def _minimax_prompt(raw: str) -> str:
def _generate_image_minimax(
prompt: str, reference_images: list[str], output_file: str, aspect_ratio: str
) -> str:
api_key = os.getenv("MINIMAX_API_KEY")
if not api_key:
bearer_value = os.getenv("MINIMAX_API_KEY")
if not bearer_value:
return "MINIMAX_API_KEY is not set"
prompt = _minimax_prompt(prompt)
if len(prompt) > MINIMAX_PROMPT_MAX_CHARS:
@ -128,11 +178,15 @@ def _generate_image_minimax(
# Reference images are passed as character subjects as-is; unlike the Gemini
# path we do not pre-validate them — invalid files surface as a MiniMax API error.
body["subject_reference"] = [
{"type": "character", "image_file": _to_data_url(p)} for p in reference_images
{"type": "character", "image_file": _to_data_url(p)}
for p in reference_images
]
response = requests.post(
f"{_minimax_host()}/v1/image_generation",
headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
headers={
"Authorization": f"Bearer {bearer_value}",
"Content-Type": "application/json",
},
json=body,
timeout=60,
)
@ -160,19 +214,21 @@ def _generate_image_gemini(
print(f"Skipping invalid reference image: {ref_img}")
if len(valid_reference_images) < len(reference_images):
skipped = len(reference_images) - len(valid_reference_images)
print(f"Note: {skipped} reference image(s) were skipped due to validation failure.")
print(
f"Note: {skipped} reference image(s) were skipped due to validation failure."
)
for reference_image in valid_reference_images:
with open(reference_image, "rb") as f:
image_b64 = base64.b64encode(f.read()).decode("utf-8")
parts.append({"inlineData": {"mimeType": "image/jpeg", "data": image_b64}})
api_key = os.getenv("GEMINI_API_KEY")
if not api_key:
bearer_value = os.getenv("GEMINI_API_KEY")
if not bearer_value:
return "GEMINI_API_KEY is not set"
response = requests.post(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3-pro-image-preview:generateContent",
headers={"x-goog-api-key": api_key, "Content-Type": "application/json"},
headers={"x-goog-api-key": bearer_value, "Content-Type": "application/json"},
json={
"generationConfig": {"imageConfig": {"aspectRatio": aspect_ratio}},
"contents": [{"parts": [*parts, {"text": prompt}]}],
@ -191,6 +247,91 @@ def _generate_image_gemini(
raise Exception("Failed to generate image")
def _write_openai_image(payload: dict, output_file: str) -> str:
images = payload.get("data") or []
if not images or not isinstance(images[0], dict):
raise Exception("OpenAI-compatible provider returned no image data")
item = images[0]
encoded = item.get("b64_json") or item.get("image_base64") or item.get("base64")
if encoded:
image_bytes = base64.b64decode(encoded)
else:
image_url = item.get("url")
if not image_url:
raise Exception(
"OpenAI-compatible provider returned neither base64 data nor a URL"
)
if image_url.startswith("data:"):
_, encoded = image_url.split(",", 1)
image_bytes = base64.b64decode(encoded)
else:
download = requests.get(image_url, timeout=120)
download.raise_for_status()
image_bytes = download.content
_ensure_output_dir(output_file)
with open(output_file, "wb") as f:
f.write(image_bytes)
return f"Successfully generated image to {output_file}"
def _generate_image_openai(
prompt: str, reference_images: list[str], output_file: str, aspect_ratio: str
) -> str:
bearer_value = os.getenv("IMAGE_GENERATION_API_KEY")
if not bearer_value:
return "IMAGE_GENERATION_API_KEY is not set"
url = f"{_openai_base_url()}/images/generations"
headers = {"Authorization": f"Bearer {bearer_value}"}
model = os.getenv("IMAGE_GENERATION_MODEL", OPENAI_DEFAULT_MODEL)
is_dall_e = model in {"dall-e-2", "dall-e-3"}
if is_dall_e and os.path.splitext(output_file)[1].lower() != ".png":
raise ValueError("DALL-E output files must use a .png extension")
if is_dall_e and reference_images:
raise ValueError(
f"{model} reference-image editing is not supported by this skill"
)
fields = {
"model": model,
"prompt": prompt,
"n": 1,
"size": _openai_size(aspect_ratio, model),
}
if is_dall_e:
fields["response_format"] = "b64_json"
else:
fields["output_format"] = _openai_output_format(output_file)
if reference_images:
url = f"{_openai_base_url()}/images/edits"
with ExitStack() as stack:
files = [
(
"image[]",
(
os.path.basename(path),
stack.enter_context(open(path, "rb")),
_guess_mime(path),
),
)
for path in reference_images
]
response = requests.post(
url, headers=headers, data=fields, files=files, timeout=180
)
else:
response = requests.post(
url,
headers={**headers, "Content-Type": "application/json"},
json=fields,
timeout=180,
)
response.raise_for_status()
return _write_openai_image(response.json(), output_file)
def generate_image(
prompt_file: str,
reference_images: list[str],
@ -202,27 +343,58 @@ def generate_image(
provider = _resolve_provider(
"IMAGE_GENERATION_PROVIDER", "gemini", bool(os.getenv("GEMINI_API_KEY"))
)
if provider in ("openai", "openai-compatible"):
return _generate_image_openai(
prompt, reference_images, output_file, aspect_ratio
)
if provider == "minimax":
return _generate_image_minimax(prompt, reference_images, output_file, aspect_ratio)
return _generate_image_minimax(
prompt, reference_images, output_file, aspect_ratio
)
if provider in ("gemini", "google"):
return _generate_image_gemini(prompt, reference_images, output_file, aspect_ratio)
raise ValueError(f"Unknown image provider: {provider!r} (use 'gemini' or 'minimax')")
return _generate_image_gemini(
prompt, reference_images, output_file, aspect_ratio
)
raise ValueError(
f"Unknown image provider: {provider!r} "
"(use 'gemini', 'minimax', 'openai', or 'openai-compatible')"
)
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Generate images using Gemini or MiniMax API")
parser.add_argument("--prompt-file", required=True, help="Absolute path to JSON prompt file")
parser.add_argument("--reference-images", nargs="*", default=[],
help="Absolute paths to reference images (space-separated)")
parser.add_argument("--output-file", required=True, help="Output path for generated image")
parser.add_argument("--aspect-ratio", required=False, default="16:9",
help="Aspect ratio of the generated image")
parser = argparse.ArgumentParser(
description="Generate images using Gemini, MiniMax, or an OpenAI-compatible API"
)
parser.add_argument(
"--prompt-file", required=True, help="Absolute path to JSON prompt file"
)
parser.add_argument(
"--reference-images",
nargs="*",
default=[],
help="Absolute paths to reference images (space-separated)",
)
parser.add_argument(
"--output-file", required=True, help="Output path for generated image"
)
parser.add_argument(
"--aspect-ratio",
required=False,
default="16:9",
help="Aspect ratio of the generated image",
)
args = parser.parse_args()
try:
print(generate_image(args.prompt_file, args.reference_images,
args.output_file, args.aspect_ratio))
print(
generate_image(
args.prompt_file,
args.reference_images,
args.output_file,
args.aspect_ratio,
)
)
except Exception as e:
print(f"Error while generating image: {e}")

View File

@ -12,30 +12,61 @@ img = load("image-generation")
@pytest.fixture(autouse=True)
def clean_env(monkeypatch):
for k in ["GEMINI_API_KEY", "MINIMAX_API_KEY", "IMAGE_GENERATION_PROVIDER",
"MINIMAX_API_HOST", "MINIMAX_IMAGE_MODEL"]:
for k in [
"GEMINI_API_KEY",
"MINIMAX_API_KEY",
"IMAGE_GENERATION_PROVIDER",
"MINIMAX_API_HOST",
"MINIMAX_IMAGE_MODEL",
"IMAGE_GENERATION_API_KEY",
"IMAGE_GENERATION_BASE_URL",
"IMAGE_GENERATION_MODEL",
"IMAGE_GENERATION_SIZE",
]:
monkeypatch.delenv(k, raising=False)
def test_resolve_prefers_gemini(monkeypatch):
monkeypatch.setenv("GEMINI_API_KEY", "g")
monkeypatch.setenv("MINIMAX_API_KEY", "m")
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "gemini"
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
assert (
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "gemini"
)
def test_resolve_falls_back_to_minimax(monkeypatch):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "minimax"
assert (
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "minimax"
)
def test_resolve_falls_back_to_openai_after_minimax(monkeypatch):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
assert (
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "minimax"
)
def test_resolve_falls_back_to_openai(monkeypatch):
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
assert (
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "openai"
)
def test_resolve_override_wins(monkeypatch):
monkeypatch.setenv("GEMINI_API_KEY", "g")
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "MiniMax")
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "minimax"
assert (
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "minimax"
)
def test_resolve_errors_when_none(monkeypatch):
with pytest.raises(ValueError):
with pytest.raises(ValueError, match="IMAGE_GENERATION_API_KEY"):
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False)
@ -48,8 +79,12 @@ def test_minimax_builds_payload_and_writes(monkeypatch, tmp_path):
captured["url"] = url
captured["headers"] = headers
captured["json"] = json
return FakeResp({"data": {"image_base64": [base64.b64encode(raw).decode()]},
"base_resp": {"status_code": 0, "status_msg": "success"}})
return FakeResp(
{
"data": {"image_base64": [base64.b64encode(raw).decode()]},
"base_resp": {"status_code": 0, "status_msg": "success"},
}
)
monkeypatch.setattr(img.requests, "post", fake_post)
out = tmp_path / "o.jpg"
@ -74,8 +109,12 @@ def test_minimax_reference_image_as_data_url(monkeypatch, tmp_path):
def fake_post(url, headers=None, json=None, **kw):
captured["json"] = json
return FakeResp({"data": {"image_base64": [base64.b64encode(b"x").decode()]},
"base_resp": {"status_code": 0}})
return FakeResp(
{
"data": {"image_base64": [base64.b64encode(b"x").decode()]},
"base_resp": {"status_code": 0},
}
)
monkeypatch.setattr(img.requests, "post", fake_post)
ref = tmp_path / "ref.jpg"
@ -88,6 +127,7 @@ def test_minimax_reference_image_as_data_url(monkeypatch, tmp_path):
assert subj[0]["type"] == "character"
assert subj[0]["image_file"].startswith("data:image/jpeg;base64,")
import base64 as _b64
encoded = subj[0]["image_file"].split(",", 1)[1]
assert _b64.b64decode(encoded) == b"\xff\xd8refbytes"
@ -96,7 +136,9 @@ def test_minimax_raises_on_base_resp_error(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
def fake_post(url, headers=None, json=None, **kw):
return FakeResp({"base_resp": {"status_code": 1004, "status_msg": "auth failed"}})
return FakeResp(
{"base_resp": {"status_code": 1004, "status_msg": "auth failed"}}
)
monkeypatch.setattr(img.requests, "post", fake_post)
prompt_file = tmp_path / "p.json"
@ -112,8 +154,12 @@ def test_minimax_extracts_json_prompt_field(monkeypatch, tmp_path):
def fake_post(url, headers=None, json=None, **kw):
captured["json"] = json
return FakeResp({"data": {"image_base64": [base64.b64encode(b"x").decode()]},
"base_resp": {"status_code": 0}})
return FakeResp(
{
"data": {"image_base64": [base64.b64encode(b"x").decode()]},
"base_resp": {"status_code": 0},
}
)
monkeypatch.setattr(img.requests, "post", fake_post)
prompt_file = tmp_path / "p.json"
@ -135,8 +181,12 @@ def test_minimax_plaintext_prompt_passes_through(monkeypatch, tmp_path):
def fake_post(url, headers=None, json=None, **kw):
captured["json"] = json
return FakeResp({"data": {"image_base64": [base64.b64encode(b"x").decode()]},
"base_resp": {"status_code": 0}})
return FakeResp(
{
"data": {"image_base64": [base64.b64encode(b"x").decode()]},
"base_resp": {"status_code": 0},
}
)
monkeypatch.setattr(img.requests, "post", fake_post)
prompt_file = tmp_path / "p.txt"
@ -167,8 +217,12 @@ def test_minimax_creates_nested_output_dir(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
def fake_post(url, headers=None, json=None, **kw):
return FakeResp({"data": {"image_base64": [base64.b64encode(b"img").decode()]},
"base_resp": {"status_code": 0}})
return FakeResp(
{
"data": {"image_base64": [base64.b64encode(b"img").decode()]},
"base_resp": {"status_code": 0},
}
)
monkeypatch.setattr(img.requests, "post", fake_post)
prompt_file = tmp_path / "p.txt"
@ -180,7 +234,7 @@ def test_minimax_creates_nested_output_dir(monkeypatch, tmp_path):
def test_unknown_provider_raises(monkeypatch, tmp_path):
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "unknown")
monkeypatch.setenv("GEMINI_API_KEY", "g")
pf = tmp_path / "p.json"
pf.write_text("x", encoding="utf-8")
@ -188,6 +242,183 @@ def test_unknown_provider_raises(monkeypatch, tmp_path):
img.generate_image(str(pf), [], str(tmp_path / "o.jpg"), "1:1")
def test_openai_compatible_generation_writes_base64(monkeypatch, tmp_path):
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
monkeypatch.setenv("IMAGE_GENERATION_BASE_URL", "https://models.example/v1/")
captured = {}
def fake_post(url, headers=None, json=None, **kwargs):
captured.update(url=url, headers=headers, json=json, kwargs=kwargs)
return FakeResp({"data": [{"b64_json": base64.b64encode(b"image").decode()}]})
monkeypatch.setattr(img.requests, "post", fake_post)
prompt_file = tmp_path / "prompt.json"
prompt_file.write_text('{"prompt": "a red deer"}', encoding="utf-8")
output_file = tmp_path / "nested" / "image.png"
result = img.generate_image(str(prompt_file), [], str(output_file), "16:9")
assert captured["url"] == "https://models.example/v1/images/generations"
assert captured["headers"]["Authorization"] == "Bearer image-key"
assert captured["json"]["model"] == "gpt-image-2.5-flare"
assert captured["json"]["size"] == "1536x1024"
assert captured["json"]["output_format"] == "png"
assert "response_format" not in captured["json"]
assert output_file.read_bytes() == b"image"
assert "Successfully generated image" in result
def test_openai_compatible_generation_downloads_url(monkeypatch, tmp_path):
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai-compatible")
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
monkeypatch.setattr(
img.requests,
"post",
lambda *args, **kwargs: FakeResp(
{"data": [{"url": "https://cdn.example/image.png"}]}
),
)
monkeypatch.setattr(
img.requests,
"get",
lambda *args, **kwargs: FakeResp(content=b"downloaded-image"),
)
prompt_file = tmp_path / "prompt.txt"
prompt_file.write_text("a red deer", encoding="utf-8")
output_file = tmp_path / "image.png"
img.generate_image(str(prompt_file), [], str(output_file), "1:1")
assert output_file.read_bytes() == b"downloaded-image"
def test_openai_compatible_dall_e_uses_response_format(monkeypatch, tmp_path):
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
monkeypatch.setenv("IMAGE_GENERATION_MODEL", "dall-e-3")
captured = {}
def fake_post(url, headers=None, json=None, **kwargs):
captured["json"] = json
return FakeResp({"data": [{"b64_json": base64.b64encode(b"image").decode()}]})
monkeypatch.setattr(img.requests, "post", fake_post)
prompt_file = tmp_path / "prompt.txt"
prompt_file.write_text("a red deer", encoding="utf-8")
img.generate_image(str(prompt_file), [], str(tmp_path / "image.png"), "1:1")
assert captured["json"]["response_format"] == "b64_json"
assert "output_format" not in captured["json"]
@pytest.mark.parametrize(
("model", "aspect_ratio", "expected"),
[
("gpt-image-2.5-flare", "16:9", "1536x1024"),
("gpt-image-2.5-flare", "9:16", "1024x1536"),
("dall-e-3", "16:9", "1792x1024"),
("dall-e-3", "9:16", "1024x1792"),
("dall-e-2", "16:9", "1024x1024"),
],
)
def test_openai_size_matches_model(model, aspect_ratio, expected):
assert img._openai_size(aspect_ratio, model) == expected
def test_openai_size_override_wins(monkeypatch):
monkeypatch.setenv("IMAGE_GENERATION_SIZE", "2048x1024")
assert img._openai_size("1:1", "gpt-image-2.5-flare") == "2048x1024"
def test_openai_dall_e_rejects_invalid_size_override(monkeypatch):
monkeypatch.setenv("IMAGE_GENERATION_SIZE", "1536x1024")
with pytest.raises(ValueError, match="dall-e-3 size must be one of"):
img._openai_size("16:9", "dall-e-3")
@pytest.mark.parametrize("extension", [".jpg", ".webp", ".unknown"])
def test_openai_dall_e_rejects_non_png_output(monkeypatch, tmp_path, extension):
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
monkeypatch.setenv("IMAGE_GENERATION_MODEL", "dall-e-3")
monkeypatch.setattr(
img.requests,
"post",
lambda *args, **kwargs: pytest.fail("request must not be sent"),
)
prompt_file = tmp_path / "prompt.txt"
prompt_file.write_text("a red deer", encoding="utf-8")
with pytest.raises(ValueError, match=r"\.png extension"):
img.generate_image(
str(prompt_file), [], str(tmp_path / f"image{extension}"), "1:1"
)
@pytest.mark.parametrize("model", ["dall-e-2", "dall-e-3"])
def test_openai_dall_e_rejects_reference_images(monkeypatch, tmp_path, model):
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
monkeypatch.setenv("IMAGE_GENERATION_MODEL", model)
monkeypatch.setattr(
img.requests,
"post",
lambda *args, **kwargs: pytest.fail("request must not be sent"),
)
prompt_file = tmp_path / "prompt.txt"
prompt_file.write_text("add snow", encoding="utf-8")
reference = tmp_path / "reference.png"
reference.write_bytes(b"png")
with pytest.raises(ValueError, match="reference-image editing is not supported"):
img.generate_image(
str(prompt_file), [str(reference)], str(tmp_path / "image.png"), "1:1"
)
def test_openai_compatible_reference_images_use_edits(monkeypatch, tmp_path):
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "openai")
monkeypatch.setenv("IMAGE_GENERATION_API_KEY", "image-key")
captured = {}
def fake_post(url, headers=None, data=None, files=None, **kwargs):
captured.update(url=url, data=data, files=files)
return FakeResp({"data": [{"b64_json": base64.b64encode(b"edited").decode()}]})
monkeypatch.setattr(img.requests, "post", fake_post)
prompt_file = tmp_path / "prompt.txt"
prompt_file.write_text("add snow", encoding="utf-8")
reference = tmp_path / "reference.png"
reference.write_bytes(b"png")
output_file = tmp_path / "image.png"
img.generate_image(str(prompt_file), [str(reference)], str(output_file), "9:16")
assert captured["url"].endswith("/images/edits")
assert captured["data"]["size"] == "1024x1536"
assert captured["files"][0][0] == "image[]"
assert output_file.read_bytes() == b"edited"
@pytest.mark.parametrize(
("filename", "expected"),
[
("image.png", "png"),
("image.jpg", "jpeg"),
("image.jpeg", "jpeg"),
("image.webp", "webp"),
("image.unknown", "png"),
],
)
def test_openai_output_format_matches_filename(filename, expected):
assert img._openai_output_format(filename) == expected
def test_guess_mime_by_extension():
assert img._guess_mime("/a/b.png") == "image/png"
assert img._guess_mime("/a/b.webp") == "image/webp"