* ✨ Render email templates inside the mail job
The web process rendered each email to full subject and HTML body
before submitting it, so every queued job carried a huge params
payload. The queue now stores only the template id plus the raw
template data, and the new :send-templated-mail job renders and
sends the email inside the worker.
The old :sendmail job stays in place to drain jobs submitted in the
previous format; the quota notice and the REPL test mail keep using
it. All templated callers (auth, profile, feedback, invitations,
comments, Nitrate) submit recipes through eml/send-template.
Closes#12445
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Unify templated mail on template id strings
The ::eml/template value was a namespaced keyword whose name had to
match the registry key, with one live mismatch (user-feedback vs
feedback). Callers now pass the registry id string itself, so keys
stay ::eml/-namespaced and values stay plain data, and the single
templates set rejects unsendable mail both at submit and in the
worker.
Closes#12445
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Encode templated mail data with the template schema
The job params column is plain JSON, so the URIs of an organization
(:logo, :avatar-bg-url) were stored as record maps and the template
check rejected them in the worker. Invites for organization teams,
organization invites and renewal notices failed without a retry.
Each registry entry now carries the JSON encoder and decoder of its
template schema: send-template encodes the data before submit and the
worker decodes it before rendering. The job schema keeps only the mail
envelope, as the template schemas own the typed data.
Closes#12445
AI-assisted-by: claude-opus-5-5
---------
* 🎉 Move backend jobs to the unified job table substrate
Replace the generic 'task' table with a unified 'job' table as the
durable substrate for all backend jobs, and migrate every producer
and consumer to it. The legacy 'task' table stops receiving writes
but stays dormant (still cleaned by tasks-gc); upload-session
chunking and the other post-branch upstream changes are preserved.
New 0154 migration creates the 'job' table: dispatch and lifecycle
columns (name, queue, priority, scheduled_at, retries, status,
timestamps, props) plus optional user-facing ledger columns
(profile_id, target, progress, error, result, resource_id,
expires_at). No modified_at trigger: the application updates it on
every write for lease and orphan detection. Partial indexes cover
the dispatcher claim query, orphan scanning, and per-profile
listing.
New app.jobs public API: submit!/invoke!, heartbeats and progress
reporting, a precompiled job-def registry (name, schema, handler,
decoder, validator), an ephemeral request/response mode, and a
management API for external workers. The dispatcher claims
new/retry rows (FOR UPDATE SKIP LOCKED) with a plain JSON redis
payload and marks lease-expired running rows as orphaned; the
runner executes per-queue with the same retry conventions.
New jobs GC (daily): deletes expired rows and retains internal
terminal rows per :jobs-retention, touching referenced storage
objects so storage-gc-touched reclaims them. Job resources live in
a dedicated job-resource bucket with a reclaim branch in
storage-gc-touched.
All production call-sites (sendmail, delete-object, demo-purge,
file-gc, offload, webhooks, quotes, nitrate bulk delete,
snapshots, telemetry) and the ten cron tasks now submit through
app.jobs; cron is a pure scheduler and the legacy worker registry,
task-table inserts, and per-namespace handlers are removed.
Includes migrations, dispatcher, runner, jobs, request/response,
jobs GC, management API, and cron test suites.
AI-assisted-by: glm-5.3-flash
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: kimi-k3
AI-assisted-by: mimo-v2.5
* ✨ Address review comments on jobs substrate
Resolves the four open review threads on the jobs PR. Orphaned jobs
are now marked aborted, a system-side terminal state with no retry:
the dispatcher sweep stores the end event with outcome aborted in a
single bulk insert in the same transaction, records the terminal
outcome next to the orphaned counter, and reports an error log;
orphans keep alerting through the log and the counter, with no
user notification.
The aborted state is terminal everywhere: the status CHECK, the
sweep, the lifecycle writers, the jobs GC retention, the backlog
gauges and the user status mapping, with tests covering each
transition. The claim comment, the 2.20 doc version and the legacy
task table notes are corrected, and objects-gc heartbeats per chunk.
AI-assisted-by: muse-spark-1.3-contributor
Prevent email bombing attacks on the send-user-feedback endpoint by
limiting the error-report field to 1MiB and adding climit rate limits:
by-profile (1 permit, queue 3) and global (4 permits), configured in
climit.edn. Make the schema public so it can be exercised by tests,
and add schema validation tests covering the new size limit.
AI-assisted-by: qwen3.7-plus
