* 🐛 Revoke websocket subscriptions when access is revoked GHSA-m53j-2766-6jqw. A websocket subscription was authorized once, when it opened, and that decision was trusted for the rest of the connection's life, so a member whose access was revoked kept receiving live file content as long as the tab stayed open. The revocation travels over the message bus instead of closing the socket from the mutation: the connection registry is local to one backend instance, while the RPC that revokes access runs on whichever instance received the request. Every instance re-verifies the subscriptions it owns with the same `has-read-permissions?` predicates that `:subscribe-file` and `:subscribe-team` authorize with, so a downgrade to viewer keeps its subscription and a non-member organization owner keeps read-only access. Two mechanisms cover each other: - Access-changing mutations announce after the write. delete-team- member, leave-team, update-team-member-role, delete-file, delete-project, move-files, move-project and delete-team announce. delete-share-link is deliberately excluded: a share-link visitor never passes the subscribe permission check, so there is no subscription to revoke. Team-wide changes publish one event per team; the watcher resolves the members itself, and both publishers defer through `db/after-commit` so the watcher never decides from pre-commit state. - Every relay re-checks access every 5 minutes by default (`subscription-revalidation-interval`), driven from the relay loop with `alts!` over the subscription channel and a one-shot tick channel, so a steady stream of edits cannot push the check away forever. Supporting changes: - The registry holds `:connections` and `:by-profile` in one atom, kept consistent by register/unregister in a single swap!; a profile lookup is one map access, and a profile whose last connection leaves is dropped from the index. - `close-file-subscription` and `close-team-subscription` announce the :leave-* message, close the relay channel and purge the bus subscription, and are idempotent. Closing the channel ends the relay loop and its re-check task. - The permission query the re-check uses filters soft-deleted files, projects and teams, matching its sibling queries. - A transient database error fails open: it logs and keeps the subscription; the next tick retries. - The revocation watcher's params schema declared the nitrate client as a bare keyword inside [:map ...], which malli resolved as a registry lookup and which took the whole system build down under enable-backend-asserts. It is now an explicit pair, optional because init-key yields nil with the admin-console flag off. The tests reproduce the leak against the real message bus, including the negative control for the interval re-check. The written records live in mem:backend/websocket-subscriptions. Closes #12155 AI-assisted-by: space-bunny-free AI-assisted-by: muse-spark-1.3-contributor * 🐛 Mirror the subscribe gates in the revocation re-checks Addresses the second review round on PR #12189 (pabloalba). The re-checks only asked whether the profile could read. The membership legs of sql:file-permissions filter soft-deleted rows since the F2 fix, but the organization-owner fallback grants read-only access over HTTP without looking at deleted_at, so a soft-deleted file or team kept an expelled org owner subscribed indefinitely. The re-checks now gate on existence as well: file-readable? and team-readable? require a live row (db/get*, non-throwing, soft-deleted rows excluded) plus the read permission, mirroring the :subscribe-file gates and extending them to teams. Both relays and revalidate-profile-subscriptions use them; the subscribe handlers keep their exact behavior. Also from the same review: - watcher-consumes-revocation-events-from-the-bus no longer races the command's after-commit announcement: it takes access away with a direct row delete and drives the watcher with a synthetic bus event, instead of deleting through the command and publishing a second notify-* that would mask a broken command announcement. - moving-a-project-closes-source-team-subscriptions pins the move-project e2e: the file subscription drops, the source-team subscription survives (membership unchanged). - soft-deleted-file/team-closes-org-owner-subscription cover the new gate with red-before/green-after tests. mem:backend/websocket-subscriptions updated. Closes #12155 AI-assisted-by: inkling-2-small-free * 🐛 Refuse dead teams in resolve-team-id (PR #12189 review) Second review round on PR #12189 (pabloalba). The file re-check added in 87ca7aca53 stops at the file row, but the organization-owner fallback grants read-only access to non-member org owners via Nitrate over HTTP, which knows nothing about Penpot soft-deletion. resolve-team-id resolved the team from unfiltered SQL, so an expelled org owner kept the file subscription open for an indefinitely soft-deleted file, project or team. Now both SQL lookups filter deleted_at, and resolve-team-id checks the team row itself (db/get*, which filters soft-deleted rows) for every caller — teams, projects and files alike. Also from the same review: - file-readable? now uses the same sql:get-minimal-file as the subscribe gate (non-throwing via throw-if-not-exists false), and team-readable? projects id and deleted-at, so db/get* can filter. - The test suite extracts the thrice-duplicated raw delete into delete-team-membership!; a sibling of the fail-open pin now redefs the existence probe to throw; memory and docstring aligned with the code. - docs/technical-guide/developer/architecture/backend.md attributed the websocket to notifications.clj; it now names app.http.websocket as the owner and notifications.clj as the revocation publisher. - mem:backend/auth-permissions-product-domains documents the fallback mechanism and the resolve-team-id contract. Closes #12155 AI-assisted-by: inkling-2-small-free * 🐛 Test resolve-team-id refusal and tidy review findings Third pass over PR #12189 after review-code on 225eb98de2. resolve-team-id now has a direct test: all three branches resolve while rows are live, and file, project and team soft-deletes each refuse their branch — the two new SQL predicates and the final team-row check are all pinned by it. From the same review: - failing-existence-check-keeps-the-subscription and the sibling a-failing-check-keeps-the-subscription are deleted: they could not distinguish a live relay from a dead one. The fail-open policy remains documented in the docstring and the memory; the memory no longer claims the pins exist. - team-readable? projects ::sql/columns like the rest of the backend; the fully-qualified keyword went away with the new alias. - The dead when-not guard in get-organization-owner-viewer-team is removed (resolve-team-id guarantees the row live); the let binding in resolve-team-id no longer shadows the destructured team-id. - docs/technical-guide names rpc/notifications.clj as the publisher and the tree lists the real top-level modules. Closes #12155 AI-assisted-by: inkling-2-small-free * 🐛 Make the revocation tests deterministic Third review round on PR #12189 (pabloalba). Both tests took access away through a command that announces, so the live watcher could close the subscriptions before the test looked: - revocation-closes-subscriptions-that-are-no-longer-authorized asserted an exact count of 2 closes after delete-team-member, and that count reads 0 whenever the watcher got there first. It now removes the member with delete-team-membership!, the raw row delete that publishes nothing, and asserts both subscriptions are still open before calling revalidate-profile-subscriptions. - team-event-closes-every-member-subscription ran delete-team and then published a second notify-team-permissions-changed, so a missing announcement in delete-team still passed. It now soft-deletes the team row (sql:team-members joins team_profile_rel and profile only, so the watcher still resolves every member), asserts nothing closed, and drives one synthetic team event. The command paths stay covered by the announce and end-to-end tests for delete-team-member and delete-team. The file split the reviewer suggested stays the optional follow-up, tracked outside this PR and waiting for the merge. Closes #12155 AI-assisted-by: deepseek-v4.1-flash
6.6 KiB
title, desc
| title | desc |
|---|---|
| Backend app | Dive into self-hosting, configuration, developer insights (architecture, data model), integration, and troubleshooting. See Penpot's Technical Guide. |
Backend app
This app is in charge of CRUD of data, integrity validation and persistence into a database and also into a file system for media attachments.
To handle deletions it uses a garbage collector mechanism: no object in the
database is deleted instantly. Instead, a field deleted_at is set with the
date and time of the deletion, and every query ignores db rows that have this
field set. Then, an async task that runs periodically, locates rows whose
deletion date is older than a given threshold and permanently deletes them.
For this, and other possibly slow tasks, there is an internal async tasks worker, that may be used to queue tasks to be scheduled and executed when the backend is idle. Other tasks are email sending, collecting data for telemetry and detecting unused media attachment, for removing them from the file storage.
Backend structure
Penpot backend app code resides under backend/src/app path in the main repository.
@startuml BackendGeneral !include https://raw.githubusercontent.com/plantuml-stdlib/C4-PlantUML/master/C4_Container.puml !define DEVICONS https://raw.githubusercontent.com/tupadr3/plantuml-icon-font-sprites/master/devicons !include DEVICONS/react.puml !include DEVICONS/java.puml !include DEVICONS/clojure.puml !include DEVICONS/postgresql.puml !include DEVICONS/redis.puml !include DEVICONS/chrome.puml
HIDE_STEREOTYPE()
Container(frontend_app, "Frontend app", "React / ClojureScript", "", "react")
System_Boundary(backend, "Backend") { Container(backend_app, "Backend app", "Clojure / JVM", "", "clojure") ContainerDb(db, "Database", "PostgreSQL", "", "postgresql") ContainerDb(redis, "Broker", "Redis", "", "redis") }
BiRel(frontend_app, backend_app, "Open", "websocket") Rel(frontend_app, backend_app, "Uses", "RPC API") Rel(backend_app, db, "Uses", "SQL") Rel(redis, backend_app, "Subscribes", "pub/sub") Rel(backend_app, redis, "Notifies", "pub/sub")
@enduml
▾ backend/src/app/
▸ cli/
▸ http/
▸ migrations/
▸ rpc/
▸ setup/
▸ srepl/
▸ util/
▸ tasks/
main.clj
config.clj
health.clj
http.clj
jobs.clj
metrics.clj
migrations.clj
nitrate.clj
rpc.clj
setup.clj
srepl.clj
system.clj
worker.clj
...
main.cljdefines the app global settings and the main entry point of the application, served by a JVM.config.cljdefines of the configuration options read from linux environment.httpcontains the HTTP server and the backend routes list.migrationscontains the SQL scripts that define the database schema, in the form of a sequence of migrations.rpcis the main module to handle the RPC API calls.http/websocket.cljmanages the websocket endpoint: clients subscribe to open files or teams and the server keeps the subscriptions; revoked access is re-checked and dropped.rpc/notifications.cljpublishes access-change events on the bus so the websocket re-checks the subscriptions it owns.setupinitializes the environment (loads config variables, sets up the database, executes migrations, loads initial data, etc).sreplsets up an interactive REPL shell, with some useful commands to be used to debug a running instance.clisets a command-line interface, with some more maintenance commands.metrics.cljhas some interceptors that watches RPC calls, calculate statistics and other metrics, and send them to external systems to store and analyze.worker.cljandtasksdefine some async tasks that are executed in parallel to the main http server (using java threads), and scheduled in a cron-like table. They are useful to do some garbage collection, data packing and similar periodic maintenance tasks.db.clj,emails.clj,media.clj,msgbus.clj,storage.clj,rlimits.cljare general libraries to use I/O resources (SQL database, send emails, handle multimedia objects, use REDIS messages, external file storage and semaphores).util/has a collection of generic utility functions.
RPC calls
The RPC (Remote Procedure Call) subsystem consists of a mechanism that allows to expose clojure functions as an HTTP endpoint. We take advantage of being using Clojure at both front and back ends, to avoid needing complex data conversions.
- Frontend initiates a "query" or "mutation" call to
:xxxmethod, and passes a Clojure object as params. - Params are string-encoded using transit, a format similar to JSON but more powerful.
- The call is mapped to
/api/rpc/query/xxxor/api/rpc/mutation/xxx. - The
rpcmodule receives the call, decode the parameters and executes the corresponding method insidesrc/app/rpc/queries/orsrc/app/rpc/mutations/. We have created adefmethodmacro to declare an RPC method and its parameter specs. - The result value is also transit-encoded and returned to the frontend.
This way, frontend can execute backend calls like it was calling an async function, with all the power of Clojure data structures.
PubSub
To manage subscriptions to a file, to be notified of changes, we use a redis server as a pub/sub broker. Whenever a user visits a file and opens a websocket, the backend creates a subscription in redis, with a topic that has the id of the file. If the user sends any change to the file, backend sends a notification to this topic, that is received by all subscribers. Then the notification is retrieved and sent to the user via the websocket.