mirror of
https://github.com/penpot/penpot.git
synced 2026-10-30 06:20:32 +00:00
a53481c3b7
* ✨ Enrich backend audit events with team and entity ids Backend audit events now carry the data a consumer needs to rebuild state without joining production tables: - prepare-rpc-event adds :team-id when the event carries a project or file id; commands that already loaded the team set it themselves. - add-profile-to-team! emits add-team-member on every real insert. - create-project and create-file emit the new row id. - clone-template, import-binfile and duplicate-project emit one event per created entity. - both binfile versions return {:file-ids [...]} from import-files!. Closes #12219 AI-assisted-by: deepseek-v4.1-flash * ✨ Emit delete and hard-delete audit events with entity data The event stream must let Nexus rebuild local state without joining production tables. This completes the enrichment with the delete paths: - delete-team keeps returning nil but now carries the deleted row (id, name, timestamps) in the audit props, like delete-project. - create-team adds created-at to its audit props. - delete-profile carries id, email, fullname and deleted-at; its props used to be empty because every param is namespaced. - objects-gc emits hard-delete-team/project/file per purged batch, with the deleted ids, only when the audit log is on. - delete-team-member needs no change: the delete is physical and the params already carry team-id and member-id. The nitrate push-audit-events tests now drop the membership events emitted by their own profile setup before counting. Closes #12219 AI-assisted-by: muse-spark-1.3-free * ✨ Announce cascade deletes and missing ids in audit events The :delete-object worker task marked teams, projects and files without leaving any event, so an event-sourced projection kept them alive for weeks. It now emits one cascade-deleted event per execution with every affected id, in the same transaction, only with :audit-log on. Rows purged by objects-gc and snapshot cascades stay silent on purpose. Also: create-team-with-invitations carries the new team id, and update-team-member-role names the demoted member on owner transfers. Closes #12219 AI-assisted-by: muse-spark-1.3-free * 🐛 Identify the SREPL profile delete in audit events The SREPL delete-profile! emitted no profile-id and empty props, so the row could not be attributed by any consumer. It now carries the profile id in both fields. delete-team! named the wrong cause in its context. Closes #12219 AI-assisted-by: muse-spark-1.3-free * 🐛 Identify the SREPL profile restore in audit events The SREPL restore-profile! emitted no profile-id, so the consumer keyed on the zero profile instead of the restored one. It now carries the profile id like a normal profile event, and no consumer special case is needed. Closes #12219 AI-assisted-by: muse-spark-1.3-free * 🐛 Stop attributing SREPL events to the subject profile profile-id names the actor, and SREPL calls have none: setting the subject id blamed the user for an admin action and would have leaked those rows into PostHog as user activity. The subject id stays in props, where the consumer resolves it, and the event profile stays zero. Closes #12219 AI-assisted-by: muse-spark-1.3-free * ✨ Expose file analysis data for the nexus pipeline The nexus file-security pipeline needs file content and scheduling state from the backend. This adds a read-only management method, `get-file-for-analysis`, returning the semantic projection of one file (pages, frames, texts, links, interactions; ids as strings; capped with a truncated flag), plus identity, revn, timestamps and share state. No prior projection code existed, so the method ships its own pure `file->projection`. Audit events now carry what the `:local` consumer schedules from: revn and modified-at on update-file, name and timestamps on every create-file emitter, deleted-at on delete-file, and new create/delete-share-link events with file and link ids. AI-assisted-by: muse-spark-1.3-contributor * ♻️ Rename deletion audit events to action names Event names are actions, not past tense: `cascade-deleted` becomes `delete-object-with-cascade` with the same shape and transaction. The three `hard-delete-*` events merge into `hard-delete-objects`: one event per purged chunk carrying only that chunk's entity key (`:team-ids`, `:project-ids` or `:file-ids`), because each GC section purges in its own chunk transactions and a single event could never share them all. Every submit stays inside its delete transaction, `triggered-by` stays annotated, and silent paths stay silent. AI-assisted-by: muse-spark-1.3-contributor * 🐛 Read worker parallelism knobs with plain config keys The runner wiring looked up `:app.main/worker-*-parallelism` (namespaced) while the schema and the environment provide plain keys, so `PENPOT_WORKER_DEFAULT_PARALLELISM` and `PENPOT_WORKER_WEBHOOK_PARALLELISM` never applied and workers always ran with the fallback. Read the keys the config holds. AI-assisted-by: muse-spark-1.3-contributor * 🐛 Fix binfile audit test with a real file row `submit-file-created-event` reads the light columns back from the row, so the mocked importer must return an existing file id: a phantom id emits no event and the test saw zero `create-file` events. Create the file first and use its id. The audit-log memory now tells both bulk-event helpers apart. AI-assisted-by: muse-spark-1.3-contributor * 🐛 Fix audit event fidelity from review findings The review of the audit enrichment found data-fidelity gaps; fix them while the branch is fresh. Membership events now belong to the actor: the flows where the owner adds somebody else pass an explicit actor id, the role is derived from the flags (the table has no role column), and the event carries the request context like its siblings. Also: team-id resolution skips read events, duplicate-project emits the project name and timestamps, delete-profile is idempotent again, the GC tests assert physical purge, and get-file-for-analysis leaves an audit trace. AI-assisted-by: muse-spark-1.3-contributor * ♻️ Nest cascade ids under :cascade in delete event The cascade event carried every affected id flat next to the object identity; nest them under `:cascade` so the payload reads as header plus affected set. The event is an action like its siblings, not a command. Drop the `:audit-log` flag re-check: `submit` already fans out per flag. AI-assisted-by: muse-spark-1.3-contributor * 📎 Update copyright name on file header The company name changed; a single header in this branch still carried the old one. Align it with the current form. AI-assisted-by: muse-spark-1.3-contributor * 🐛 Fix nil event-id on webhook trigger audit rows process-event generated the row id inside append-audit-entry, so the :process-webhook-event task never received it and the trigger row stored a nil :event-id. Assign the id up-front and share it between the stored row and the worker params; the telemetry shadow row keeps its own id to avoid a PK collision. Add regression tests covering the handoff and the end-to-end trigger row. AI-assisted-by: muse-spark-1.3-contributor
403 lines
16 KiB
Clojure
403 lines
16 KiB
Clojure
;; This Source Code Form is subject to the terms of the Mozilla Public
|
|
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
;;
|
|
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
|
|
|
(ns backend-tests.rpc-webhooks-test
|
|
(:require
|
|
[app.common.uri :as u]
|
|
[app.common.uuid :as uuid]
|
|
[app.db :as db]
|
|
[app.http :as http]
|
|
[app.rpc :as-alias rpc]
|
|
[app.storage :as sto]
|
|
[backend-tests.helpers :as th]
|
|
[clojure.test :as t]
|
|
[mockery.core :refer [with-mocks]]))
|
|
|
|
(t/use-fixtures :once th/state-init)
|
|
(t/use-fixtures :each th/database-reset)
|
|
|
|
(defn create-webhook-params [id team]
|
|
{::th/type :create-webhook
|
|
::rpc/profile-id id
|
|
:team-id team
|
|
:uri (u/uri "http://example.com")
|
|
:mtype "application/json"})
|
|
|
|
(defn check-webhook-format
|
|
([result]
|
|
(t/is (contains? result :id))
|
|
(t/is (contains? result :team-id))
|
|
(t/is (contains? result :created-at))
|
|
(t/is (contains? result :profile-id))
|
|
(t/is (contains? result :updated-at))
|
|
(t/is (contains? result :uri))
|
|
(t/is (contains? result :mtype))))
|
|
|
|
(t/deftest webhook-crud
|
|
(with-mocks [http-mock {:target 'app.http.client/req
|
|
:return {:status 200}}]
|
|
|
|
(let [prof (th/create-profile* 1 {:is-active true})
|
|
team-id (:default-team-id prof)
|
|
proj-id (:default-project-id prof)
|
|
whook (volatile! nil)]
|
|
|
|
(t/testing "create webhook"
|
|
(let [params {::th/type :create-webhook
|
|
::rpc/profile-id (:id prof)
|
|
:team-id team-id
|
|
:uri (u/uri "http://example.com")
|
|
:mtype "application/json"}
|
|
out (th/command! params)]
|
|
|
|
(t/is (nil? (:error out)))
|
|
(t/is (= 1 (:call-count @http-mock)))
|
|
|
|
(let [result (:result out)]
|
|
(check-webhook-format result)
|
|
|
|
(t/is (= (:uri params) (:uri result)))
|
|
(t/is (= (:team-id params) (:team-id result)))
|
|
(t/is (= (:mtype params) (:mtype result)))
|
|
(vreset! whook result))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "update webhook 1 (success)"
|
|
(let [params {::th/type :update-webhook
|
|
::rpc/profile-id (:id prof)
|
|
:id (:id @whook)
|
|
:uri (:uri @whook)
|
|
:mtype "application/transit+json"
|
|
:is-active false}
|
|
out (th/command! params)]
|
|
|
|
(t/is (nil? (:error out)))
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
|
|
(let [result (:result out)]
|
|
(check-webhook-format result)
|
|
|
|
(t/is (= (:id params) (:id result)))
|
|
(t/is (= (:id @whook) (:id result)))
|
|
(t/is (= (:uri params) (:uri result)))
|
|
(t/is (= (:team-id @whook) (:team-id result)))
|
|
(t/is (= (:mtype params) (:mtype result))))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "update webhook 2 (change uri)"
|
|
(let [params {::th/type :update-webhook
|
|
::rpc/profile-id (:id prof)
|
|
:id (:id @whook)
|
|
:uri (str (:uri @whook) "/test")
|
|
:mtype "application/transit+json"
|
|
:is-active false}
|
|
out (th/command! params)]
|
|
|
|
(t/is (nil? (:error out)))
|
|
(t/is (map? (:result out)))
|
|
(t/is (= 1 (:call-count @http-mock)))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "update webhook 3 (not authorized)"
|
|
(let [params {::th/type :update-webhook
|
|
::rpc/profile-id uuid/zero
|
|
:id (:id @whook)
|
|
:uri (str (:uri @whook) "/test")
|
|
:mtype "application/transit+json"
|
|
:is-active false}
|
|
out (th/command! params)]
|
|
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
(let [error (:error out)
|
|
error-data (ex-data error)]
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :not-found))
|
|
(t/is (= (:code error-data) :object-not-found)))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "delete webhook (success)"
|
|
(let [params {::th/type :delete-webhook
|
|
::rpc/profile-id (:id prof)
|
|
:id (:id @whook)}
|
|
out (th/command! params)]
|
|
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
(t/is (nil? (:error out)))
|
|
(t/is (nil? (:result out)))
|
|
|
|
(let [rows (th/db-exec! ["select * from webhook"])]
|
|
(t/is (= 0 (count rows))))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "delete webhook (unauthorized)"
|
|
(let [params {::th/type :delete-webhook
|
|
::rpc/profile-id uuid/zero
|
|
:id (:id @whook)}
|
|
out (th/command! params)]
|
|
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
(let [error (:error out)
|
|
error-data (ex-data error)]
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :not-found))
|
|
(t/is (= (:code error-data) :object-not-found))))))))
|
|
|
|
(t/deftest webhooks-permissions-crud-viewer-only
|
|
(with-mocks [http-mock {:target 'app.http.client/req
|
|
:return {:status 200}}]
|
|
(let [owner (th/create-profile* 1 {:is-active true})
|
|
viewer (th/create-profile* 2 {:is-active true})
|
|
team (th/create-team* 1 {:profile-id (:id owner)})]
|
|
(th/create-team-role* {:team-id (:id team)
|
|
:profile-id (:id viewer)
|
|
:role :viewer})
|
|
;; Assert all roles for team
|
|
(let [roles (th/db-query :team-profile-rel {:team-id (:id team)})]
|
|
(t/is (= 2 (count roles))))
|
|
|
|
(t/testing "viewer cannot create a webhook (requires editor role)"
|
|
(let [viewers-webhook (create-webhook-params (:id viewer) (:id team))
|
|
out (th/command! viewers-webhook)]
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
(let [error (:error out)
|
|
error-data (ex-data error)]
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :not-found))
|
|
(t/is (= (:code error-data) :object-not-found)))))
|
|
|
|
(th/reset-mock! http-mock))))
|
|
|
|
(t/deftest webhooks-permissions-crud-viewer-owner
|
|
(with-mocks [http-mock {:target 'app.http.client/req
|
|
:return {:status 200}}]
|
|
(let [owner (th/create-profile* 1 {:is-active true})
|
|
viewer (th/create-profile* 2 {:is-active true})
|
|
team (th/create-team* 1 {:profile-id (:id owner)})
|
|
whook (volatile! nil)]
|
|
(th/create-team-role* {:team-id (:id team)
|
|
:profile-id (:id viewer)
|
|
:role :viewer})
|
|
(t/testing "owner creates a wehbook"
|
|
(let [owners-webhook (create-webhook-params (:id owner) (:id team))
|
|
out (th/command! owners-webhook)
|
|
result (:result out)]
|
|
(t/is (nil? (:error out)))
|
|
(t/is (= 1 (:call-count @http-mock)))
|
|
(check-webhook-format result)
|
|
(t/is (= (:uri owners-webhook) (:uri result)))
|
|
(t/is (= (:team-id owners-webhook) (:team-id result)))
|
|
(t/is (= (:mtype owners-webhook) (:mtype result)))
|
|
(vreset! whook result)))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "viewer updates owner's webhook (unauthorized)"
|
|
(let [params {::th/type :update-webhook
|
|
::rpc/profile-id (:id viewer)
|
|
:id (:id @whook)
|
|
:uri (str (:uri @whook) "/test")
|
|
:mtype "application/transit+json"
|
|
:is-active false}
|
|
out (th/command! params)]
|
|
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
|
|
(let [error (:error out)
|
|
error-data (ex-data error)]
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :not-found))
|
|
(t/is (= (:code error-data) :object-not-found)))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "viewer deletes owner's webhook (unauthorized)"
|
|
(let [params {::th/type :delete-webhook
|
|
::rpc/profile-id (:id viewer)
|
|
:id (:id @whook)}
|
|
out (th/command! params)
|
|
error (:error out)
|
|
error-data (ex-data error)]
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :not-found))
|
|
(t/is (= (:code error-data) :object-not-found)))))))
|
|
|
|
(t/deftest webhooks-viewer-cannot-create
|
|
(with-mocks [http-mock {:target 'app.http.client/req
|
|
:return {:status 200}}]
|
|
(let [owner (th/create-profile* 1 {:is-active true})
|
|
viewer (th/create-profile* 2 {:is-active true})
|
|
team (th/create-team* 1 {:profile-id (:id owner)})]
|
|
(th/create-team-role* {:team-id (:id team)
|
|
:profile-id (:id viewer)
|
|
:role :viewer})
|
|
|
|
(t/testing "viewer cannot create a webhook on the team"
|
|
(let [params (create-webhook-params (:id viewer) (:id team))
|
|
out (th/command! params)]
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
(let [error (:error out)
|
|
error-data (ex-data error)]
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :not-found))
|
|
(t/is (= (:code error-data) :object-not-found))))))))
|
|
|
|
(t/deftest webhooks-quotes
|
|
(with-mocks [http-mock {:target 'app.http.client/req
|
|
:return {:status 200}}]
|
|
|
|
(let [prof (th/create-profile* 1 {:is-active true})
|
|
team-id (:default-team-id prof)
|
|
params {::th/type :create-webhook
|
|
::rpc/profile-id (:id prof)
|
|
:team-id team-id
|
|
:uri "http://example.com"
|
|
:mtype "application/json"}
|
|
out1 (th/command! params)
|
|
out2 (th/command! params)
|
|
out3 (th/command! params)
|
|
out4 (th/command! params)
|
|
out5 (th/command! params)
|
|
out6 (th/command! params)
|
|
out7 (th/command! params)
|
|
out8 (th/command! params)
|
|
out9 (th/command! params)]
|
|
|
|
(t/is (= 8 (:call-count @http-mock)))
|
|
(t/is (nil? (:error out1)))
|
|
(t/is (nil? (:error out2)))
|
|
(t/is (nil? (:error out3)))
|
|
(t/is (nil? (:error out4)))
|
|
(t/is (nil? (:error out5)))
|
|
(t/is (nil? (:error out6)))
|
|
(t/is (nil? (:error out7)))
|
|
(t/is (nil? (:error out8)))
|
|
|
|
(let [error (:error out9)
|
|
error-data (ex-data error)]
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :restriction))
|
|
(t/is (= (:code error-data) :webhooks-quote-reached))))))
|
|
|
|
(t/deftest removed-user-cannot-edit-webhook
|
|
(with-mocks [http-mock {:target 'app.http.client/req
|
|
:return {:status 200}}]
|
|
|
|
(let [owner (th/create-profile* 1 {:is-active true})
|
|
editor (th/create-profile* 2 {:is-active true})
|
|
team (th/create-team* 1 {:profile-id (:id owner)})]
|
|
|
|
(th/create-team-role* {:team-id (:id team)
|
|
:profile-id (:id editor)
|
|
:role :editor})
|
|
|
|
(let [params {::th/type :create-webhook
|
|
::rpc/profile-id (:id editor)
|
|
:team-id (:id team)
|
|
:uri (u/uri "http://example.com")
|
|
:mtype "application/json"}
|
|
out (th/command! params)]
|
|
|
|
(t/is (nil? (:error out)))
|
|
(let [whook (:result out)]
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "owner can edit editor's webhook (team owns it)"
|
|
(let [params {::th/type :update-webhook
|
|
::rpc/profile-id (:id owner)
|
|
:id (:id whook)
|
|
:uri (u/uri "http://example.com/updated")
|
|
:mtype "application/transit+json"
|
|
:is-active true}
|
|
out (th/command! params)]
|
|
(t/is (nil? (:error out)))
|
|
(t/is (= 1 (:call-count @http-mock)))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "remove editor from team"
|
|
(let [params {::th/type :delete-team-member
|
|
::rpc/profile-id (:id owner)
|
|
:team-id (:id team)
|
|
:member-id (:id editor)}
|
|
out (th/command! params)]
|
|
(t/is (nil? (:error out)))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "removed editor cannot update webhook"
|
|
(let [params {::th/type :update-webhook
|
|
::rpc/profile-id (:id editor)
|
|
:id (:id whook)
|
|
:uri (u/uri "http://example.com/evil")
|
|
:mtype "application/transit+json"
|
|
:is-active true}
|
|
out (th/command! params)]
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
(let [error (:error out)
|
|
error-data (ex-data error)]
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :not-found))
|
|
(t/is (= (:code error-data) :object-not-found)))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "removed editor cannot delete webhook"
|
|
(let [params {::th/type :delete-webhook
|
|
::rpc/profile-id (:id editor)
|
|
:id (:id whook)}
|
|
out (th/command! params)]
|
|
(t/is (= 0 (:call-count @http-mock)))
|
|
(let [error (:error out)
|
|
error-data (ex-data error)]
|
|
(t/is (th/ex-info? error))
|
|
(t/is (= (:type error-data) :not-found))
|
|
(t/is (= (:code error-data) :object-not-found)))))
|
|
|
|
(th/reset-mock! http-mock)
|
|
|
|
(t/testing "owner can still delete editor's webhook"
|
|
(let [params {::th/type :delete-webhook
|
|
::rpc/profile-id (:id owner)
|
|
:id (:id whook)}
|
|
out (th/command! params)]
|
|
(t/is (nil? (:error out)))
|
|
(t/is (nil? (:result out)))
|
|
(let [rows (th/db-exec! ["select * from webhook"])]
|
|
(t/is (= 0 (count rows)))))))))))
|
|
|
|
(t/deftest webhook-audit-events-carry-team-id
|
|
;; `update-webhook` and `delete-webhook` have no team id in their params, so
|
|
;; the command copies it from the stored webhook into the audit props.
|
|
(with-mocks [http-mock {:target 'app.http.client/req :return {:status 200}}]
|
|
(let [prof (th/create-profile* 1 {:is-active true})
|
|
team (th/create-team* 1 {:profile-id (:id prof)})
|
|
whook (-> (th/command! {::th/type :create-webhook
|
|
::rpc/profile-id (:id prof)
|
|
:team-id (:id team)
|
|
:uri (u/uri "http://example.com")
|
|
:mtype "application/json"})
|
|
:result)
|
|
upd (th/command! {::th/type :update-webhook
|
|
::rpc/profile-id (:id prof)
|
|
:id (:id whook)
|
|
:uri (u/uri "http://example.com/updated")
|
|
:mtype "application/json"
|
|
:is-active true})
|
|
del (th/command-raw! {::th/type :delete-webhook
|
|
::rpc/profile-id (:id prof)
|
|
:id (:id whook)})]
|
|
(t/is (= (:id team)
|
|
(get-in (meta (:result upd)) [:app.loggers.audit/props :team-id])))
|
|
(t/is (= (:id team)
|
|
(get-in (meta del) [:app.loggers.audit/props :team-id]))))))
|