mirror of
https://github.com/penpot/penpot.git
synced 2026-11-12 13:00:37 +00:00
34c702bc80
* ✨ Partition file_change by file_id Partition `file_change` by `HASH (file_id)` in 16 partitions, with the same bounds as `file_data`, so a join between them is resolved partition by partition. The primary key becomes `(file_id, id)`, which a partitioned table needs, and every access filters by `file_id`. The new table carries a `kind` column (1 snapshot, 2 revision) as a reserve: the queries still tell them apart by the label, and `kind` takes over when they are rewritten. The payload of a snapshot stays in `file_data`; the `data` column is kept for reading old `legacy-db` payloads. The four snapshot commands require `file-id`, because it prunes the partition. The objects GC locks and deletes by the pair, in batches. The old table is kept as `file_change_legacy` as the way back. The automatic migration refuses to run above 10 GiB or 2M rows and points to the runbook; big instances partition online with that runbook, which copies by id, follows the `updated_at` watermark in delta passes and checks that no row of the source is missing from the target. Closes #12416 AI-assisted-by: deepseek-v4.1-flash * 🐛 Address the review of the file_change partition The production runbook promised a rollback it did not have: the section was a sketch, and its statements could write to the table about to be dropped. The rollback is removed, and so is every comment that described it. `file_change_legacy` stays as a frozen copy of the previous layout. The cutover check now aborts the transaction when a row of the old table is missing in the new one, instead of leaving the decision to the operator. The five secondary indexes of the new table are built after the last delta pass: the bulk and the delta passes only need the primary key, so the batches no longer maintain them. The copy of 0158 is now replayed in a test against the pre-0158 shape with a labelled and an unlabelled row, plus an empty instance that must leave no legacy table behind. The updated_at assertion of the snapshot mutations is bounded by the test clock, and the repair revision stops writing the `features` of the file, like the normal save. AI-assisted-by: deepseek-v4.1-flash * 📎 Fix linter issues * 🐛 Fix stale helper call in rpc_file_snapshot_test The suite does not load: `th/run-pending-tasks!` does not exist in backend-tests.helpers. The pending jobs runner is `run-pending-jobs` (no arguments), the name every other call site of the suite uses. AI-assisted-by: inkling-2-small-free * 📚 Align the partition change with the 2.20.0 milestone 2.19.0 is already shipped; this work targets 2.20.0. The operations file of the partition procedure carries the 2.20.0 prefix, the references that point at it (the migration, the runbook table and the memory) follow, and the RPC doc/changes tags of the snapshot commands record 2.20 as the release that makes `file-id` required. AI-assisted-by: inkling-2-small-free * 🐛 Fix review findings on the partition rebase The 0157 legacy test was missed by the rebase and inserted file_change rows without the new mandatory `kind` column, so the backend suite fails against the migrated schema. The inserts now carry `:kind` (snapshot-kind for the labelled rows, revision-kind for the rest). The post-cutover verification block of the runbook terminates the first query with a semicolon so the block pastes into psql without a syntax error. The guard message of the migration names the step (0158-partition-file-change) instead of "0158", ambiguous since two migrations share that number. Closes #12416 AI-assisted-by: inkling-2-small-free * 🐛 Fix the stale label in the snapshot restore test The rename block of snapshots-crud runs before the restore block, so the restored version is already named "label1-updated" when the restore publishes its message. The command names the version by the label the row has at that moment, not by the one it had when it was created, so the test was asserting a stale value. Assert the current label, and say why in the comment. AI-assisted-by: deepseek-v4.1-flash * ♻️ Split the version event tests into one test per command The workspace version events and the plugin proxy send the file-id of the open file on every snapshot command, because the command needs it to prune the partition of file_change. Both tests checked that with a doseq and t/testing blocks, so a failure named the whole group and not the command that broke. Give each command its own test. The workspace ones share a helper that runs the event and returns the command and params it sends; the plugin ones keep their own mock of rp/cmd!. AI-assisted-by: deepseek-v4.1-flash
693 lines
25 KiB
Clojure
693 lines
25 KiB
Clojure
;; This Source Code Form is subject to the terms of the Mozilla Public
|
|
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
;;
|
|
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
|
|
|
(ns app.rpc.quotes
|
|
"Penpot resource usage quotes."
|
|
(:require
|
|
[app.common.data :as d]
|
|
[app.common.exceptions :as ex]
|
|
[app.common.logging :as l]
|
|
[app.common.schema :as sm]
|
|
[app.common.time :as ct]
|
|
[app.config :as cf]
|
|
[app.db :as db]
|
|
[app.jobs :as jobs]
|
|
[cuerdas.core :as str]))
|
|
|
|
(defmulti check-quote ::id)
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; PUBLIC API
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:quote
|
|
[:map {:title "Quote"}
|
|
[::team-id {:optional true} ::sm/uuid]
|
|
[::project-id {:optional true} ::sm/uuid]
|
|
[::file-id {:optional true} ::sm/uuid]
|
|
[::incr {:optional true} [::sm/int {:min 0}]]
|
|
[::id :keyword]
|
|
[::profile-id ::sm/uuid]])
|
|
|
|
(def valid-quote?
|
|
(sm/lazy-validator schema:quote))
|
|
|
|
(def ^:private enabled (volatile! true))
|
|
|
|
(defn enable!
|
|
"Enable quotes checking at runtime (from server REPL)."
|
|
[]
|
|
(vswap! enabled (constantly true)))
|
|
|
|
(defn disable!
|
|
"Disable quotes checking at runtime (from server REPL)."
|
|
[]
|
|
(vswap! enabled (constantly false)))
|
|
|
|
(defn- check
|
|
[cfg quote]
|
|
(let [quote (merge cfg quote)
|
|
id (::id quote)]
|
|
|
|
(when-not (valid-quote? quote)
|
|
(ex/raise :type :internal
|
|
:code :invalid-quote-definition
|
|
:hint "found invalid data for quote schema"
|
|
:quote (name id)))
|
|
|
|
(-> (assoc quote ::target (name id))
|
|
(check-quote))))
|
|
|
|
(defn check!
|
|
([cfg]
|
|
(when (contains? cf/flags :quotes)
|
|
(when @enabled
|
|
(db/run! cfg check {}))))
|
|
|
|
([cfg & others]
|
|
(when (contains? cf/flags :quotes)
|
|
(when @enabled
|
|
(db/run! cfg (fn [cfg]
|
|
(run! (partial check cfg) others)))))))
|
|
|
|
(defn- send-notification!
|
|
[cfg]
|
|
(l/warn :hint "max quote reached"
|
|
:target (::target cfg)
|
|
:profile-id (some-> cfg ::profile-id str)
|
|
:team-id (some-> cfg ::team-id str)
|
|
:project-id (some-> cfg ::project-id str)
|
|
:file-id (some-> cfg ::file-id str)
|
|
:quote (::quote cfg)
|
|
:total (::total cfg)
|
|
:incr (::inc cfg 1))
|
|
|
|
(when-let [admins (seq (cf/get :admins))]
|
|
(let [subject (str/istr "[quotes:notification]: max quote reached ~(::target cfg)")
|
|
content (str/istr "- Param: profile-id '~(::profile-id cfg)}'\n"
|
|
"- Param: team-id '~(::team-id cfg)'\n"
|
|
"- Param: project-id '~(::project-id cfg)'\n"
|
|
"- Param: file-id '~(::file-id cfg)'\n"
|
|
"- Quote ID: '~(::target cfg)'\n"
|
|
"- Max: ~(::quote cfg)\n"
|
|
"- Total: ~(::total cfg) (INCR ~(::incr cfg 1))\n")]
|
|
(jobs/submit cfg
|
|
{::jobs/name :sendmail
|
|
::jobs/delay (ct/duration "30s")
|
|
::jobs/max-retries 4
|
|
::jobs/priority 200
|
|
::jobs/dedupe true
|
|
::jobs/label "quotes-notification"
|
|
::jobs/params {:to (vec admins)
|
|
:subject subject
|
|
:body content}}))))
|
|
|
|
(defn- generic-check!
|
|
[{:keys [::db/conn ::incr ::quote-sql ::count-sql ::default ::target] :or {incr 1} :as cfg}]
|
|
(let [quote (->> (db/exec! conn quote-sql)
|
|
(map :quote)
|
|
(reduce max (- Integer/MAX_VALUE)))
|
|
quote (if (pos? quote) quote default)
|
|
total (:total (db/exec-one! conn count-sql))]
|
|
|
|
(when (> (+ total incr) quote)
|
|
(if (contains? cf/flags :soft-quotes)
|
|
(send-notification! (assoc cfg ::quote quote ::total total))
|
|
(ex/raise :type :restriction
|
|
:code :max-quote-reached
|
|
:target target
|
|
:quote quote
|
|
:count total)))))
|
|
|
|
(def ^:private sql:get-quotes-1
|
|
"SELECT id, quote
|
|
FROM usage_quote
|
|
WHERE target = ?
|
|
AND profile_id = ?
|
|
AND team_id IS NULL
|
|
AND project_id IS NULL
|
|
AND file_id IS NULL;")
|
|
|
|
(def ^:private sql:get-quotes-2
|
|
"SELECT id, quote
|
|
FROM usage_quote
|
|
WHERE target = ?
|
|
AND ((team_id = ? AND (profile_id = ? OR profile_id IS NULL)) OR
|
|
(profile_id = ? AND team_id IS NULL AND project_id IS NULL AND file_id IS NULL));")
|
|
|
|
(def ^:private sql:get-quotes-3
|
|
"SELECT id, quote
|
|
FROM usage_quote
|
|
WHERE target = ?
|
|
AND ((project_id = ? AND (profile_id = ? OR profile_id IS NULL)) OR
|
|
(team_id = ? AND (profile_id = ? OR profile_id IS NULL)) OR
|
|
(profile_id = ? AND team_id IS NULL AND project_id IS NULL AND file_id IS NULL));")
|
|
|
|
(def ^:private sql:get-quotes-4
|
|
"SELECT id, quote
|
|
FROM usage_quote
|
|
WHERE target = ?
|
|
AND ((file_id = ? AND (profile_id = ? OR profile_id IS NULL)) OR
|
|
(project_id = ? AND (profile_id = ? OR profile_id IS NULL)) OR
|
|
(team_id = ? AND (profile_id = ? OR profile_id IS NULL)) OR
|
|
(profile_id = ? AND team_id IS NULL AND project_id IS NULL AND file_id IS NULL));")
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: TEAMS-PER-PROFILE
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:teams-per-profile
|
|
[:map [::profile-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-teams-per-profile-quote?
|
|
(sm/lazy-validator schema:teams-per-profile))
|
|
|
|
(def ^:private sql:get-teams-per-profile
|
|
"SELECT count(*) AS total
|
|
FROM team_profile_rel
|
|
WHERE profile_id = ?")
|
|
|
|
(defmethod check-quote ::teams-per-profile
|
|
[{:keys [::profile-id ::target] :as cfg}]
|
|
(assert (valid-teams-per-profile-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-teams-per-profile Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-1 target profile-id])
|
|
(assoc ::count-sql [sql:get-teams-per-profile profile-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: ACCESS-TOKENS-PER-PROFILE
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:access-tokens-per-profile
|
|
[:map [::profile-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-access-tokens-per-profile-quote?
|
|
(sm/lazy-validator schema:access-tokens-per-profile))
|
|
|
|
(def ^:private sql:get-access-tokens-per-profile
|
|
"SELECT count(*) AS total
|
|
FROM access_token
|
|
WHERE profile_id = ?")
|
|
|
|
(defmethod check-quote ::access-tokens-per-profile
|
|
[{:keys [::profile-id ::target] :as cfg}]
|
|
(assert (valid-access-tokens-per-profile-quote? cfg) "invalid quote parameters")
|
|
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-access-tokens-per-profile Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-1 target profile-id])
|
|
(assoc ::count-sql [sql:get-access-tokens-per-profile profile-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: PROJECTS-PER-TEAM
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:projects-per-team
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-projects-per-team-quote?
|
|
(sm/lazy-validator schema:projects-per-team))
|
|
|
|
(def ^:private sql:get-projects-per-team
|
|
"SELECT count(*) AS total
|
|
FROM project AS p
|
|
WHERE p.team_id = ?
|
|
AND p.deleted_at IS NULL")
|
|
|
|
(defmethod check-quote ::projects-per-team
|
|
[{:keys [::profile-id ::team-id ::target] :as cfg}]
|
|
(assert (valid-projects-per-team-quote? cfg) "invalid quote parameters")
|
|
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-projects-per-team Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-2 target team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-projects-per-team team-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: FONT-VARIANTS-PER-TEAM
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:font-variants-per-team
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-font-variant-per-team-quote?
|
|
(sm/lazy-validator schema:font-variants-per-team))
|
|
|
|
(def ^:private sql:get-font-variants-per-team
|
|
"SELECT count(*) AS total
|
|
FROM team_font_variant AS v
|
|
WHERE v.team_id = ?")
|
|
|
|
(defmethod check-quote ::font-variants-per-team
|
|
[{:keys [::profile-id ::team-id ::target] :as cfg}]
|
|
(assert (valid-font-variant-per-team-quote? cfg) "invalid quote parameters")
|
|
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-font-variants-per-team Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-2 target team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-font-variants-per-team team-id])
|
|
(generic-check!)))
|
|
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: INVITATIONS-PER-TEAM
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:invitations-per-team
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-invitations-per-team-quote?
|
|
(sm/lazy-validator schema:invitations-per-team))
|
|
|
|
(def ^:private sql:get-invitations-per-team
|
|
"SELECT count(*) AS total
|
|
FROM team_invitation
|
|
WHERE team_id = ?")
|
|
|
|
(defmethod check-quote ::invitations-per-team
|
|
[{:keys [::profile-id ::team-id ::target] :as cfg}]
|
|
(assert (valid-invitations-per-team-quote? cfg) "invalid quote parameters")
|
|
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-invitations-per-team Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-2 target team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-invitations-per-team team-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: PROFILES-PER-TEAM
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:profiles-per-team
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-profiles-per-team-quote?
|
|
(sm/lazy-validator schema:profiles-per-team))
|
|
|
|
(def ^:private sql:get-profiles-per-team
|
|
"SELECT (SELECT count(*)
|
|
FROM team_profile_rel
|
|
WHERE team_id = ?) +
|
|
(SELECT count(*)
|
|
FROM team_invitation
|
|
WHERE team_id = ?
|
|
AND valid_until > now()) AS total;")
|
|
|
|
;; NOTE: the total number of profiles is determined by the number of
|
|
;; effective members plus ongoing valid invitations.
|
|
|
|
(defmethod check-quote ::profiles-per-team
|
|
[{:keys [::profile-id ::team-id ::target] :as cfg}]
|
|
(assert (valid-profiles-per-team-quote? cfg) "invalid quote parameters")
|
|
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-profiles-per-team Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-2 target team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-profiles-per-team team-id team-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: FILES-PER-PROJECT
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:files-per-project
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::project-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-files-per-project-quote?
|
|
(sm/lazy-validator schema:files-per-project))
|
|
|
|
(def ^:private sql:get-files-per-project
|
|
"SELECT count(*) AS total
|
|
FROM file AS f
|
|
WHERE f.project_id = ?
|
|
AND f.deleted_at IS NULL")
|
|
|
|
(defmethod check-quote ::files-per-project
|
|
[{:keys [::profile-id ::project-id ::team-id ::target] :as cfg}]
|
|
(assert (valid-files-per-project-quote? cfg) "invalid quote parameters")
|
|
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-files-per-project Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-3 target project-id profile-id team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-files-per-project project-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: COMMENT-THREADS-PER-FILE
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:comment-threads-per-file
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::project-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-comment-threads-per-file-quote?
|
|
(sm/lazy-validator schema:comment-threads-per-file))
|
|
|
|
(def ^:private sql:get-comment-threads-per-file
|
|
"SELECT count(*) AS total
|
|
FROM comment_thread AS ct
|
|
WHERE ct.file_id = ?")
|
|
|
|
(defmethod check-quote ::comment-threads-per-file
|
|
[{:keys [::profile-id ::file-id ::team-id ::project-id ::target] :as cfg}]
|
|
(assert (valid-comment-threads-per-file-quote? cfg) "invalid quote parameters")
|
|
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-comment-threads-per-file Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-4 target file-id profile-id project-id
|
|
profile-id team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-comment-threads-per-file file-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: COMMENTS-PER-FILE
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:comments-per-file
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::project-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-comments-per-file-quote?
|
|
(sm/lazy-validator schema:comments-per-file))
|
|
|
|
(def ^:private sql:get-comments-per-file
|
|
"SELECT count(*) AS total
|
|
FROM comment AS c
|
|
JOIN comment_thread AS ct ON (ct.id = c.thread_id)
|
|
WHERE ct.file_id = ?")
|
|
|
|
(defmethod check-quote ::comments-per-file
|
|
[{:keys [::profile-id ::file-id ::team-id ::project-id ::target] :as cfg}]
|
|
(assert (valid-comments-per-file-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-comments-per-file Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-4 target file-id profile-id project-id
|
|
profile-id team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-comments-per-file file-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: SNAPSHOTS-PER-FILE
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:snapshots-per-file
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::project-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]
|
|
[::file-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-snapshots-per-file-quote?
|
|
(sm/lazy-validator schema:snapshots-per-file))
|
|
|
|
(def ^:private sql:get-snapshots-per-file
|
|
;; A version is told apart by its label, and the partial index with the same
|
|
;; predicate answers the count without touching the table.
|
|
"SELECT count(*) AS total
|
|
FROM file_change AS fc
|
|
WHERE fc.file_id = ?
|
|
AND fc.created_by = 'user'
|
|
AND fc.deleted_at IS NULL
|
|
AND fc.label IS NOT NULL")
|
|
|
|
(defmethod check-quote ::snapshots-per-file
|
|
[{:keys [::profile-id ::file-id ::team-id ::project-id ::target] :as cfg}]
|
|
(assert (valid-snapshots-per-file-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-snapshots-per-file Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-4 target file-id profile-id project-id
|
|
profile-id team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-snapshots-per-file file-id])
|
|
(generic-check!)))
|
|
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: SNAPSHOTS-PER-TEAM
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:snapshots-per-team
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-snapshots-per-team-quote?
|
|
(sm/lazy-validator schema:snapshots-per-team))
|
|
|
|
(def ^:private sql:get-snapshots-per-team
|
|
;; The files of the team are resolved first, so the count filters by
|
|
;; `file_id`: the planner can prune the partitions of `file_change` when it
|
|
;; chooses an index scan (with a sequential scan it reads all of them). A
|
|
;; version is told apart by its label.
|
|
"SELECT count(*) AS total
|
|
FROM file_change AS fc
|
|
WHERE fc.file_id = ANY (SELECT f.id
|
|
FROM file AS f
|
|
JOIN project AS p ON (p.id = f.project_id)
|
|
WHERE p.team_id = ?)
|
|
AND fc.created_by = 'user'
|
|
AND fc.deleted_at IS NULL
|
|
AND fc.label IS NOT NULL")
|
|
|
|
(defmethod check-quote ::snapshots-per-team
|
|
[{:keys [::profile-id ::team-id ::target] :as cfg}]
|
|
(assert (valid-snapshots-per-team-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-snapshots-per-team Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-2 target team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-snapshots-per-team team-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: TEAM-ACCESS-REQUESTS-PER-TEAM
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:team-access-requests-per-team
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-team-access-requests-per-team-quote?
|
|
(sm/lazy-validator schema:team-access-requests-per-team))
|
|
|
|
(def ^:private sql:get-team-access-requests-per-team
|
|
"SELECT count(*) AS total
|
|
FROM team_access_request AS tar
|
|
WHERE tar.team_id = ?")
|
|
|
|
(defmethod check-quote ::team-access-requests-per-team
|
|
[{:keys [::profile-id ::team-id ::target] :as cfg}]
|
|
(assert (valid-team-access-requests-per-team-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-team-access-requests-per-team Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-2 target team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-team-access-requests-per-team team-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: TEAM-ACCESS-REQUESTS-PER-REQUESTER
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:team-access-requests-per-requester
|
|
[:map
|
|
[::profile-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-team-access-requests-per-requester-quote?
|
|
(sm/lazy-validator schema:team-access-requests-per-requester))
|
|
|
|
(def ^:private sql:get-team-access-requests-per-requester
|
|
"SELECT count(*) AS total
|
|
FROM team_access_request AS tar
|
|
WHERE tar.requester_id = ?")
|
|
|
|
(defmethod check-quote ::team-access-requests-per-requester
|
|
[{:keys [::profile-id ::target] :as cfg}]
|
|
(assert (valid-team-access-requests-per-requester-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-team-access-requests-per-requester Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-1 target profile-id])
|
|
(assoc ::count-sql [sql:get-team-access-requests-per-requester profile-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: UPLOAD-SESSIONS-PER-PROFILE
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:upload-sessions-per-profile
|
|
[:map [::profile-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-upload-sessions-per-profile-quote?
|
|
(sm/lazy-validator schema:upload-sessions-per-profile))
|
|
|
|
(def ^:private sql:get-upload-sessions-per-profile
|
|
"SELECT count(*) AS total
|
|
FROM upload_session
|
|
WHERE profile_id = ?
|
|
AND deleted_at IS NULL")
|
|
|
|
(defmethod check-quote ::upload-sessions-per-profile
|
|
[{:keys [::profile-id ::target] :as cfg}]
|
|
(assert (valid-upload-sessions-per-profile-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-upload-sessions-per-profile Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-1 target profile-id])
|
|
(assoc ::count-sql [sql:get-upload-sessions-per-profile profile-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: MEDIA-STORAGE-BYTES-PER-TEAM
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:media-storage-bytes-per-team
|
|
[:map
|
|
[::profile-id ::sm/uuid]
|
|
[::team-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-media-storage-bytes-per-team-quote?
|
|
(sm/lazy-validator schema:media-storage-bytes-per-team))
|
|
|
|
(def ^:private sql:get-media-storage-bytes-per-team
|
|
"SELECT COALESCE(SUM(so.size), 0) AS total
|
|
FROM (
|
|
SELECT fmo.media_id AS so_id
|
|
FROM file_media_object AS fmo
|
|
JOIN file AS f ON (f.id = fmo.file_id)
|
|
JOIN project AS p ON (p.id = f.project_id)
|
|
WHERE p.team_id = ?
|
|
AND fmo.deleted_at IS NULL
|
|
AND f.deleted_at IS NULL
|
|
UNION
|
|
SELECT fmo.thumbnail_id AS so_id
|
|
FROM file_media_object AS fmo
|
|
JOIN file AS f ON (f.id = fmo.file_id)
|
|
JOIN project AS p ON (p.id = f.project_id)
|
|
WHERE p.team_id = ?
|
|
AND fmo.thumbnail_id IS NOT NULL
|
|
AND fmo.deleted_at IS NULL
|
|
AND f.deleted_at IS NULL
|
|
UNION
|
|
SELECT v.otf_file_id AS so_id
|
|
FROM team_font_variant AS v
|
|
WHERE v.team_id = ?
|
|
AND v.otf_file_id IS NOT NULL
|
|
AND v.deleted_at IS NULL
|
|
UNION
|
|
SELECT v.ttf_file_id AS so_id
|
|
FROM team_font_variant AS v
|
|
WHERE v.team_id = ?
|
|
AND v.ttf_file_id IS NOT NULL
|
|
AND v.deleted_at IS NULL
|
|
UNION
|
|
SELECT v.woff1_file_id AS so_id
|
|
FROM team_font_variant AS v
|
|
WHERE v.team_id = ?
|
|
AND v.woff1_file_id IS NOT NULL
|
|
AND v.deleted_at IS NULL
|
|
UNION
|
|
SELECT v.woff2_file_id AS so_id
|
|
FROM team_font_variant AS v
|
|
WHERE v.team_id = ?
|
|
AND v.woff2_file_id IS NOT NULL
|
|
AND v.deleted_at IS NULL
|
|
) AS refs
|
|
JOIN storage_object AS so ON (so.id = refs.so_id)
|
|
WHERE so.deleted_at IS NULL")
|
|
|
|
(defmethod check-quote ::media-storage-bytes-per-team
|
|
[{:keys [::profile-id ::team-id ::target] :as cfg}]
|
|
(assert (valid-media-storage-bytes-per-team-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get :quotes-media-storage-bytes-per-team
|
|
(* 20 1024 1024 1024)))
|
|
(assoc ::quote-sql [sql:get-quotes-2 target team-id profile-id profile-id])
|
|
(assoc ::count-sql [sql:get-media-storage-bytes-per-team
|
|
team-id team-id team-id team-id team-id team-id])
|
|
(generic-check!)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: BINFILES JOBS PER PROFILE
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def ^:private schema:jobs-per-profile
|
|
[:map [::profile-id ::sm/uuid]])
|
|
|
|
(def ^:private valid-jobs-per-profile-quote?
|
|
(sm/lazy-validator schema:jobs-per-profile))
|
|
|
|
;; Only the jobs that are still on their way count: a terminal job is not
|
|
;; pending work anymore, and the jobs GC removes it. The tenant filters
|
|
;; like every other query of the substrate: instances share one database
|
|
;; and one profile can have work in more than one of them.
|
|
(def ^:private
|
|
sql:get-jobs-per-profile
|
|
"SELECT count(*) AS total
|
|
FROM job
|
|
WHERE tenant = ?
|
|
AND profile_id = ?
|
|
AND status IN ('new', 'scheduled', 'running', 'retry')
|
|
AND name = ANY(?)")
|
|
|
|
(defn- names-of-family
|
|
"The registered names of a family of jobs. The quote counts what the
|
|
registry knows, so a new job of the family is covered without touching
|
|
this file."
|
|
[cfg family]
|
|
(->> (jobs/get-defs cfg)
|
|
(vals)
|
|
(filter #(= family (::jobs/family %)))
|
|
(map #(d/name (::jobs/name %)))
|
|
(vec)))
|
|
|
|
(defn- check-jobs-quote
|
|
[cfg family config-key]
|
|
(let [{:keys [::db/conn ::profile-id ::target]} cfg]
|
|
(assert (valid-jobs-per-profile-quote? cfg) "invalid quote parameters")
|
|
(-> cfg
|
|
(assoc ::default (cf/get config-key Integer/MAX_VALUE))
|
|
(assoc ::quote-sql [sql:get-quotes-1 target profile-id])
|
|
(assoc ::count-sql [sql:get-jobs-per-profile
|
|
(cf/get :tenant)
|
|
profile-id
|
|
(db/create-array conn "text" (names-of-family cfg family))])
|
|
(generic-check!))))
|
|
|
|
(defmethod check-quote ::export-jobs-per-profile
|
|
[cfg]
|
|
(check-jobs-quote cfg :export :quotes-export-jobs-per-profile))
|
|
|
|
(defmethod check-quote ::import-jobs-per-profile
|
|
[cfg]
|
|
(check-jobs-quote cfg :import :quotes-import-jobs-per-profile))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; QUOTE: DEFAULT
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(defmethod check-quote :default
|
|
[{:keys [::id]}]
|
|
(ex/raise :type :internal
|
|
:code :quote-not-defined
|
|
:quote id
|
|
:hint "backend using a quote identifier not defined"))
|