mirror of
https://github.com/penpot/penpot.git
synced 2026-09-24 12:56:15 +00:00
452f38cf5d
* ✨ Add storage operation metrics for S3 and buckets Expose Prometheus metrics for the object storage subsystem. The S3 backend now attaches an AWS SDK MetricPublisher that counts API calls, retries and latency per operation and target. The storage layer counts logical operations and deduplication outcomes per Penpot bucket, and the assets handlers count served requests per route. Closes #11676 AI-assisted-by: muse-spark-1.3-contributor * ✨ Fix storage metrics labels, errors and test gaps Address the review findings on the storage metrics commit. Label reads with the object's own backend, count failed asset serving as errors without swallowing them, and cover the failed S3 call, S3 asset path and permission-denied branches with tests. Also share the label helper and reuse the metrics test helper. Closes #11676 AI-assisted-by: muse-spark-1.3-contributor * ✨ Harden storage metrics and fill test gaps Address the second-round review findings on storage metrics. Unknown backends now fail explicitly and count as errors, exists stays paired with its dedup outcome, and the thumbnail, missing storage, expired reads, unknown buckets and write failure paths are covered by tests. Label coercion goes through the shared metrics helper. Closes #11676 AI-assisted-by: muse-spark-1.3-contributor * ✨ Harden storage metrics accuracy and coverage Address the full-branch review findings on storage metrics. Touch and delete emit only on changed rows, reads emit after the backend fetch, unknown backends fail explicitly, and tempfile mismatches count as unauthorized. Publisher nil policy, pairing rules and attempt semantics are documented and covered by tests. Closes #11676 AI-assisted-by: muse-spark-1.3-contributor * ✨ Address full-branch review findings on storage metrics Touch and delete resolve labels from the row, reads stay paired, failures are covered by tests, and logging, ranges and docs are tightened. Includes the label helper unit tests and the retries wording clarification. Closes #11676 AI-assisted-by: muse-spark-1.3-contributor * ⚡ Label touch and del metrics from UPDATE RETURNING The storage metrics change resolved metric labels for touch-object! and del-object! with an extra SELECT per id-based call. Since app.main instruments storage unconditionally, every GC collector and binfile import paid that extra round trip: deleting a team with 10k media objects doubled the storage_object statements exactly on the paths that already process the most rows. touch-object! and del-object! now take only the object id (UUID) and read the labels from the updated row itself via RETURNING id, backend, metadata: one statement, no pre-read, and labels that always match the row actually mutated. del-object! additionally guards on deleted_at IS NULL, so a repeated delete returns false and emits no metric. Also from the review of the full branch: extract the duplicated serve/emit/rethrow block in app.http.assets into one helper; give penpot_storage_s3_timing explicit histogram buckets up to 60s (the default cap at 7.5s hid the slow S3 calls the metric exists for); drop the unused ::target-id config key from the S3 backend and hardcode the :default target label until per-bucket routing lands. AI-assisted-by: glm-5.3-flash * ✨ Harden storage metric recording and definitions The metric definition schema is now closed and declares every key the collectors read: buckets, quantiles, max-age and reg. A typo such as a misspelled ::mdef/buckets used to compile and silently fall back to the default histogram buckets; it now fails at startup. The asset result-label fallback coerced an absent status to 500, so a future serve path without a status would have counted successes as errors. The mapping is now explicit and documented: served below 400, unauthorized for 401/403, not-found for 404, and error for everything else, including an absent status. The never-fail try/catch around metric recording existed four times with drift. One app.metrics/run-safe! helper replaces them: it no-ops on a nil metrics instance and logs the first failure per hint at warn level, then at debug, so a broken setup surfaces once without flooding the log. The S3 publisher keeps its outer try/catch: it is the SDK MetricPublisher contract boundary. AI-assisted-by: glm-5.3-flash * ✨ Make metrics mandatory and run! safe by default Recording a metric must never change the behavior of the operation being measured, so `run!` now catches recording failures itself: the first failure per metric id logs at warn, later ones at debug. This replaces the `run-safe!` helper, whose four copies had drifted, and applies the guarantee to every emit site instead of only storage. The metrics instance precondition is a plain assert, and the collector lookup stays outside the recording guard, so a missing instance fails hard even when asserts are disabled. Metrics is therefore no longer optional: the storage, s3-backend and db-pool schemas require `::mtx/metrics`, and the assets handler cfg always carries it. `wrap-publisher` no longer returns nil for a nil instance, and the db pool wires the prometheus tracker unconditionally. AI-assisted-by: deepseek-v4.1-flash
121 lines
5.1 KiB
Clojure
121 lines
5.1 KiB
Clojure
;; This Source Code Form is subject to the terms of the Mozilla Public
|
|
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
;;
|
|
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
|
|
|
(ns backend-tests.metrics-test
|
|
(:require
|
|
[app.metrics :as mtx]
|
|
[app.metrics.definition :as-alias mdef]
|
|
[clojure.test :as t]
|
|
[integrant.core :as ig])
|
|
(:import
|
|
io.prometheus.client.Collector$MetricFamilySamples
|
|
io.prometheus.client.Collector$MetricFamilySamples$Sample
|
|
io.prometheus.client.CollectorRegistry))
|
|
|
|
(def ^:private valid-definitions? @#'app.metrics/valid-definitions?)
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; Helpers
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(defn- sample-names
|
|
[metrics]
|
|
(->> (mtx/get-registry metrics)
|
|
(.metricFamilySamples)
|
|
(enumeration-seq)
|
|
(mapcat (fn [^Collector$MetricFamilySamples family]
|
|
(map (fn [^Collector$MetricFamilySamples$Sample sample]
|
|
(.-name sample))
|
|
(.samples family))))
|
|
(set)))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; Tests
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(t/deftest process-metrics-are-exported
|
|
;; the process cpu and file descriptor families come from the
|
|
;; prometheus client `StandardExports`, registered by `app.metrics`.
|
|
;; They are read reflectively from the OS MXBean and depend on the
|
|
;; `jdk.management` module at runtime: a pruned jlink JRE turns the
|
|
;; MXBean into `sun.management.BaseOperatingSystemImpl`, the reflective
|
|
;; getters fail and the families are silently dropped (that is how the
|
|
;; production backend lost `process_open_fds`). This test pins the
|
|
;; contract the fd alert relies on.
|
|
(let [metrics (ig/init-key :app.metrics/metrics {:default {}})
|
|
names (sample-names metrics)]
|
|
|
|
(t/is (contains? names "process_open_fds"))
|
|
(t/is (contains? names "process_max_fds"))
|
|
(t/is (contains? names "process_cpu_seconds_total"))))
|
|
|
|
(t/deftest label-coercion
|
|
(t/are [value fallback expected]
|
|
(= expected (mtx/label value fallback))
|
|
"fs" "unknown" "fs"
|
|
:s3 "unknown" "s3"
|
|
200 "unknown" "200"
|
|
nil "unknown" "unknown"
|
|
nil "default" "default"))
|
|
|
|
(t/deftest run-contains-label-arity-bug
|
|
;; A wrong label count throws inside the prometheus client; the
|
|
;; recording failure must not propagate to the caller.
|
|
(let [registry (CollectorRegistry.)
|
|
collector (mtx/create-collector {::mdef/name "penpot_test_run_arity"
|
|
::mdef/help "test helper"
|
|
::mdef/type :counter
|
|
::mdef/labels ["a" "b"]
|
|
:app.metrics/registry registry})
|
|
instance (reify mtx/IMetrics
|
|
(get-collector [_ _] collector))]
|
|
(t/is (nil? (mtx/run! instance :id :x :inc 1 :labels ["only-one"])))))
|
|
|
|
(t/deftest run-fails-loudly-without-metrics-instance
|
|
;; A missing or invalid instance is a wiring bug: it must fail on every
|
|
;; invocation instead of silently dropping the measurement. With
|
|
;; asserts enabled this is an AssertionError; with asserts disabled the
|
|
;; collector lookup (outside the recording guard) throws instead.
|
|
(t/is (thrown? Throwable (mtx/run! nil :id :x :inc 1)))
|
|
(t/is (thrown? Throwable (mtx/run! :not-metrics :id :x :inc 1))))
|
|
|
|
(t/deftest run-records-on-success
|
|
(let [registry (CollectorRegistry.)
|
|
collector (mtx/create-collector {::mdef/name "penpot_test_run"
|
|
::mdef/help "test helper"
|
|
::mdef/type :counter
|
|
::mdef/labels ["result"]
|
|
:app.metrics/registry registry})
|
|
instance (reify mtx/IMetrics
|
|
(get-collector [_ _] collector))]
|
|
(t/is (true? (mtx/run! instance :id :x :inc 1 :labels ["ok"])))))
|
|
|
|
(t/deftest definitions-schema-accepts-all-consumed-keys
|
|
(t/is (true? (valid-definitions?
|
|
{:test-timing
|
|
{::mdef/name "penpot_test_timing"
|
|
::mdef/help "test"
|
|
::mdef/type :histogram
|
|
::mdef/labels ["op"]
|
|
::mdef/buckets [5 10 25]}
|
|
:test-summary
|
|
{::mdef/name "penpot_test_summary"
|
|
::mdef/help "test"
|
|
::mdef/type :summary
|
|
::mdef/quantiles [[0.5 0.01]]
|
|
::mdef/max-age 60}}))))
|
|
|
|
|
|
(t/deftest definitions-schema-rejects-unknown-keys
|
|
;; A typo such as ::mdef/bucksets must fail at startup instead of
|
|
;; silently falling back to the default histogram buckets. The
|
|
;; definition map is closed, so unknown keys do not validate.
|
|
(t/is (false? (valid-definitions?
|
|
{:bad {::mdef/name "penpot_bad_metric"
|
|
::mdef/help "typo check"
|
|
::mdef/type :histogram
|
|
::mdef/bucksets [100]}}))))
|