Andrey Antukh
7b5817f407
♻️ Make several adjustments to the dashboard deleted page ( #7999 )
...
* ♻️ Make several sustantial adjustments to the dashboard deleted page
* 📎 Add PR feedback changes
2025-12-30 09:52:29 +01:00
Andrey Antukh
94719eebf8
♻️ Make storage and other objects deletion task vclock aware
...
This simplifes the mental model on how it works and simplifies testing
of the related code.
This also normalizes storage object deletion in the same way as the
rest of objects in penpot (now future deletion date on storage object
also means storage object to be deleted).
2025-11-19 17:28:55 +01:00
Andrey Antukh
7d5c1c9b5f
✨ Make file-gc-scheduler task compatible with virtual clock
...
And simplify implementation
2025-11-05 10:47:31 +01:00
Andrey Antukh
5717708b56
♻️ Refactor file storage
...
Make it more scallable and make it easily extensible
2025-10-13 12:24:05 +02:00
Andrey Antukh
17ec360720
⚡ Add several missing type hints for avoid reflection and autoboxing
2025-08-18 13:03:10 +02:00
Andrey Antukh
283eb0419c
♻️ Refactor time related namespaces
...
Mainly removes the custom app.util.time namespace
from frontend and backend and normalize all to use
the app.common.time namespace
2025-08-01 11:20:01 +02:00
Andrey Antukh
a706907b26
♻️ Normalize logical deletion to future dates
...
Instead of managing the ...
2025-05-20 22:25:57 +02:00
Andrey Antukh
694a2084e2
✨ Add file cleaner to file-gc process
2025-05-08 13:35:25 +02:00
Andrey Antukh
1187d64f69
Merge remote-tracking branch 'origin/staging' into develop
2025-02-24 12:49:04 +01:00
Andrey Antukh
3074fc9ab5
♻️ Remove deprecated with-atomic and refactor tx-run! ( #5915 )
...
* ♻️ Remove deprecated with-atomic and refactor tx-run!
* ✨ Do not hold open connection for the whole clone-template operation
2025-02-24 11:15:44 +01:00
Andrey Antukh
d2c800fc0f
🐛 Add missing handling of file_change on delete-object task
2025-02-21 14:24:18 +01:00
Andrey Antukh
624750ad16
🐛 Fix incorrect order of execution of internal procs on objects-gc ( #5929 )
...
* ⚡ Improve efficiency on objects-gc tasks
Replacing db/cursor with db/plan
* 🐛 Use correct order of subprocs on objects-gc
Mainly affects the file deletion
* 📎 Increase chunk-size to 100 on objects-gc
2025-02-21 13:01:20 +01:00
Andrey Antukh
15f7fac8f4
✨ Add more reasonable default fetch size for db/plan
...
Making it more general purpose and not only for streaming
2025-01-27 12:53:11 +01:00
Andrey Antukh
16934b734b
🎉 Add srepl fix function for process media refs on a file
2025-01-27 12:38:12 +01:00
Andrey Antukh
541ed6282f
✨ Normalize file retrieving and persistence on binfile, file-gc and helpers
2025-01-27 12:38:10 +01:00
Andrey Antukh
471699960f
🐛 Update media references after instantiation of a component ( #5652 )
...
🐛 Update media references after instantiation of a component
2025-01-27 11:58:13 +01:00
Andrey Antukh
da0704081f
📎 Normalize default opts for db/plan function
2025-01-20 23:23:31 +01:00
Andrey Antukh
151aedcf91
♻️ Make the components cleaning mechanism on file-gc task more efficient
2025-01-20 16:35:14 +01:00
Andrey Antukh
5513daf17d
♻️ Make the media cleaning mechanism on file-gc task more efficient
...
Replaces the use of db/cursor with db/plan, that teorethically allows
processing large results without consuming all result set in memory
2025-01-20 16:34:05 +01:00
Andrey Antukh
1bb25bb89d
🔥 Remove file-xlog-gc task
...
It is no longer necessary because snapshots are managed by the objects-gc task
2024-12-10 12:19:12 +01:00
Andrey Antukh
88fb5e7ab5
♻️ Update integrant to latest version
...
This upgrade also includes complete elimination of use spec
from the backend codebase, completing the long running migration
to fully use malli for validation and decoding.
2024-11-13 19:09:19 +01:00
Andrey Antukh
32126d1874
♻️ Refactor file changes gc tasks
...
Make it more friendly with the current snapshoting mechanism
2024-10-30 13:39:38 +01:00
Andrey Antukh
a8814dcaba
🐛 Add missing fields on file-gc libraries fetching sql
2024-09-12 21:07:19 +02:00
Andrey Antukh
50df2279a7
🐛 Make the media cleaning on file-gc task aware of snapshots
...
It now takes in account the snapshots, and prevents
deletion of media files used in snapshots.
2024-09-03 14:50:17 +02:00
Andrey Antukh
ceaafdbb1c
✨ Add offload mechanism for file snapshots
2024-08-26 13:52:42 +02:00
Andrey Antukh
8dea5d5158
♻️ Make file-xlog-gc task more scalable
2024-08-26 11:15:59 +02:00
Andrey Antukh
77c45ed109
✨ Add better error reporting on offload-file-data task
2024-08-12 11:55:19 +02:00
Andrey Antukh
ab461ba560
🐛 Launch offload only if file-gc has processed the file
2024-08-12 11:55:19 +02:00
Andrey Antukh
ba167f256b
⚡ Add performance enhancements on telemetry related queries
2024-08-09 14:28:18 +02:00
Andrey Antukh
0e92bcc0de
🎉 Add file-data offload mechanism
2024-08-09 14:28:18 +02:00
Andrey Antukh
253b9e5bd8
✨ Split file-gc task in two separated tasks
...
Add a new file-gc-scheduler task for analizing all files for
elegibility and leave file-gc task with the responsability to
performn the GC operation.
2024-08-09 14:13:33 +02:00
Andrey Antukh
5cf54c6384
✨ Improve file snapshoting mechanism
2024-07-29 10:19:34 +02:00
Andrey Antukh
9174bb140b
♻️ Refactor configuration validation
...
Replace spec with schema
2024-07-10 15:16:28 +02:00
Andrey Antukh
f9af7f0f09
🐛 Make profile deletion follow the delete-object flow
...
This removes the need of the specific task for cleaning
orphan teams.
2024-06-27 14:00:52 +02:00
Andrey Antukh
67489c0bb9
🐛 Fix profile deletion issue with 1 participant
2024-06-27 13:56:39 +02:00
Andrey Antukh
ec4260830c
♻️ Add API consistency fixes for task calling
...
Also adds a helper for calling tasks synchronously
2024-06-25 13:24:17 +02:00
Andrey Antukh
aa1cf3e03a
✨ Add some redundancy to delete_object task
2024-06-25 13:06:44 +02:00
Andrey Antukh
574c8d1789
✨ Move library-absorb operation to async task
...
And make it not mandatory in case of failure
2024-05-27 11:19:15 +02:00
Andrey Antukh
39119ac040
✨ Reuse team deletion logic on orphan teams gc task
2024-05-27 11:17:00 +02:00
Andrey Antukh
761bbb7334
✨ Add srepl helpers for delete/restore teams, projects, and files
2024-05-27 11:17:00 +02:00
Andrey Antukh
f3346786ea
🔥 Remove unused object-update task
2024-05-23 16:36:43 +02:00
Andrey Antukh
584a0fdba1
✨ Reduce locking on objects-gc task
...
The main issue was the long running gc operation that
affects storage objects with deduplication. The long running
transacion ends locking some storage object rows which collaterally
made operations like import-binfile become blocked indefinitelly
because of the same rows (because of deduplication).
The solution used in this commit is split operations on small
chunks so we no longer use long running transactions that holds
too many locks. With this approach we will make a window to work
concurrently all operarate the distinct operations that requires
locks on the same rows.
2024-05-23 16:35:54 +02:00
Andrey Antukh
e27c0b2086
✨ Add a task for asynchronous object update operation
2024-04-16 16:42:44 +02:00
Andrey Antukh
9c25723ee3
📎 Add note about fragments on object-gc ns
2024-04-10 15:31:49 +02:00
Andrey Antukh
6b552fd8a9
🐛 Don't run file-gc on deleted files
2024-04-10 15:31:49 +02:00
Andrey Antukh
eaf546ba5e
✨ Add improvements to telemetry task
2024-03-25 15:07:55 +01:00
Andrey Antukh
a6562619a3
🐛 Fix incorrect fragment cleaning on file-gc task
2024-03-22 12:59:36 +01:00
Andrey Antukh
91118bec70
✨ Improve internal naming of setup/props
...
This reverts commit a6f70c77cbe72875875ab83587d5dcd9cfc61e34.
2024-03-14 10:48:23 +01:00
alonso.torres
a6f70c77cb
Revert " ✨ Improve internal naming of setup/props"
...
This reverts commit f525c6df5e9785df9b534b6de1b5308084627699.
2024-03-13 16:21:12 +01:00
Andrey Antukh
eabec6be20
🔥 Remove not needed events
2024-03-13 11:40:16 +01:00
