* ⚡ Improve performance and fix orphan detection in validate-file
- Add `*ref-shape-cache*` dynamic var to memoize `find-ref-shape`
lookups per page, avoiding repeated O(depth) ancestor walks.
- Add `*children-sets*` pre-computed maps for O(1) parent-child
containment checks, replacing linear `some` scans.
- Short-circuit `inside-component-main?` when the shape context
already implies a main component.
- Use single-pass reduce with early exit for duplicate detection
(children, swap slots) instead of count/distinct or frequencies.
- Guard `check-missing-slot` to skip expensive `find-near-match`
when the shape already has a swap slot.
- Refactor variant-set validation to use `run!` with direct `get`.
- Refactor `check-ref-cycles` to use a single `reduce-kv` pass.
- Fix `get-orphan-shapes`: the original `map` pipeline produced
nils so orphan shapes were never validated; rewrite with
`reduce-kv` for correct results.
- Add `validate-file-affected!` for change-scoped validation,
replacing full file validation in `process-changes-and-validate`
to only validate pages and components touched by the changes.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Improved validation
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
Replace general usage of virtual threads with platform threads
and use virtual threads for lightweight procs such that websocket
connections. This decision is made mainly because virtual threads
does not appear on thread dumps in an easy way so debugging issues
becomes very difficult.
The threads requirement of penpot for serving http requests
is not very big so having so this decision does not really affects
the resource usage.
The frontend-only features are now ignored from files and from teams
and they do not autoassigns automatically to team and file on file
creation or update operations.
this allows almost all api operations to success usin application/json
encoding with the exception of the update-file, which we need to
approach a bit differently;
the reason update-file is different, is because the operations vector
is right now defined without the context of shape type, so we are just
unable to properly parse the value to correct type using the schema
decoding mechanism
The feature fdata/pointer-map tracking mechanism interacts pretty bad
with possible local mutations on the validation subsystem. The fix
consist on disabling the tracking mechanism on the validation.
Relevant changes:
- Add the ability to create migration in both directions, defaulting
to identity if not provided
- Move the version attribute to file table column for to make it more
accessible (previously it was on data blob)
- Reduce db update operations on file-update rpc method
The climit previously of this commit is heavily used inside a
transactions, so in heavy contention operation such that file thumbnail
creation can cause a db pool exhaust.
This commit fixes this issue setting up a better resource limiting
mechanism that works outside the transactions so, contention will
no longer hold an open connection/transaction.
It also adds general improvement to the traceability to the climit
mechanism: it now properly logs the profile-id that is currently
cause some contention on specific resources.
It also add a general/root climit that is applied to all requests
so if someone start making abussive requests, we can clearly detect
it.
Mainly move all pointer-map related helpers from app.rpc.commands.files
to the the app.features.fdata namespace and normalizes codestile around
feature handling on all affected code.
This commit also comes with several features related bugifxes on the
components-v2 migration code:
- properly migrate legacy feature names on apply components-v2 migration
- start using new fdata feature related functions
- prevent generation of a ephimeral pointer on each graphic migration
operation; on large files this caused a very noticiable overhead of
creating a big number of completly unused pointer maps
- do persistence after validation and not before
