* ✨ Migrate legacy file payloads to file_data and delete orphans
Copy file.data and labelled file_change.data to file_data on boot
(type main/snapshot, backend db, deleted_at preserved), then delete
file_change rows with data but no label, which no read path reaches.
Large instances pre-migrate with batched statements and fake this
step. Closes#12784.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Add 2.19.0 operations runbook for legacy file_data migration
Manual pre-migration procedure for big instances, same statements as
migration 0157 in batched transactions, following the convention of
the file-change partition runbook. Closes#12784.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Address review feedback on the legacy file data migration
Fix the issues raised in the review of the migration that moves the
legacy file payloads to file_data.
The migration deletes the unlabelled file_change rows that carry data,
but restore-files still cleared their date, and the retention test
asserted they came back. A row without a date never reaches objects-gc,
which is the leak fixed in #12419. Restore now unmarks only snapshots,
the rows that carry a label, and the retention test checks that the
legacy payload keeps its date instead of coming back.
Cover the conflict path that had no test: a pre-existing snapshot that
collides with a labelled change row, and a second run that must leave
the legacy columns null and the orphan row absent.
The runbook analyses file_data after the inserts, so the planner does
not wait for autovacuum.
Drop app.srepl.procs.fdata-storage: the copy it performed is what the
migration replaces and nothing requires it.
Closes#12784
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Keep file_change revisions on their own deletion schedule
Restoring a file from the trash cleared the deletion mark on every
file_change row of that file, revisions included. A revision is born
already scheduled for deletion one hour later, and objects-gc only
looks at rows that carry a date, so unmarking one left it in the
table for ever: production had 358,517 of them across 3,614 files.
The file deletion cascade had the mirror problem: it rewrote the date
of rows that already had one, pushing a revision out to the trash
lifetime.
Restrict the restore to durable history and date only the rows that
have no date yet, so a revision keeps the schedule it was born with.
A pinning test covers the snapshot update path, which cannot reach a
revision because it resolves its row through a read filter. The
backend memories now carry the model of the three file_data classes
and this invariant, in place of the two superseded stubs.
Closes#12418
AI-assisted-by: deepseek-v4.1-flash
* 📚 Add autovacuum and Postgres exporter notes to Grafana memory
Record what this Grafana instance can and cannot show about autovacuum
so the next session does not have to re-derive it:
- The Postgres exporter scrapes the primary as well as the
read-only replica, so the primary's pg_stat_activity and
pg_settings_* are in Prometheus even though the SQL
datasource cannot reach them.
- It publishes no pg_stat_progress_* collector, and
pg_stat_activity_count has no backend_type label: autovacuum
workers are the usename/application_name-less bucket with
state=active. The launcher shows up as the AutoVacuumMain
wait event, and it is absent on the replica.
- A metric the exporter does not publish needs a custom query
block in its YAML queries file and a restart of the exporter.
- The Postgres server ships its own log to Loki (the job is
postgresql, primary only), whose streams carry only filename,
instance, job and level. The completed vacuum lines live
there: a per-table history, not a view of what runs now.
- The PostgreSQL section now ends with the vacuum panels,
with the Vacuum TXID age table last; an in-flight per-table
view would need a pg_stat_progress_vacuum collector in the
exporter itself.
AI-assisted-by: deepseek-v4.1-flash
