Andrey Antukh 464756ec59 ✨ Rework the import dialog
The import wizard now opens with a Check/Upload/Import stepper derived
from the dialog status, plus a phase title and description per stage,
and its file rows follow the new design: folder icon, file name and a
size-plus-extension subtitle.

The step reflects the real state: analyze is Check, a milestone in the
upload stage is Upload, and queue, start and job milestones are Import.
While checking, the dialog shows no rows and no spinner, and the exact
files inside a binfile appear once the analysis resolves.

Each row paints a progress bar: the upload stage drives it with the
real chunk counter, and every import stage keeps the animated bar,
because the client cannot know a global percentage of the work. The
queued label joins the same status line as the milestone text and now
reads "Waiting job slot".

The change also drops the context-notification wrapper and the legacy
loader around file names, and documents the two CSS pitfalls found on
the way: a logical inline-size wins over an inline physical width, and
Rumext does not convert dynamic :style maps for plain elements, so
inline styles leave CLJS as #js objects.

AI-assisted-by: glm-5.3-flash

🔧 Enable NativeMemoryTracking in the backend devenv

Add -XX:NativeMemoryTracking=summary to JAVA_OPTS so the backend and
REPL JVMs expose native memory breakdowns through jcmd.

Useful to diagnose non-heap growth (metaspace, code cache, GC, thread)
while working with imports and other JVM-heavy paths.

AI-assisted-by: deepseek-v4.1-flash

✨ Rework the import wizard progress and the size of its files

The milestone in course is written once, in the header, where the
message of the phase was: every file of a container reports the same
one, so a row repeated it. A row keeps its name and its size, and the
button to drop a file only while the wizard still offers its import
step, which is where dropping one makes sense.

The title counts the files and counts them done with what the job
reports about the file it is working on, so the number moves while a
container rebuilds its files one by one. The bar measures the whole
import: the real bytes of every container while they are uploaded, and
animated while the wizard checks, waits for a slot or rebuilds.

A file of a version 3 package shows what it occupies inside its
container: its own entries plus the storage objects its media and its
thumbnails reference, which the container keeps shared under
`objects/`. What the container itself occupies stays in
`:container-size`, the size the upload of the package is measured on.

The ready step waited for every known file instead of for the end of
the analysis, so Continue left the files that arrived later out of the
import, with a row that never showed anything. The upload step was
recognized only once a row carried a milestone, so the wizard said it
was importing during the first chunk; it now holds until every file
leaves the upload behind.

Also fixes a misspelled design system token that left the success
color unset.

AI-assisted-by: deepseek-v4.1-flash

✨ Cover the import sizes and the done counter with tests

The size a file of a version 3 package shows is its own entries plus
the storage objects its media and its thumbnails reference, and what
the container occupies stays apart: the test drives the analysis with a
fake container that has both, and checks that the manifest is not
charged to any file.

The done counter of the header counts the files the job left behind:
all of them for a container that finished, the ones before the file it
is working on for a container in course, none for one that waits for a
slot, and the files of every container add up.

The test that answers one entry per file also needed the mock of the
entries of a container, which the analysis now reads.

AI-assisted-by: deepseek-v4.1-flash

✨ Say the wait for a slot in the header, not in every row

A row of the import wizard no longer paints a waiting state: while no
job of the import has started, the header says it waits for a free
slot, and the rows keep their name and their size. Only a file whose
container already finished uploading says something of its own
("Ready"), plus the error text when it fails.

The clock icon and the `dashboard.import.status.waiting` key of both
locales go away with it.

The notes of the redesign come along: the description of the wizard in
`mem:frontend/dashboard-viewer-subtleties`, and a new
`mem:clojure/promesa` with the call styles of the library and how they
bridge to rx.

AI-assisted-by: deepseek-v4.1-flash
2026-10-21 12:01:33 +02:00

130 lines
4.3 KiB
Bash

#!/usr/bin/env bash
export PENPOT_ADMIN_CONSOLE_SHARED_KEY=super-secret-nitrate-api-key
export PENPOT_EXPORTER_SHARED_KEY=super-secret-exporter-api-key
export PENPOT_NEXUS_SHARED_KEY=super-secret-nexus-api-key
export PENPOT_SECRET_KEY=super-secret-devenv-key
export PENPOT_MEDIA_PROCESSOR_SHARED_KEY=super-secret-media-processor-key
# DEPRECATED: only used for subscriptions
export PENPOT_MANAGEMENT_API_KEY=super-secret-management-api-key
# Runtime config that varies per devenv instance (PENPOT_HOST, PENPOT_PUBLIC_URI,
# PENPOT_DATABASE_*, PENPOT_REDIS_URI, PENPOT_OBJECTS_STORAGE_*, AWS_*) is owned by
# docker/devenv/defaults.env and injected via the main service's env block.
if [ -f /home/selfsigned.crt ]; then
export NODE_EXTRA_CA_CERTS=/home/selfsigned.crt;
fi
# Background worker flag is per-instance. Defaults to enabled (ws0); ws1+
# overlays set PENPOT_BACKEND_WORKER=false so scheduled and async tasks only
# run on ws0, keeping notification Pub/Sub bound to a single Valkey. See
# mem:devenv/core for the rationale.
__worker_flag=""
if [[ "${PENPOT_BACKEND_WORKER:-true}" == "true" ]]; then
__worker_flag="enable-backend-worker"
fi
export PENPOT_MEDIA_PROCESSING_SERVICE_URI=http://localhost:6065
export PENPOT_FLAGS="\
$PENPOT_FLAGS \
enable-login-with-password \
enable-login-with-ldap \
disable-login-with-oidc \
disable-login-with-google \
disable-login-with-github \
disable-login-with-gitlab \
disable-telemetry \
$__worker_flag \
enable-backend-asserts \
disable-feature-fdata-pointer-map \
enable-feature-fdata-objects-map \
enable-audit-log \
enable-transit-readable-response \
disable-remote-media-processing \
enable-demo-users \
enable-user-feedback \
disable-secure-session-cookies \
enable-smtp \
enable-account-lockout \
enable-prepl-server \
enable-urepl-server \
enable-nrepl-server \
enable-rpc-climit \
enable-rpc-rlimit \
enable-quotes \
enable-soft-rpc-rlimit \
enable-auto-file-snapshot \
enable-webhooks \
enable-access-tokens \
disable-tiered-file-data-storage \
enable-file-validation \
enable-file-schema-validation \
enable-redis-cache \
enable-link-preview \
enable-subscriptions";
# Uncomment for nexus integration testing
# export PENPOT_FLAGS="$PENPOT_FLAGS enable-nexus";
# export PENPOT_NEXUS_URI="http://localhost:6070/";
# Default deletion delay for devenv
export PENPOT_DELETION_DELAY="24h"
# Setup default upload media file size to 100MiB
export PENPOT_MEDIA_MAX_FILE_SIZE=104857600
# Setup default multipart upload size to 300MiB
export PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE=314572800
export PENPOT_USER_FEEDBACK_DESTINATION="support@example.com"
export PENPOT_ADMIN_CONSOLE_URI=http://localhost:3000/admin-console
export JAVA_OPTS="\
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager \
-Djdk.attach.allowAttachSelf \
-Dlog4j2.configurationFile=log4j2-devenv.xml \
-Djdk.tracePinnedThreads=full \
-XX:+UnlockExperimentalVMOptions \
-XX:+UseShenandoahGC \
-XX:+UseCompactObjectHeaders \
-XX:ShenandoahGCMode=generational \
-XX:NativeMemoryTracking=summary \
-XX:-OmitStackTraceInFastThrow \
--sun-misc-unsafe-memory-access=allow \
--enable-preview \
--enable-native-access=ALL-UNNAMED \
--add-opens=java.base/java.nio=ALL-UNNAMED";
function setup_s3_bucket() {
if [ "${PENPOT_OBJECTS_STORAGE_BACKEND}" != "s3" ]; then
return 0
fi
local attempts=0
local endpoint="${PENPOT_OBJECTS_STORAGE_S3_ENDPOINT}"
local bucket="${PENPOT_OBJECTS_STORAGE_S3_BUCKET}"
local region="${PENPOT_OBJECTS_STORAGE_S3_REGION}"
until aws s3api head-bucket \
--endpoint-url "$endpoint" \
--region "$region" \
--bucket "$bucket" >/dev/null 2>&1; do
if aws s3 mb "s3://${bucket}" \
--endpoint-url "$endpoint" \
--region "$region" >/dev/null 2>&1; then
return 0
fi
attempts=$((attempts + 1))
if [ "$attempts" -ge 30 ]; then
echo "setup_s3_bucket: unable to create s3://${bucket} after ${attempts} attempts" >&2
return 1
fi
sleep 1
done
}