mirror of
https://github.com/penpot/penpot.git
synced 2026-08-21 20:28:58 +00:00
* ✨ Add plugin with composable test framework and component tests The plugin provides a framework for writing composable tests against the Plugin API, and applies it to systematic end-to-end testing of component semantics. The framework's core ideas: a test is written once as a composition of operations over a starting configuration; choice points among the operations (optional steps, alternatives) expand the composition into a full sweep of test variants, so a single case definition yields broad combinatorial coverage; and the operations drive the real Plugin API with real change propagation, testing the full production implementation. The initial application is a suite of component test cases covering synchronization, overrides, swap slots and variants — the TypeScript/e2e continuation of the ClojureScript composable test suite (frontend_tests.composable_tests). Several cases originate from reproducing real defects (e.g. #10109 and the swap-slot corruptions). Tests run from an interactive panel in Penpot: cases are listed with plain-language descriptions, tests can be run selectively, results stream in live, and every checkbox carries a stable DOM id — so the panel can equally be driven programmatically (the basis for running the suite in CI), as documented in the plugin's README. Lives at plugins/apps/composable-test-suite as a regular member of the plugins workspace (init script, start:plugin:composable-test-suite, shared dev port 4202, covered by build:plugins via the new ./apps/*-test-suite filter). Related to #10584. AI-assisted-by: claude-fable-5 * ✨ Run the composable test suite headlessly in CI Adds a headless run mode for the composable test suite, following the plugin-api-test-suite's CI architecture, and a workflow that runs it as a per-PR gate. An in-sandbox entry (src/ci/headless.ts) runs the suite without the panel UI — the framework's runner was UI-free by construction, so no refactoring was needed — and streams each result through console markers, addressed by the same composite identifiers the panel uses (e.g. MainEditSyncs-2), with durations and, on failure, the error and the applied-steps transcript. It is built as a single self-executing bundle and evaluated directly inside a real Penpot plugin sandbox by the driver (ci/run-ci.ts), so no plugin dev server or port is involved. The driver needs no backend and no login: it serves the prebuilt frontend bundle via the frontend e2e static server and intercepts every backend RPC with Playwright fixtures. The mocked backend is not a limitation for this suite — everything it asserts is frontend store logic executed in memory — which the full run confirms: all 48 tests behave identically to the interactive panel, including variants and swap slots, with the single (currently expected) failure of MainEditSyncs-2 reproducing bug #10109 under the mock. TEST_FILTER selects tests by identifier substring; CI_TIMEOUT_MS bounds the run. The mock harness mirrors the frontend e2e harness (see the provenance note in the driver). Related to #10584. AI-assisted-by: claude-fable-5 * 📚 Restructure the composable-tests memory around both suites Present the composable component tests top-down: the shared framework principles upfront, then the two implementations — the ClojureScript suite in the frontend test tree and the TypeScript suite in the plugin, which tests fully end-to-end with a slightly more elaborate set of abstractions — and the plugin's headless CI run, pointing to the plugin's README for operational details. Also records this session's additions (geometry operations, case N, the CI harness). AI-assisted-by: claude-fable-5 * 📎 Refine the PR-description conventions in the creating-prs memory Encourage digestible descriptions: bullet items over prose (grouped by area with bold lead-ins for larger PRs) and no manual line wraps, since the rendered markdown adapts to the viewport. Also drop the outdated 'MCP' from the standard Note line. AI-assisted-by: claude-fable-5 * 🔧 Set Prettier endOfLine to auto in plugins workspace Prettier defaults to endOfLine "lf", which is incompatible with checkouts on Windows that use core.autocrlf=true * 🐛 Fix problems with suite --------- Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
76 lines
2.9 KiB
TypeScript
76 lines
2.9 KiB
TypeScript
import { TestCase } from "./TestCase.ts";
|
|
import { RunnableTest } from "./RunnableTest.ts";
|
|
import { TestRunObserver } from "./TestRunObserver.ts";
|
|
import { TestTree } from "./TestTree.ts";
|
|
|
|
/** A group of runnable tests enumerated from one case. */
|
|
interface Group {
|
|
identifier: string;
|
|
description: string;
|
|
tests: RunnableTest[];
|
|
}
|
|
|
|
/**
|
|
* The enumerated test suite. On construction it expands every case into its
|
|
* concrete variants ONCE, assigning each a stable opaque id, and holds them
|
|
* grouped by case and addressable by id. Both the rendered tree and any run are
|
|
* served from this single held enumeration, so the ids the UI sees and the ids it
|
|
* runs are guaranteed to refer to the same tests.
|
|
*
|
|
* Running a selected subset streams per-test notifications through a
|
|
* `TestRunObserver`; each test rebuilds its configuration fresh, so tests do not
|
|
* interfere and a failure does not stop the rest.
|
|
*/
|
|
export class TestSuite {
|
|
private readonly groups: Group[];
|
|
private readonly byId: Map<string, RunnableTest>;
|
|
|
|
constructor(cases: readonly TestCase[]) {
|
|
this.groups = TestSuite.enumerateGroups(cases);
|
|
this.byId = new Map(this.groups.flatMap((group) => group.tests.map((test) => [test.id, test])));
|
|
}
|
|
|
|
/** The serialisable tree (groups and their tests) for the UI to render. */
|
|
tree(): TestTree {
|
|
return {
|
|
groups: this.groups.map((group) => ({
|
|
identifier: group.identifier,
|
|
description: group.description,
|
|
tests: group.tests.map((test) => ({ id: test.id, name: test.name })),
|
|
})),
|
|
};
|
|
}
|
|
|
|
/** All test ids, in enumeration order (e.g. for a "run all"). */
|
|
allIds(): string[] {
|
|
return this.groups.flatMap((group) => group.tests.map((test) => test.id));
|
|
}
|
|
|
|
/**
|
|
* Runs the tests with the given `ids` (unknown ids are skipped), reporting each
|
|
* test's start and finish through `observer`, in the given order.
|
|
*/
|
|
async run(ids: readonly string[], observer: TestRunObserver): Promise<void> {
|
|
let posY = 0;
|
|
for (const id of ids) {
|
|
const test = this.byId.get(id);
|
|
if (test === undefined) continue;
|
|
observer.onTestStarted(id);
|
|
const result = await test.run(posY);
|
|
posY += 150; // advance y-position for next test
|
|
observer.onTestFinished(id, result);
|
|
}
|
|
}
|
|
|
|
/** Expands each case into a group of runnable tests with stable ids. */
|
|
private static enumerateGroups(cases: readonly TestCase[]): Group[] {
|
|
return cases.map((testCase, caseIndex) => {
|
|
const operations = testCase.operation.enumerateVariants();
|
|
const tests = operations.map((operation, i) => {
|
|
return new RunnableTest(`t${caseIndex}_${i}`, `instance #${i + 1}`, operation);
|
|
});
|
|
return { identifier: testCase.identifier, description: testCase.description, tests };
|
|
});
|
|
}
|
|
}
|