📎 Standardize test scripts and add execution discipline docs

- Remove conditional build from test scripts (frontend, common)
- Remove test:jvm from common package.json (JVM tests via clojure directly)
- Remove test from backend package.json (JVM tests via clojure directly)
- Unify common/scripts/test-quiet.js with frontend's BUILD_STEPS pattern
- Add execution discipline section to mem:testing (no piping, tee to file)
- Add READ mem:testing FIRST directives to module testing docs

AI-assisted-by: deepseek-v4-flash
This commit is contained in:
Andrey Antukh 2026-07-16 14:57:05 +00:00
parent 8e8fb67793
commit 779983d38e
8 changed files with 48 additions and 26 deletions

View File

@ -101,9 +101,10 @@ misleading linter/compiler output. See `mem:tools/paren-repair`.
## Testing
IMPORTANT: all CLI commands must be executed from the `backend/` subdirectory.
IMPORTANT: all CLI commands must be executed from the `backend/` subdirectory. JVM tests are invoked directly via `clojure -M:dev:test` — there is no pnpm wrapper. If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`. See `mem:testing` for execution discipline.
* **Coverage:** If code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated.
* **Isolated run:** `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace.
* **Regression run:** `clojure -M:dev:test` to ensure no regressions in related functional areas.
* **Principles:** Cross-cutting testing principles, anti-patterns, and verification checklist: `mem:testing`.

View File

@ -4,20 +4,22 @@
## Unit tests
READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all CLJS/JS and JVM test runs.
Common tests live under `common/test/common_tests/` and use `clojure.test`.
They are CLJC and run on both JVM and JS.
From `common/`:
- Full JVM test run: `clojure -M:dev:test`
- Full JS test run: `pnpm run test:quiet`
- Full JS test run (always builds, suppressed output): `pnpm run test:quiet`
- Full JS test run (always builds, build output visible): `pnpm run test`
- Focus a JVM test namespace: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test`
- Focus a JVM test var: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test/test-basic-switch`
- Focus a JS test namespace: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test`
- Focus a JS test var: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test/test-sync-when-changing-attribute`
- Quiet logging during a JS run: append `--log-level warn` (or `trace|debug|info|warn|error`)
- Build JS test target only: `pnpm run build:test`
- After `pnpm run build:test`, direct compiled runner: `node target/tests/test.js --focus common-tests.logic.comp-sync-test/test-sync-when-changing-attribute --log-level warn`
- Watch tests: `pnpm run watch:test`
- Build JS test target only (no run): `pnpm run build:test`
- After `build:test` has been run, run the compiled runner directly: `node target/tests/test.js [--focus ...] [--log-level ...]`.
New common JS test namespaces must be required/listed in `common_tests/runner.cljc`;
new vars in existing namespaces need no runner change. Multiple JVM `--focus` flags

View File

@ -4,15 +4,18 @@ Frontend validation: CLJS + React/Rumext + RxJS/Potok; SCSS modules; shared CLJC
## Unit tests
READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all CLJS/JS test runs.
Frontend unit tests live under `frontend/test/frontend_tests/` and use `cljs.test`. They should be deterministic, avoid DOM/UI integration where possible, and mock side effects such as RPC, storage, timers, or network access.
From `frontend/`:
- Full unit test run: `pnpm run test:quiet`.
- Full unit test run (always builds, suppressed output): `pnpm run test:quiet`.
- Full unit test run (always builds, build output visible): `pnpm run test`.
- Focus a frontend CLJS test namespace: `pnpm run test:quiet -- --focus frontend-tests.logic.components-and-tokens`.
- Focus one frontend CLJS test var: `pnpm run test:quiet -- --focus frontend-tests.logic.components-and-tokens/change-spacing-token-in-main-updates-copy-layout`.
- Quiet `app.*` logging during a run: append `--log-level warn` (or `trace|debug|info|warn|error`).
- Build test target only: `pnpm run build:test`.
- After `pnpm run build:test`, direct compiled runner focus is faster: `node target/tests/test.js --focus frontend-tests.logic.components-and-tokens/change-spacing-token-in-main-updates-copy-layout`.
- Build test target only (no run): `pnpm run build:test`.
- After `build:test` has been run, run the compiled runner directly: `node target/tests/test.js [--focus ...] [--log-level ...]`.
- Watch tests: `pnpm run watch:test`.
New frontend test namespaces must be required/listed in `frontend_tests/runner.cljs`; new vars in existing namespaces need no runner change.

View File

@ -135,6 +135,20 @@ E2E tests should not be added unless explicitly requested.
| Snapshot abuse | Nobody reviews, break on any change | Focused assertions |
| Skipping tests to make suite pass | Hides real failures | Fix the test or fix the code |
## Execution discipline
When running CLJS/JS tests (frontend, common):
- **Always use `pnpm run test:quiet`** — it silently builds the test bundle then runs the test runner, giving you clean test output.
- **Never pipe test output through `tail`, `head`, or similar filters** — doing so can silently hide test failures. Use `--focus` to narrow scope instead.
- **If you need to filter output, tee to a temp file first:** `pnpm run test:quiet 2>&1 | tee /tmp/penpot-test-output.txt`. The full output is preserved on disk so you can `grep`/`tail`/`head` the file without re-running.
- Use `pnpm run test` when you want to see build output alongside test results (always builds, then runs).
- After `build:test` has been run once, you can invoke the runner directly: `node target/tests/test.js [--focus ...] [--log-level ...]`.
When running JVM tests (backend, common):
- Use `clojure -M:dev:test` directly (no pnpm wrapper).
- The same no-piping rule applies: use `--focus` to narrow scope.
## Verification Checklist
After completing any implementation:

View File

@ -21,7 +21,6 @@
"scripts": {
"lint": "clj-kondo --parallel --lint ../common/src src/",
"check-fmt": "cljfmt check --parallel=true src/ test/",
"fmt": "cljfmt fix --parallel=true src/ test/",
"test": "clojure -M:dev:test"
"fmt": "cljfmt fix --parallel=true src/ test/"
}
}

View File

@ -29,8 +29,7 @@
"lint": "pnpm run lint:clj",
"watch:test": "concurrently \"clojure -M:dev:shadow-cljs watch test\" \"nodemon -C -d 2 -w target/tests/ --exec 'node target/tests/test.js'\"",
"build:test": "clojure -M:dev:shadow-cljs compile test",
"test:js": "[ -f target/tests/test.js ] || pnpm run build:test; node target/tests/test.js",
"test:quiet": "node ./scripts/test-quiet.js",
"test:jvm": "clojure -M:dev:test"
"test": "pnpm run build:test && node target/tests/test.js",
"test:quiet": "node ./scripts/test-quiet.js"
}
}

View File

@ -1,18 +1,23 @@
import { spawnSync } from "node:child_process";
const BUILD_STEPS = [
{ label: "Building test bundle", cmd: "pnpm", args: ["run", "build:test"] },
];
const progress = (msg) => process.stderr.write(`${msg}\n`);
progress("Building test bundle...");
const build = spawnSync("pnpm", ["run", "build:test"], {
stdio: ["ignore", "pipe", "pipe"],
maxBuffer: 64 * 1024 * 1024,
});
if (build.status !== 0) {
progress("Building test bundle failed");
if (build.stdout?.length) process.stdout.write(build.stdout);
if (build.stderr?.length) process.stderr.write(build.stderr);
process.exit(build.status ?? 1);
for (const step of BUILD_STEPS) {
progress(`${step.label}...`);
const result = spawnSync(step.cmd, step.args, {
stdio: ["ignore", "pipe", "pipe"],
maxBuffer: 64 * 1024 * 1024,
});
if (result.status !== 0) {
progress(`${step.label} failed`);
if (result.stdout?.length) process.stdout.write(result.stdout);
if (result.stderr?.length) process.stderr.write(result.stderr);
process.exit(result.status ?? 1);
}
}
progress("Running tests...");
@ -21,5 +26,4 @@ const result = spawnSync(
["target/tests/test.js", ...process.argv.slice(2)],
{ stdio: "inherit" },
);
process.exit(result.status ?? 1);

View File

@ -33,7 +33,7 @@
"lint:js": "exit 0",
"lint:scss": "pnpm exec stylelint '{src,resources}/**/*.scss'",
"build:test": "pnpm run build:wasm && clojure -M:dev:shadow-cljs compile test",
"test": "[ -f target/tests/test.js ] || pnpm run build:test; node target/tests/test.js",
"test": "pnpm run build:test && node target/tests/test.js",
"test:quiet": "node ./scripts/test-quiet.js",
"test:storybook": "vitest run --project=storybook",
"watch:test": "mkdir -p target/tests && concurrently \"clojure -M:dev:shadow-cljs watch test\" \"nodemon -C -d 2 -w target/tests --exec 'node target/tests/test.js'\"",