[P0] Scaffold the project, test harness, and CI #4

New issue

Closed

opened 2026-07-17 23:58:16 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-07-17 23:58:16 +00:00

(Migrated from github.com)

Goal

Create a minimal, reproducible project skeleton that enforces test-driven development and honest failure reporting before feature implementation.

Requirements

Select and document the runtime and dependency-management approach from #3.
Establish source, test, documentation, and configuration layouts.
Add formatting, linting, static typing, unit-test, integration-test, security-test, and security-scan commands.
Add docs/testing.md documenting red → green → refactor, regression-test-first bug fixes, test-layer boundaries, fixture rules, and failure semantics.
Add a pull-request template requiring the failing test added first, the observed red result, the final green result, and the acceptance criteria covered.
Make the canonical test command fail non-zero on assertion failures, collection/import errors, fixture/setup/teardown errors, process crashes, signals, timeouts, subprocess failures, and zero collected tests when tests are expected.
Preserve pipeline exit status when output is piped through tools such as tee; shell entrypoints use strict error handling and pipefail where applicable.
Prohibit false-green patterns in test and CI paths, including || true, || :, unconditional exit 0, set +e around tests, swallowed exceptions, continue-on-error, allow_failure, and pass-with-no-tests flags.
Required tests may not be skipped, disabled, marked expected-failure, or filtered out without an explicit linked issue and a CI-visible failure or quarantine policy approved by the project owner.
Add CI for supported Linux environments without uploading capture artifacts.
Add CI checks that deliberately run a known-failing fixture and confirm the wrapper and workflow return non-zero.
Add pre-commit checks that detect committed secrets, plaintext fixture leaks, disabled-test markers, and prohibited false-green constructs in test/CI scripts.
Provide a deterministic development environment, including Nix support.

Acceptance criteria

A clean checkout can run all checks with one documented command.
A deliberately failing assertion makes the local command and CI job fail.
A test process crash, timeout, collection failure, and zero-test selection each make the command fail.
Piped test output preserves the test runner’s non-zero status.
No CI job uses failure-tolerant settings for required checks.
CI passes on the default branch only when all required tests execute and pass.
Tests use synthetic fixtures only.
Logs and CI artifacts contain no host screenshots, usernames, paths, or secrets.

Depends on

## Goal Create a minimal, reproducible project skeleton that enforces test-driven development and honest failure reporting before feature implementation. ## Requirements - Select and document the runtime and dependency-management approach from #3. - Establish source, test, documentation, and configuration layouts. - Add formatting, linting, static typing, unit-test, integration-test, security-test, and security-scan commands. - Add `docs/testing.md` documenting red → green → refactor, regression-test-first bug fixes, test-layer boundaries, fixture rules, and failure semantics. - Add a pull-request template requiring the failing test added first, the observed red result, the final green result, and the acceptance criteria covered. - Make the canonical test command fail non-zero on assertion failures, collection/import errors, fixture/setup/teardown errors, process crashes, signals, timeouts, subprocess failures, and zero collected tests when tests are expected. - Preserve pipeline exit status when output is piped through tools such as `tee`; shell entrypoints use strict error handling and `pipefail` where applicable. - Prohibit false-green patterns in test and CI paths, including `|| true`, `|| :`, unconditional `exit 0`, `set +e` around tests, swallowed exceptions, `continue-on-error`, `allow_failure`, and pass-with-no-tests flags. - Required tests may not be skipped, disabled, marked expected-failure, or filtered out without an explicit linked issue and a CI-visible failure or quarantine policy approved by the project owner. - Add CI for supported Linux environments without uploading capture artifacts. - Add CI checks that deliberately run a known-failing fixture and confirm the wrapper and workflow return non-zero. - Add pre-commit checks that detect committed secrets, plaintext fixture leaks, disabled-test markers, and prohibited false-green constructs in test/CI scripts. - Provide a deterministic development environment, including Nix support. ## Acceptance criteria - A clean checkout can run all checks with one documented command. - A deliberately failing assertion makes the local command and CI job fail. - A test process crash, timeout, collection failure, and zero-test selection each make the command fail. - Piped test output preserves the test runner’s non-zero status. - No CI job uses failure-tolerant settings for required checks. - CI passes on the default branch only when all required tests execute and pass. - Tests use synthetic fixtures only. - Logs and CI artifacts contain no host screenshots, usernames, paths, or secrets. ## Depends on - #3