COLEMAN 4.1 — Know what your CI budget actually buys¶
A faster first failure is useful. Missing the failures that matter is not a saving. COLEMAN 4.1 adds a reproducible operational pilot that makes that distinction inspectable, using the native TCPCI runner and independent execution accounting.
From a ranking to accountable CI evidence¶
Run a small, MIT-licensed synthetic CI workflow with base order, shortest first, native Random and native Greedy. Freeze its source identity, evaluation period, execution budget and quality thresholds before execution. Inspect every build's failing-test recall, first failure, occupied runner time, elapsed execution time and separately measured prioritization overhead.
The fixture's three-second budget costs 0.03 modeled credits at its explicit price. A candidate that spends less by missing failures fails the detection gate. Faster feedback with unchanged recall stays a feedback improvement, without being sold as improved detection. Synthetic replay remains insufficient evidence for production adoption. Negative and inconclusive findings are first-class outputs.
Small contracts, reusable evidence¶
- Versioned, immutable pilot manifests and typed execution events with explicit units, source identity, licenses, complete test universes and frozen thresholds.
- Independent serial/parallel event accounting that keeps runner-seconds separate from elapsed time, and excluded failures inside the recall denominator.
- Explicit unknown/timeout outcomes, censored no-detection times and genuine zero values; no silent reinterpretation of existing native metrics.
- Validated report snapshots, deterministic serialization, conflicting-retry detection and restart from completed candidate/build pairs using existing artifact infrastructure.
- An installed-package native workflow, a recorded bounded evidence report and a practical adoption checklist covering real overhead, protected comparisons, mandatory suites and conservative fallback.
Start with the operational pilot guide and runnable example. Install the COLEMAN wheel attached to this GitHub release; the similarly named PyPI package is a different project. Verify published asset digests and Sigstore attestations using the installation guide.
Compatibility and limits¶
Python 3.14+ and existing native scheduling/results/checkpoints remain unchanged. No new dependency is required. Pilot schema v1 rejects unknown versions; old native artifacts keep their original readers and meanings, with no implicit migration.
This release validates a bounded synthetic replay, not production savings, algorithm superiority, calibrated probabilities or commercial demand. Per-build ranges are descriptive rather than confidence intervals. Real source admission, diverse cross-project evidence and dependence-aware confirmatory comparisons remain work in the public roadmap.