Skip to content

COLEMAN 4.17 — Compare CI baselines with evidence you can inspect

Run comparable test-prioritization experiments through COLEMAN's native runner, keep the evidence behind every decision, and resume interrupted work without rewriting its history. A frozen matrix makes candidate exclusions and incomplete results visible alongside successful executions.

What you can use

  • Freeze the source snapshot, development fold, candidates, seeds, native-work budgets and cooperative resource limits in a versioned BaselinePlan.
  • Compare twelve built-in candidates: seeded Random, original order, prior-cost and failure-history heuristics, UCB, Thompson sampling, discounted UCB, LinUCB, the native Random/UCB portfolio and a labeled unattainable reference.
  • Reuse the existing RunSpec runner, native policies, RNFail feedback, Parquet sinks and per-build checkpoints through the base-package matrix API.
  • Inspect complete rankings, selected prefixes, independently checked quality and work measurements, resource observations and every physical attempt.
  • Verify completed artifacts by their actual bytes. Retain failed, aborted and incompatible cells with explicit reasons; an interrupted invocation cannot publish a falsely complete matrix.

The complete installed CI example executes all twelve internal candidates and records 36 build observations. Its three builds belong to one constructed family and correctly return insufficient evidence. The DANTE declaration remains incompatible until its external runtime, outcome unit and common-universe inputs are qualified.

Reproducible mathematics and compatibility

The matrix computes source work with a stable binary64 sum before native budget selection. Parallel reductions cannot change that declared budget. Native binary32 test costs and sequential binary64 prefixes remain explicit, with independent rational checks of full-universe NAPFD, midpoint-cost APFDc, recall, selected work and censored first-failure time.

Zero-failure quality ratios are unavailable, while legacy native Parquet retains its existing fallback. Missing values remain distinct from observed zero. Existing artifacts keep their meaning and readers; unknown contract versions reject without implicit migration. Recorded runtimes get separate scopes. Successful process pools now finish normal worker cleanup before closing; interruption still terminates and joins workers.

Evidence and limits

The complete reviewed source suite passed 3,816 tests with one platform skip and 95.14% combined coverage. Ubuntu CI passed 3,814 tests with three skips and 95.10% coverage. Qualification includes versioned goldens, round trips, exact retries, metric-corruption guards, real process interruption and checkpoint recovery, future-availability rejection and fresh native matrices with different Polars thread counts.

The actual signed wheel was installed outside checkout: all 322 SDK files match the tag, the domain import guard excludes CLI/MCP, and the tagged native example preserves all 258 artifacts on retry. All six asset digests, both distribution signatures, SBOM and version-only automatic bump were checked.

Original Karaf data also executes ten eligible methods across eight builds, retaining 80 independently checked raw observations. Its mathematical digest matches both pre-release runs with different thread counts. Actual published 4.16 results resume at checkpoint 8 with all 28 protected files unchanged.

The published wheel also revalidates all 61 original source bindings and 63 data cards with the original default Polars thread configuration. The admission digest remains unchanged, with six original raw-producer proofs and four uncovered historical/HCS regimes. Changing legacy stage-ETL thread settings can reject fixed-profile artifact bytes in both 4.16 and 4.17; preserve the original producer configuration rather than rewriting an existing receipt.

This is development evidence preparation. Source admission, independent project families and protected confirmation remain separate gates. Costs are declared additive test work, not verified whole-runner occupancy or prospective CI savings. The release makes no policy-superiority or universal-generalization claim. Public provider feasibility and the final corpus audit continue under #261.