Inspect COLEMAN's quality and security¶

You should be able to inspect the engineering behind a CI decision platform as readily as its experiment outputs. COLEMAN exposes test results, advisory audit receipts, structural limits, documentation checks and signed release artifacts. Use this page to find that evidence and understand what each check establishes.

Read the badges¶

The README links each badge to its evidence. Workflow badges show the latest matching execution: tests and documentation use pushes to main, while lint and structural checks use pull-request events. A PR badge can therefore describe a different commit from the latest published release. Check its commit and event before relying on it for a particular revision.

The coverage gate 95% badge describes the configured minimum, not a measured percentage. Open the test run's log for its actual combined line/branch coverage, passed tests and skips. The coverage configuration defines the threshold, included source and exclusions. Coverage.py evaluates the gate using its reporting precision; the current configuration uses the default whole-percent rounding. Inspect the measured result and configuration when a comparison requires finer precision.

Sonar badges report detected source vulnerabilities and a security rating in the COLEMAN dashboard. The service retains coleman4hcs in its project identifier. This is the dashboard linked by current source-analysis checks; its analysis revision and date identify the code measured. Dependency advisories have their own audit and are not counted by the source-vulnerability badge.

The former Codecov dashboard also retains the old identifier, but its public main report is historical. It is not used as a current-coverage badge. Current coverage qualification uses the pytest coverage gate and the matching Actions log. Consult GitHub's badge semantics for branch and event selection.

Tests and coverage¶

The test workflow runs the Python 3.14 suite with branch measurement and a configured 95% combined line/branch coverage gate. Its frozen environment includes the optional MCP extra for integration tests. The log retains the exact test count, coverage and skips; a new test count does not require manually updating the README.

Tests cover native prioritization and source adapters as well as failure modes:

• Versioned result round-trips, golden artifacts, legacy readers and unsupported versions.
• Units, explicit missingness, valid signed values and non-finite rejection.
• Deterministic identities and digests, duplicate handling and idempotent replay.
• Interrupted publication and checkpoint continuation.
• Application/transport import boundaries and optional integration behavior.
• Installed-package decision benchmarks and small, owned CI fixtures.

Examples of the executable contracts live in result tests, artifact tests, checkpoint tests and installed benchmark tests. Normal tests use owned fixtures without downloading external corpora.

For a development checkout with the contributor environment installed:

uv run --frozen --no-sync python -m pytest

This runs the configured coverage gate too. Follow contributor setup to install the locked test dependencies. Coverage measures exercised code paths; the evaluation guide explains the separate evidence needed for conclusions about prioritization effectiveness.

Source, dependencies and secrets¶

Security evidence comes from complementary checks:

The lock audit includes optional runtime extras, development/documentation tools and packages selected only on other operating systems. It queries advisories without installing or executing project dependencies. Known vulnerabilities, lookup failures and unsupported package sources fail the job. There is no blanket ignore list. Reproduce it with the complete commands in dependency security maintenance.

A zero source-vulnerability count and a clean advisory audit answer different questions. Read the revision, scope and collection time of each result; new advisories can require another update. Follow the security policy to report a vulnerability privately.

Code, documentation and complexity gates¶

The workflow file named pylint.yml runs Ruff and the NumPy parameter gate. The contributor workflow also uses Pyright and ty, Interrogate for docstring presence, package builds, outside-checkout examples and manual inspection of rendered documentation. The maintainability review records the broader docstring backlog; a passing bounded parameter gate does not certify every public signature.

Reproduce the automated code/documentation gates from the installed development environment:

make lint
make format-check
make docstrings
make cost-structural
make docs
uv run --frozen --no-sync python scripts/generate_llms_txt.py
git diff --exit-code -- llms.txt

The contributor validation commands cover the additional local checks. PR title validation and labels support review routing; the relevant technical checks qualify the proposed change before integration.

Installed correctness and performance observations¶

The decision observations workflow builds a wheel with locked tooling, installs that wheel and runs outside the checkout. Choose the correctness or performance profile on manual dispatch. The Tuesday schedule executes only when the repository variable ENABLE_DECISION_PERFORMANCE is true; it is not an unconditional PR gate.

The workflow retains reports, including available failed-attempt artifacts. Inspect source revision, environment, resource bounds and the selected profile before comparing measurements. The benchmark guide explains independent oracles, certificates and timing limits. Neither coverage instrumentation timings nor one fast hosted run establish a performance advantage.

Inspect a published distribution¶

The release workflow uses locked release/build tools and publishes these assets in GitHub releases:

Read the dependency versions and hashes from uv.lock at the corresponding release tag. For example, the installation tutorial's wheel uses the lock committed at v4.19.1. Choose the tag matching your wheel or source distribution when inspecting a different release. The lock's repository revision remains directly accessible independently of the release asset list.

The distribution inventory contract explains the SBOM's scope: it identifies the wheel's package metadata and digest; unbundled dependencies are audited separately through the lock. Signing binds distribution bytes to a signing identity. Inspect the expected repository, workflow identity and release before trusting an artifact.

Start with the published-wheel installation example, which pins the release and SHA-256. Then inspect the corresponding bundles and lock when your adoption process requires them. Engineering checks, artifact identity and scientific evidence are all inspectable parts of the same decision.