Research behind the workflows¶

Use this reading path to connect a capability to the question it was designed to study. A paper's findings apply to its own subjects, assumptions and experiment; they do not validate every later policy, adapter or dataset in this repository.

Adaptive prioritization¶

Prado Lima and Vergilio. A Multi-Armed Bandit Approach for Test Case Prioritization in Continuous Integration Environments. IEEE Transactions on Software Engineering 48(2), 453–465, 2022. DOI: 10.1109/TSE.2020.2992428.

The original COLEMAN article motivates learning from test feedback as suites change across CI cycles. Read it alongside the prioritization loop, policy choices and native experiments. The DOI contains 2020, reflecting its publication history; the volume/issue citation is 2022. CITATION.cff records the preferred citation.

Li, Chu, Langford and Schapire. A Contextual-Bandit Approach to Personalized News Article Recommendation. WWW, 2010. Author preprint.

This introduces LinUCB in a recommendation setting. It provides algorithmic background for a contextual family, rather than evidence about this CI runner. Continue with contextual feature availability and point-in-time observations.

Auer, Cesa-Bianchi and Fischer. Finite-time Analysis of the Multiarmed Bandit Problem. Machine Learning 47, 235–256, 2002. Publisher and DOI.

Read this for classical confidence-based exploration under its stated reward assumptions. The policy guide distinguishes that theoretical setting from a particular CI history, reward transformation or implementation.

Structured policies and subset feedback¶

Chen, Wang and Yuan. Combinatorial Multi-Armed Bandit: General Framework and Applications. ICML, PMLR 28(1), 151–159, 2013. Original proceedings and paper.

This provides a foundation for thinking about subset actions, component feedback and reward structure. Read it alongside the policy families and finite coverage selection. A native full-ranking adaptation or a finite solver is not automatically the paper's online feedback/oracle model; check the actual assumptions before applying a guarantee.

Information availability and data leakage¶

Kaufman, Rosset, Perlich and Stitelman. Leakage in data mining. ACM Transactions on Knowledge Discovery from Data 6(4), 2012. Publisher and DOI.

This explains how information unavailable at prediction time can compromise an experiment. In CI, an eventual verdict, corrected duration or late report can be present in a later export without being available to an earlier policy. Late results and corrections explains that problem through an execution timeline. Point-in-time observations provides the complete API example, while protected evaluation adds the temporal and grouped comparison protocol.

Configurations and variants¶

Prado Lima, Mendonça, Vergilio and Assunção. Learning-based prioritization of test cases in continuous integration of highly-configurable software. SPLC, 2020. DOI: 10.1145/3382025.3414967.

The same authors. Cost-effective learning-based strategies for test case prioritization in continuous integration of highly-configurable software. Empirical Software Engineering 27, article 133, 2022. Publisher and DOI.

These studies motivate prioritization across changing variants. Use WTS and VTS explained before following the LibSSH workflow. The current original CTest evidence adds source corroboration; it does not reproduce all findings or qualify every historical source assumption.

Datasets and source studies¶

A source paper describes the original study. COLEMAN's adapter guides separately state byte pins, units, exclusions and current qualification limits. Use both; do not infer missing source clocks or test inventories from a publication's aggregate statistics.

Evaluation and interpretation¶

The protected-evaluation guide declares supported comparison assumptions, pairing, multiplicity and practical-equivalence rules. The bounded paired guide documents its conservative bounds and independence requirements. Read the actual implemented protocol before choosing an analysis by name.

Repeated observations explains why builds, seeds and variants cannot be pooled as independent projects. Source admission separates source integrity from scientific eligibility. Evidence status explains the missing prerequisites for historical cross-corpus confirmation.

The independent decision benchmarks validate bounded mathematical routines against independent references. Such checks establish correctness under declared assumptions, rather than empirical policy superiority.

Cite the software and the source¶

Cite the original COLEMAN article when the method contributes to your work. Cite the HCS studies when using WTS/VTS, and each dataset's original publication and version when using its observations. Record the actual software version, source/adapter/runtime identities, experiment specification and analysis protocol in your reproducibility material.