Choose policies and rewards¶
A policy determines how accumulated feedback becomes a ranking. A reward defines the learning signal. Keep the source, feedback visibility and budget fixed when comparing policies; comparable baseline matrices provide an explicit candidate-grid workflow.
Start with simple comparators¶
Use Random as an uninformed ranking comparator and UCB as a small adaptive
starting point. Add history-based heuristics through the baseline matrix before
interpreting the value of a more elaborate method. The
original COLEMAN study
provides the research context for bandit-based TCPCI.
Available native policies¶
The native runner recognizes these 37 names. Required constructor parameters
belong in the corresponding reward branch of algorithm; some windowed methods
also require window_sizes. Use the configuration reference
and constructor API for the exact fields. A configuration pack
is a preset for a subset of these names, not the complete policy inventory.
| Family | Native configuration names | What changes in the ranking |
|---|---|---|
| Random and greedy | Random, Greedy, EpsilonGreedy, DecayEpsilonGreedy, OptimisticGreedy, EpsilonDecreasing |
Start with random order, exploit accumulated value, or vary explicit exploration |
| Confidence-based | UCB, UCB1, UCB2, KLUCB, UCBTuned, UCBV, MOSSUCB |
Combine accumulated value with different exploration/confidence constructions |
| Bayesian and stochastic | ThompsonSampling, BayesianUCB, BootstrappedThompson, Softmax, Pursuit, PHE |
Sample, randomize or update selection preferences using the observed feedback |
| Adversarial | EXP3, EXP3IX |
Use exponential-weight exploration mechanisms |
| Changing-history | SlidingWindowUCB, DiscountedUCB, ChangeDetectionUCB, FRRMAB, SlMAB |
Adapt the use of older feedback through windows, discounting, detected changes or rank-based credit |
| Contextual | LinUCB, SWLinUCB, LinTS, SWLinTS, ContextualEpsilonGreedy, SWContextualEpsilonGreedy |
Condition a test's score on configured feature vectors |
| Combinatorial | CombinatorialUCB, CombinatorialThompson |
Prioritize a subset while retaining the native full-ranking interface |
| Dueling/ranking | DuelingUCB, PairwiseThompsonRanking |
Build a ranking using pairwise preference estimates |
| Portfolio | PortfolioUCB |
Select among declared candidate policies online |
The policy families expose different hypotheses to test. Keep their actual native reward/update and feedback contracts visible: sharing an algorithm name does not establish the regret guarantee or feedback model of the original publication.
Connect a policy to its research¶
| Research foundation | What it helps you understand | Start with |
|---|---|---|
| Auer, Cesa-Bianchi and Fischer, Finite-time Analysis of the Multiarmed Bandit Problem, 2002 | Classical exploration through confidence bounds under the paper's reward assumptions | UCB1, then compare the configurable UCB variants |
| Li, Chu, Langford and Schapire, A Contextual-Bandit Approach to Personalized News Article Recommendation, 2010 | Linear contextual exploration | LinUCB, with features actually available before the CI decision |
| Chen, Wang and Yuan, Combinatorial Multi-Armed Bandit: General Framework and Applications, 2013 | How subset actions, their observed components and reward structure affect a bandit problem | Combinatorial policy interfaces and their actual subset/full-ranking behavior |
| Prado Lima and Vergilio, A Multi-Armed Bandit Approach for Test Case Prioritization in Continuous Integration Environments, 2022 | Applying adaptive feedback to volatile CI test suites | Native policies, RNFail/TimeRank, scheduling budgets and repeated builds |
For complete citations and the HCS/dataset/evaluation studies, follow the research guide. Evaluate extensions under the actual CI workload, source availability, feedback and cost protocol.
Compare exploration settings¶
The README sweep example runs three UCB exploration coefficients across two seeds using the existing sweep engine. Compare configuration choices on a declared validation period, retain simple baselines, and freeze the evaluation protocol before protected outcomes. Comparable baseline matrices bind the candidate grid, observations and execution costs; protected evaluation provides the separate inference path.
Rewards and quality metrics serve different purposes¶
RNFail and TimeRank are native reward choices. RNFail uses failure feedback;
TimeRank additionally incorporates ranking information. Other reward classes
are described in the reward API. Check the runner's supported
configuration names before using a class name as a config value.
A reward transforms feedback for learning. NAPFD and APFDc describe prioritization quality under their own definitions. Neither a reward nor an internal score is a calibrated failure probability or a statistical comparison.
See native result interpretation and versioned measurements for the corresponding output contracts.
Contextual policies¶
Contextual policies consume configured feature columns, with previous-build
features declared under contextual_information.config.previous_build. The
current execution's eventual duration and verdict must not leak into the
pre-decision feature vector. A previous-row transformation alone does not prove
source availability; retained timestamps and source meaning still matter.
Li et al.'s contextual-bandit paper introduces LinUCB in a recommendation setting. Its presence here motivates a policy family; CI effectiveness needs its own source-appropriate evaluation.
Continue with point-in-time observations, protected evaluation and the research reading path.
Extend a policy¶
Use the native policy/agent interfaces and lifecycle hooks described in extensions and parallelism. Reuse orchestration, sinks and checkpoints. Document update state, randomness, feedback assumptions and the conditions under which a score is meaningful.