Skip to content

Bandit

coleman4hcs.bandit

coleman4hcs.bandit - Multi-Armed Bandit Models for Test Case Prioritization.

This module provides implementations of different multi-armed bandit (MAB) models to be used in the context of software testing. Specifically, it provides a general Bandit class, a dynamic version that allows for the addition and removal of arms, and an extension that incorporates evaluation metrics to provide feedback on the selected arms.

Classes:

Name Description
Bandit

Represents the classic multi-armed bandit model.
DynamicBandit

An extension of the basic Bandit that supports dynamic management of its arms.
EvaluationMetricBandit

A dynamic bandit that provides feedback based on a given evaluation metric.
Notes

The module facilitates the use of MAB models for test case prioritization in software testing scenarios. By integrating feedback from evaluation metrics, it allows for adaptive prioritization strategies that can react to changes in the testing environment or system under test.

Bandit

Bases: ABC

Represents a multi-armed bandit model.

Parameters:

Name Type Description Default
arms list of dict

The arms of the bandit (test case records). Required columns are defined by tc_fieldnames.

 required

Attributes:

Name Type Description
tc_fieldnames list of str

Column names expected in the arms data.
arms DataFrame or None

DataFrame containing the current arms of the bandit.

__init__ 

__init__(arms)

Initialize a Bandit with its arms.

Parameters:

Name Type Description Default
arms list of dict

The arms of the bandit (test case records). Required columns are defined by tc_fieldnames.

 required

add_arms 

add_arms(arms)

Add one or multiple arms to the bandit.

Parameters:

Name Type Description Default
arms list of dict

List of arms to add.

 required

get_arms 

get_arms()

Retrieve the list of arm names currently associated with the bandit.

Returns:

Type Description
list of str

List of arm names (test cases).

pull abstractmethod 

pull(action)

Simulate pulling an arm.

To be implemented by subclasses.

Parameters:

Name Type Description Default
action list

The action (prioritized test suite) to pull.

 required

reset 

reset()

Reset the arms to an empty DataFrame.

update_priority 

update_priority(action)

Update the Priority column with the priorities.

Parameters:

Name Type Description Default
action list of str

List of test cases in order of prioritization.

 required

DynamicBandit

Bases: Bandit, ABC

A Bandit that allows dynamic management of its arms.

Extends the base Bandit to support updating the set of arms at runtime.

update_arms 

update_arms(arms)

Update the arms of the bandit.

Parameters:

Name Type Description Default
arms list of dict

The new set of arms to replace the current ones.

 required

EvaluationMetricBandit

Bases: DynamicBandit

A Dynamic Bandit that provides feedback based on an evaluation metric.

Parameters:

Name Type Description Default
arms list of dict

The arms of the bandit (test case records).

 required
evaluation_metric EvaluationMetric

The evaluation metric used to provide feedback.

 required

Attributes:

Name Type Description
evaluation_metric EvaluationMetric

The evaluation metric instance.

__init__ 

__init__(arms, evaluation_metric)

Initialize the EvaluationMetricBandit.

Parameters:

Name Type Description Default
arms list of dict

The arms of the bandit (test case records).

 required
evaluation_metric EvaluationMetric

The evaluation metric used to provide feedback.

 required

__str__ 

__str__()

Return a string representation of the bandit.

Returns:

Type Description
str

String representation of the evaluation metric.

pull 

pull(action)

Submit prioritized test set for evaluation and get new measurements.

Parameters:

Name Type Description Default
action list of str

The prioritized test suite list.

 required

Returns:

Type Description
EvaluationMetric

The result ("state") of an evaluation by the evaluation metric.

Raises:

Type Description
ValueError

If the action list is empty.