Skip to content

Environment

coleman4hcs.environment

Environment for agent-bandit interactions in Coleman4HCS.

This module contains the Environment class which simulates the agent's interactions and collects results during the reinforcement learning process. The environment provides a platform where agents learn from interactions with the bandit to make better decisions over time.

The module also contains the following features:

  • Mechanism to reset the environment and agent's memories.
  • Ability to run a single experiment or multiple experiments.
  • Support for both simple and contextual agents.
  • Periodic saving of experiments to handle long-running experiments.
  • Facilities for creating and storing results obtained during experiments.
  • Support for scenarios with variants, commonly found in Heterogeneous Computing Systems (HCS).
  • Helper methods for loading and saving experiment states for recovery purposes.

Classes:

Name Description
Environment

Represents the learning environment where agents interact with bandits.

Environment

The environment class that simulates the agent's interactions and collects results.

Parameters:

Name Type Description Default
agents list

List of agent instances participating in the simulation.

 required
scenario_provider object

The scenario provider supplying test case data.

 required
evaluation_metric EvaluationMetric

The evaluation metric used to assess prioritization performance.

 required

Attributes:

Name Type Description
agents list

List of agent instances.
scenario_provider object

The scenario provider instance.
evaluation_metric EvaluationMetric

The evaluation metric instance.
monitor MonitorCollector or None

Monitor for collecting feedback during the process.
variant_monitors dict

Dictionary of monitors for each variant.

__init__ 

__init__(agents, scenario_provider, evaluation_metric)

Initialize the Environment.

Parameters:

Name Type Description Default
agents list

List of agent instances participating in the simulation.

 required
scenario_provider object

The scenario provider supplying test case data.

 required
evaluation_metric EvaluationMetric

The evaluation metric used to assess prioritization performance.

 required

create_file 

create_file(name)

Create a file to store the results obtained during the experiment.

Parameters:

Name Type Description Default
name str

The name of the file to store the results.

 required

load_experiment 

load_experiment(experiment)

Load a backup of the experiment.

Parameters:

Name Type Description Default
experiment int

The current experiment number.

 required

Returns:

Type Description
tuple

A tuple containing the restore step, agents, monitor, variant monitors, and bandit.

reset 

reset()

Reset the environment for a new simulation.

reset_agents_memory 

reset_agents_memory()

Reset all agents' memory to an initial state.

run 

run(experiments=1, trials=100, bandit_type=EvaluationMetricBandit, restore=True)

Execute a simulation over multiple experiments.

Parameters:

Name Type Description Default
experiments int

Number of experiments. Default is 1.

 1
trials int

The max number of scenarios that will be analyzed. Default is 100.

 100
bandit_type type

The bandit class to use. Default is EvaluationMetricBandit.

 EvaluationMetricBandit
restore bool

Restore the experiment if it fails. Default is True.

 True

run_prioritization 

run_prioritization(agent, bandit, bandit_duration, experiment, t, virtual_scenario)

Run the prioritization process for a given agent and scenario.

Parameters:

Name Type Description Default
agent Agent

The agent that is being used for the prioritization.

 required
bandit Bandit

The bandit mechanism used for choosing actions.

 required
bandit_duration float

Time taken by the bandit process.

 required
experiment int

The current experiment number.

 required
t int

The current step or iteration of the simulation.

 required
virtual_scenario VirtualScenario

The virtual scenario being considered.

 required

Returns:

Type Description
tuple

A tuple containing the chosen action, the ending time, the experiment name, and the starting time.

run_prioritization_hcs 

run_prioritization_hcs(agent, action, avail_time_ratio, bandit_duration, end, exp_name, experiment, start, t, virtual_scenario)

Run the prioritization process for a given agent and HCS scenario.

Parameters:

Name Type Description Default
agent Agent

The agent that is being used for the prioritization.

 required
action list of str

The chosen action by the agent.

 required
avail_time_ratio float

The available time ratio for the experiment.

 required
bandit_duration float

Time taken by the bandit process.

 required
end float

The ending time of the process.

 required
exp_name str

The name of the experiment.

 required
experiment int

The current experiment number.

 required
start float

The starting time of the process.

 required
t int

The current step or iteration of the simulation.

 required
virtual_scenario VirtualHCSScenario

The virtual HCS scenario being considered.

 required

run_single 

run_single(experiment, trials=100, bandit_type=EvaluationMetricBandit, restore=True)

Execute a single simulation experiment.

Parameters:

Name Type Description Default
experiment int

Current experiment number.

 required
trials int

The max number of scenarios that will be analyzed. Default is 100.

 100
bandit_type type

The bandit class to use. Default is EvaluationMetricBandit.

 EvaluationMetricBandit
restore bool

Restore the experiment if it fails (e.g., energy down). Default is True.

 True

save_experiment 

save_experiment(experiment, t, bandit)

Save a backup for the experiment.

Parameters:

Name Type Description Default
experiment int

The current experiment number.

 required
t int

The current step or iteration of the simulation.

 required
bandit Bandit

The current bandit being used in the simulation.

 required

Raises:

Type Description
Exception

If there is an error saving the experiment.

save_periodically 

save_periodically(restore, t, experiment, bandit, interval=50000)

Save the experiment periodically based on a predefined interval.

Parameters:

Name Type Description Default
restore bool

Flag to indicate if the experiment should be restored.

 required
t int

The current step or iteration of the simulation.

 required
experiment int

The current experiment number.

 required
bandit Bandit

The current bandit being used in the simulation.

 required
interval int

The interval at which the experiment should be saved. Default is 50000.

 50000

store_experiment 

store_experiment(csv_file_name)

Save the results obtained during the experiment.

Parameters:

Name Type Description Default
csv_file_name str

The name of the file to store the results.

 required