miprometheus.grid_workers

GridWorker

class miprometheus.grid_workers.GridWorker(name='GridWorker', use_gpu=False)

Base abstract class for the grid workers. All grid workers should subclass it and override the relevant methods.

__init__(name='GridWorker', use_gpu=False)

Base constructor for all grid workers:

• Initializes the AppState singleton:

  >>> self.app_state = AppState()
  

• Initializes the Parameter Registry:

  >>> self.params = ParamInterface()
  

• Defines the logger:

  >>> self.logger = logging.getLogger(name=self.name)
  

• Creates parser and adds default worker command line arguments (you can display them with --h).

Parameters:

• name (str) – Name of the worker (DEFAULT: “GridWorker”).
• use_gpu (bool) – Indicates whether the worker should use GPU or not. Value coming from the subclasses (e.g. miprometheus.grid_workers.GridTrainerCPU vs miprometheus.grid_workers.GridTrainerGPU) (DEFAULT: False).

setup_grid_experiment()

Setups the overall grid of experiments.

Base method:

• Parses command line arguments,
• Sets the 3 default sections (training / validation / test) for the param registry, sets seeds to unspecified and disable multiprocessing.

Note

Child classes should override this method, but still call its parent to draw the basic functionality implemented here.

run_grid_experiment()

Main function of the miprometheus.grid_workers.GridWorker, which essentially maps an experiment to available core or device.

Note

Abstract. Should be implemented in the subclasses.

get_available_cpus()

Returns the number of available CPUs on the current machine.

GridTrainerCPU

class miprometheus.grid_workers.GridTrainerCPU(name='GridTrainerCPU', use_gpu=False)

Grid Worker managing several training experiments on CPUs.

Reuses a miprometheus.workers.Trainer (can specify miprometheus.workers.OfflineTrainer or miprometheus.workers.OnlineTrainer) to start one experiment.

__init__(name='GridTrainerCPU', use_gpu=False)

Constructor for the miprometheus.grid_workers.GridTrainerCPU:

• Calls the base constructor to set the worker’s name and add default command lines arguments,
• Adds some GridTrainer specific command line arguments.

Parameters:

• name (str) – Name of the worker (DEFAULT: “GridTrainerCPU”).
• use_gpu (bool) – Indicates whether the worker should use GPU or not.

setup_grid_experiment()

Setups a specific experiment.

• Calls GridWorker.setup_grid_experiment() to parse arguments, sets the 3 default sections (training / validation / test) and sets their torch.utils.data.DataLoader params.
• Verifies that the specified config file is valid,
• Parses it and recursively creates the configurations files for the grid tasks, overwriting specific sections if indicated: grid_overwrite and/or overwrite (task specific),
• Creates the output dir.

run_grid_experiment()

Main function of the miprometheus.grid_workers.GridTrainerCPU.

Maps the grid experiments to CPU cores in the limit of the maximum concurrent runs allowed or maximum available cores.

run_experiment(experiment_configs: str, prefix='')

Setups the overall grid of experiments.

Parameters:

• experiment_configs (str) – Configuration file(s) passed to the trainer using its –c argument. If indicating several config files, they must be separated with coma “,”.
• prefix (str) – Prefix to position before the command string (e.g. ‘cuda-gpupick -n 1’). Optional.

Note

• Visualization is deactivated to avoid any user interaction.
• Command-line arguments such as the logging interval (--li), tensorboard (--t) and log level (--ll) are passed to the used miprometheus.workers.Trainer
• Not using the –model command-line argument of the miprometheus.workers.Trainer to load a pretrained model. Please use instead the configuration parameter load in the model section.

GridTrainerGPU

class miprometheus.grid_workers.GridTrainerGPU(name='GridTrainerGPU', use_gpu=True)

Grid Worker managing several training experiments on GPUs.

Reuses a miprometheus.workers.Trainer (can specify miprometheus.workers.OfflineTrainer or miprometheus.workers.OnlineTrainer) to start one experiment.

Inherits from miprometheus.grid_workers.GridTrainerCPU as the constructor & GridTrainerCPU.setup_grid_experiment() are identical.

__init__(name='GridTrainerGPU', use_gpu=True)

Constructor for the miprometheus.grid_workers.GridTrainerGPU:

• Calls the constructor of miprometheus.grid_workers.GridTrainerCPU as it is identical.

Parameters:

• name (str) – Name of the worker (DEFAULT: “GridTrainerGPU”).
• use_gpu (bool) – Indicates whether the worker should use GPU or not.

setup_grid_experiment()

Setups a specific experiment.

• Calls GridTrainerGPU.setup_grid_experiment() to parse arguments, parse config files etc.
• Checks the presence of CUDA-compatible devices.

run_grid_experiment()

Main function of the miprometheus.grid_workers.GridTrainerGPU.

Maps the grid experiments to CUDA devices in the limit of the maximum concurrent runs allowed.

GridTesterCPU

class miprometheus.grid_workers.GridTesterCPU(name='GridTesterCPU', use_gpu=False)

Implementation of the Grid Tester running on CPUs.

Reuses the miprometheus.workers.Tester to start one test experiment.

__init__(name='GridTesterCPU', use_gpu=False)

Constructor for the miprometheus.grid_workers.GridTesterCPU:

• Calls the base constructor to set the worker’s name and add default command lines arguments,
• Adds some GridTester specific command line arguments.

Parameters:

• name (str) – Name of the worker (DEFAULT: “GridTesterCPU”).
• use_gpu (bool) – Indicates whether the worker should use GPU or not.

setup_grid_experiment()

Setups the overall grid of experiments:

• Calls GridWorker.setup_grid_experiment() to parse arguments,
• Recursively creates the paths to the experiments folders, verifying that they are valid (e.g. they contain a saved model, model_best.pt).

run_grid_experiment()

Main function of the miprometheus.grid_workers.GridTesterCPU.

Maps the grid experiments to CPU cores in the limit of the maximum concurrent runs allowed or maximum available cores.

run_experiment(experiment_path: str, prefix='')

Runs a test on the specified model (experiment_path) using the miprometheus.workers.Tester.

Parameters:

• experiment_path (str) – Path to an experiment folder containing a trained model.
• prefix (str) – Prefix to position before the command string (e.g. ‘cuda-gpupick -n 1’). Optional.

..note:

- Visualization is deactivated to avoid any user interaction.
- Command-line arguments such as the logging interval (``--li``) and log level (``--ll``) are passed              to the :py:class:`miprometheus.workers.Tester`.

GridTesterGPU

class miprometheus.grid_workers.GridTesterGPU(name='GridTesterGPU', use_gpu=True)

Implementation of the GridTester running on GPUs.

Reuses the miprometheus.workers.Tester to start one test experiment.

Inherits from miprometheus.grid_workers.GridTesterCPU as the constructor is identical.

__init__(name='GridTesterGPU', use_gpu=True)

Constructor for the miprometheus.grid_workers.GridTesterGPU:

• Calls the constructor of miprometheus.grid_workers.GridTesterCPU as it is identical.

Parameters:

• name (str) – Name of the worker (DEFAULT: “GridTesterGPU”).
• use_gpu (bool) – Indicates whether the worker should use GPU or not.

setup_grid_experiment()

Setups a specific experiment.

• Calls GridTesterCPU.setup_grid_experiment() to parse arguments, parse config files etc.
• Checks the presence of CUDA-compatible devices.

run_grid_experiment()

Main function of the miprometheus.grid_workers.GridTesterGPU.

Maps the grid experiments to CUDA device in the limit of the maximum concurrent runs allowed or maximum available devices.

GridAnalyzer

class miprometheus.grid_workers.GridAnalyzer(name='GridAnalyzer')

Implementation of the miprometheus.grid_workers.GridAnalyzer.

Post-processes the test results of a grid of experiments and gather them in a csv file.

This csv file will gather the training statistics (seeds, accuracies, terminal conditions,…), the validation statistics and the test statistics.

Inherits from miprometheus.grid_workers.GridWorker.

__init__(name='GridAnalyzer')

Constructor for the miprometheus.grid_workers.GridAnalyzer:

• Calls basic constructor of miprometheus.grid_workers.GridWorker

Parameters:name (str) – Name of the worker (DEFAULT: “GridAnalyzer”).

static check_if_file_exists(dir_, filename_)

Checks if filename_ exists in dir_.

Parameters:

• dir (str) – Path to file.
• filename (str) – Name of the file to be opened and analysed.

Returns:

True if the file exists in the directory, else False

check_file_content(dir_, filename_)

Checks if the number of lines in the file is > 1.

Parameters:

• dir (str) – Path to file.
• filename (str) – Name of the file to be opened and analyzed.

Returns:

True if the number of lines in the file is strictly greater than one.

static get_lines_number(filename_)

Returns the number of lines in filename_.

Parameters:filename (str) – Filepath to be opened and line-read. Returns:Number of lines in the file.

get_experiment_tests(experiment_path_)

Returns a list of folders containing valid test experiments data:

• A configuration (testing_configuration.yaml),
• A csv file containing a data point for the aggregated statistics (testing_set_agg_statistics.csv)

Parameters:experiment_path (str) – Path to experiment (training) folder. Returns:A list of valid test experiment folders.

setup_grid_experiment()

Setups the overall experiment:

• Parses arguments and sets logger level,

• Checks the presence of experiments folder,

• Recursively traverses the experiment folders, cherry-picking subfolders containing:

  • 1. ‘training_configuration.yaml’ (training configuration file),
  • 2. ‘models/model_best.pt’ (checkpoint of the best saved model).

run_experiment(experiment_path: str)

Collects the training / validation / test statistics for a given experiment path.

Analyzes whether the given training experiment folder contains subfolders with test experiments data:

• Loads and parses training configuration file,

• Loads checkpoint with model and training and validation statistics,

• Recursively traverses subdirectories looking for test experiments,

  Note

  We require that the test statistics csv files are valid, i.e. contain at least one line with collected statistics (excluding the header).

• Collects statistics from training, validation (from model checkpoint) and test experiments (from test csv files found in subdirectories).

Parameters:experiment_path (str) – Path to an experiment folder containing a training statistics. Returns:Four dictionaries containing: - Status info (model, problem etc.), - Training statistics, - Validation statistics, - Test statistics.

static merge_list_dicts(list_dicts)

Merges a list of dictionaries by filling the missing fields with spaces into one dict.

Parameters:list_dicts (list) – List of dictionaries, potentially containing different headers, which will be merged. Returns:dict, resulting of the merge.

run_grid_experiment()

Collects four list of dicts from each experiment path contained in self.experiments_lists.

Merges all them together and saves result to a single csv file.