hookeai.model_architectures.rc_base_model.train.training.EarlyStopper

class EarlyStopper(validation_dataset, validation_frequency=1, trigger_tolerance=1, improvement_tolerance=0.01)

Bases: object

Early stopping procedure (implicit regularizaton).

_validation_dataset

Time series data set. Each sample is stored as a dictionary where each feature (key, str) data is a torch.Tensor(2d) of shape (sequence_length, n_features).

Type:

torch.utils.data.Dataset

_validation_frequency

Frequency of validation procedures, i.e., frequency with respect to training epochs at which model is validated to evaluate early stopping criterion.

Type:

int

_trigger_tolerance

Number of consecutive model validation procedures without performance improvement to trigger early stopping.

Type:

int

_improvement_tolerance

Minimum relative improvement required to count as a performance improvement.

Type:

float

_validation_steps_history

Validation steps history.

Type:

list

_validation_loss_history

Validation loss history.

Type:

list

_min_validation_loss

Minimum validation loss.

Type:

float

_n_not_improve

Number of consecutive model validations without improvement.

Type:

int

_best_model_state

Model state corresponding to the best performance.

Type:

dict

_best_optimizer_state

Optimizer state corresponding to the best performance.

Type:

dict

_best_training_epoch

Training epoch corresponding to the best performance.

Type:

int

get_validation_loss_history(self)

Get validation loss history.

is_evaluate_criterion(self, epoch)

Check whether to evaluate early stopping criterion.

evaluate_criterion(self, model, optimizer, epoch, loss_nature='node_features_out', loss_type='mse', loss_kwargs={}, batch_size=1, device_type='cpu')

Evaluate early stopping criterion.

_validate_model(self, model, optimizer, epoch,

loss_nature=’node_features_out’, loss_type=’mse’, loss_kwargs={}, batch_size=1, device_type=’cpu’)

Perform model validation.

load_best_performance_state(self, model, optimizer)

Load minimum validation loss model and optimizer states.

Constructor.

Parameters:

• validation_dataset (torch.utils.data.Dataset) – Time series data set. Each sample is stored as a dictionary where each feature (key, str) data is a torch.Tensor(2d) of shape (sequence_length, n_features).

• validation_frequency (int, default=1) – Frequency of validation procedures, i.e., frequency with respect to training epochs at which model is validated to evaluate early stopping criterion.

• trigger_tolerance (int, default=1) – Number of consecutive model validation procedures without performance improvement to trigger early stopping.

• improvement_tolerance (float, default=1e-2) – Minimum relative improvement required to count as a performance improvement.

List of Public Methods

evaluate_criterion	Evaluate early stopping criterion.
get_validation_loss_history	Get validation loss history.
is_evaluate_criterion	Check whether to evaluate early stopping criterion.
load_best_performance_state	Load minimum validation loss model and optimizer states.

Methods

__init__(validation_dataset, validation_frequency=1, trigger_tolerance=1, improvement_tolerance=0.01)

Constructor.

Parameters:

• validation_dataset (torch.utils.data.Dataset) – Time series data set. Each sample is stored as a dictionary where each feature (key, str) data is a torch.Tensor(2d) of shape (sequence_length, n_features).

• validation_frequency (int, default=1) – Frequency of validation procedures, i.e., frequency with respect to training epochs at which model is validated to evaluate early stopping criterion.

• trigger_tolerance (int, default=1) – Number of consecutive model validation procedures without performance improvement to trigger early stopping.

• improvement_tolerance (float, default=1e-2) – Minimum relative improvement required to count as a performance improvement.

_validate_model(model, optimizer, epoch, loss_nature='features_out', loss_type='mse', loss_kwargs={}, is_normalized_loss=False, batch_size=1, device_type='cpu')

Perform model validation.

Parameters:

• model (torch.nn.Module) – Recurrent neural network model.

• optimizer (torch.optim.Optimizer) – PyTorch optimizer.

• epoch (int) – Training epoch.

• loss_nature ({'features_out',}, default='features_out') –

  Loss nature:

  ’features_out’ : Based on output features

• loss_type ({'mse',}, default='mse') –

  Loss function type:

  ’mse’ : MSE (torch.nn.MSELoss)

• loss_kwargs (dict, default={}) – Arguments of torch.nn._Loss initializer.

• is_normalized_loss (bool, default=False) – If True, then samples prediction loss are computed from normalized output data, False otherwise. Normalization of output data requires that model data scalers are available.

• batch_size (int, default=1) – Number of samples loaded per batch.

• device_type ({'cpu', 'cuda'}, default='cpu') – Type of device on which torch.Tensor is allocated.

Returns:

avg_predict_loss – Average prediction loss per sample.

Return type:

float

evaluate_criterion(model, optimizer, epoch, loss_nature='features_out', loss_type='mse', loss_kwargs={}, is_normalized_loss=False, batch_size=1, device_type='cpu')

Evaluate early stopping criterion.

Parameters:

• model (torch.nn.Module) – Recurrent neural network model.

• optimizer (torch.optim.Optimizer) – PyTorch optimizer.

• epoch (int) – Training epoch.

• loss_nature ({'features_out',}, default='features_out') –

  Loss nature:

  ’features_out’ : Based on output features

• loss_type ({'mse',}, default='mse') –

  Loss function type:

  ’mse’ : MSE (torch.nn.MSELoss)

• loss_kwargs (dict, default={}) – Arguments of torch.nn._Loss initializer.

• is_normalized_loss (bool, default=False) – If True, then samples prediction loss are computed from normalized output data, False otherwise. Normalization of output data requires that model data scalers are available.

• batch_size (int, default=1) – Number of samples loaded per batch.

• device_type ({'cpu', 'cuda'}, default='cpu') – Type of device on which torch.Tensor is allocated.

Returns:

is_stop_training – True if early stopping criterion has been triggered, False otherwise.

Return type:

bool

get_validation_loss_history()

Get validation loss history.

Returns:

validation_loss_history – Validation loss history.

Return type:

list[float]

is_evaluate_criterion(epoch)

Check whether to evaluate early stopping criterion.

Parameters:

epoch (int) – Training epoch.

Returns:

is_evaluate_criterion – If True, then early stopping criterion should be evaluated, False otherwise.

Return type:

bool

load_best_performance_state(model, optimizer)

Load minimum validation loss model and optimizer states.

Both model and optimizer are updated ‘in-place’ with stored state data.

Parameters:

• model (torch.nn.Module) – Recurrent neural network model.

• optimizer (torch.optim.Optimizer) – PyTorch optimizer.

Returns:

best_training_epoch – Training epoch corresponding to the best performance.

Return type:

int