pypots.optim package

pypots.optim.adam

The optimizer wrapper for PyTorch Adam.

class pypots.optim.adam.Adam(lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False, lr_scheduler=None)

Bases: Optimizer

The optimizer wrapper for PyTorch Adam torch.optim.Adam.

Parameters:

• lr (float) – The learning rate of the optimizer.

• betas (Tuple[float, float]) – Coefficients used for computing running averages of gradient and its square.

• eps (float) – Term added to the denominator to improve numerical stability.

• weight_decay (float) – Weight decay (L2 penalty).

• amsgrad (bool) – Whether to use the AMSGrad variant of this algorithm from the paper [42].

• lr_scheduler (pypots.optim.lr_scheduler.base.LRScheduler) – The learning rate scheduler of the optimizer.

init_optimizer(params)

Initialize the torch optimizer wrapped by this class.

Parameters:

params (Iterable) – An iterable of torch.Tensor or dict. Specifies what Tensors should be optimized.

Return type:

None

add_param_group(param_group)

Add a param group to the optimizer param_groups.

Parameters:

param_group (dict) – Specifies the parameters to be optimized and group-specific optimization options.

Return type:

None

load_state_dict(state_dict)

Loads the optimizer state.

Parameters:

state_dict – Optimizer state. It should be an object returned from state_dict().

Return type:

None

state_dict()

Returns the state of the optimizer as a dict.

Returns:

The state dict of the optimizer, which contains two entries: 1). state - a dict holding current optimization state. Its content differs between optimizer classes. 2). param_groups - a list containing all parameter groups where each parameter group is a dict

Return type:

state_dict

step(closure=None)

Performs a single optimization step (parameter update).

Parameters:

closure (Optional[Callable]) – A closure that reevaluates the model and returns the loss. Optional for most optimizers. Refer to the torch.optim.Optimizer.step() docstring for more details.

Return type:

None

zero_grad(set_to_none=True)

Sets the gradients of all optimized torch.Tensor to zero.

Parameters:

set_to_none (bool) – Instead of setting to zero, set the grads to None. Refer to the torch.optim.Optimizer.zero_grad() docstring for more details.

Return type:

None

pypots.optim.adamw

The optimizer wrapper for PyTorch AdamW.

class pypots.optim.adamw.AdamW(lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.01, amsgrad=False, lr_scheduler=None)

Bases: Optimizer

The optimizer wrapper for PyTorch AdamW torch.optim.AdamW.

Parameters:

• lr (float) – The learning rate of the optimizer.

• betas (Tuple[float, float]) – Coefficients used for computing running averages of gradient and its square.

• eps (float) – Term added to the denominator to improve numerical stability.

• weight_decay (float) – Weight decay (L2 penalty).

• amsgrad (bool) – Whether to use the AMSGrad variant of this algorithm from the paper [42].

• lr_scheduler (pypots.optim.lr_scheduler.base.LRScheduler) – The learning rate scheduler of the optimizer.

init_optimizer(params)

Initialize the torch optimizer wrapped by this class.

Parameters:

params (Iterable) – An iterable of torch.Tensor or dict. Specifies what Tensors should be optimized.

Return type:

None

add_param_group(param_group)

Add a param group to the optimizer param_groups.

Parameters:

param_group (dict) – Specifies the parameters to be optimized and group-specific optimization options.

Return type:

None

load_state_dict(state_dict)

Loads the optimizer state.

Parameters:

state_dict – Optimizer state. It should be an object returned from state_dict().

Return type:

None

state_dict()

Returns the state of the optimizer as a dict.

Returns:

The state dict of the optimizer, which contains two entries: 1). state - a dict holding current optimization state. Its content differs between optimizer classes. 2). param_groups - a list containing all parameter groups where each parameter group is a dict

Return type:

state_dict

step(closure=None)

Performs a single optimization step (parameter update).

Parameters:

closure (Optional[Callable]) – A closure that reevaluates the model and returns the loss. Optional for most optimizers. Refer to the torch.optim.Optimizer.step() docstring for more details.

Return type:

None

zero_grad(set_to_none=True)

Sets the gradients of all optimized torch.Tensor to zero.

Parameters:

set_to_none (bool) – Instead of setting to zero, set the grads to None. Refer to the torch.optim.Optimizer.zero_grad() docstring for more details.

Return type:

None

pypots.optim.adagrad

The optimizer wrapper for PyTorch Adagrad.

class pypots.optim.adagrad.Adagrad(lr=0.01, lr_decay=0, weight_decay=0.01, initial_accumulator_value=0.01, eps=1e-08, lr_scheduler=None)

Bases: Optimizer

The optimizer wrapper for PyTorch Adagrad torch.optim.Adagrad.

Parameters:

• lr (float) – The learning rate of the optimizer.

• lr_decay (float) – Learning rate decay.

• weight_decay (float) – Weight decay (L2 penalty).

• eps (float) – Term added to the denominator to improve numerical stability.

• initial_accumulator_value (float) – A floating point value. Starting value for the accumulators, must be positive.

• lr_scheduler (pypots.optim.lr_scheduler.base.LRScheduler) – The learning rate scheduler of the optimizer.

init_optimizer(params)

Initialize the torch optimizer wrapped by this class.

Parameters:

params (Iterable) – An iterable of torch.Tensor or dict. Specifies what Tensors should be optimized.

Return type:

None

add_param_group(param_group)

Add a param group to the optimizer param_groups.

Parameters:

param_group (dict) – Specifies the parameters to be optimized and group-specific optimization options.

Return type:

None

load_state_dict(state_dict)

Loads the optimizer state.

Parameters:

state_dict – Optimizer state. It should be an object returned from state_dict().

Return type:

None

state_dict()

Returns the state of the optimizer as a dict.

Returns:

The state dict of the optimizer, which contains two entries: 1). state - a dict holding current optimization state. Its content differs between optimizer classes. 2). param_groups - a list containing all parameter groups where each parameter group is a dict

Return type:

state_dict

step(closure=None)

Performs a single optimization step (parameter update).

Parameters:

closure (Optional[Callable]) – A closure that reevaluates the model and returns the loss. Optional for most optimizers. Refer to the torch.optim.Optimizer.step() docstring for more details.

Return type:

None

zero_grad(set_to_none=True)

Sets the gradients of all optimized torch.Tensor to zero.

Parameters:

set_to_none (bool) – Instead of setting to zero, set the grads to None. Refer to the torch.optim.Optimizer.zero_grad() docstring for more details.

Return type:

None

pypots.optim.rmsprop

The optimizer wrapper for PyTorch RMSprop.

class pypots.optim.rmsprop.RMSprop(lr=0.001, momentum=0, alpha=0.99, eps=1e-08, centered=False, weight_decay=0, lr_scheduler=None)

Bases: Optimizer

The optimizer wrapper for PyTorch RMSprop torch.optim.RMSprop.

Parameters:

• lr (float) – The learning rate of the optimizer.

• momentum (float) – Momentum factor.

• alpha (float) – Smoothing constant.

• eps (float) – Term added to the denominator to improve numerical stability.

• centered (bool) – If True, compute the centered RMSProp, the gradient is normalized by an estimation of its variance

• weight_decay (float) – Weight decay (L2 penalty).

• lr_scheduler (pypots.optim.lr_scheduler.base.LRScheduler) – The learning rate scheduler of the optimizer.

init_optimizer(params)

Initialize the torch optimizer wrapped by this class.

Parameters:

params (Iterable) – An iterable of torch.Tensor or dict. Specifies what Tensors should be optimized.

Return type:

None

add_param_group(param_group)

Add a param group to the optimizer param_groups.

Parameters:

param_group (dict) – Specifies the parameters to be optimized and group-specific optimization options.

Return type:

None

load_state_dict(state_dict)

Loads the optimizer state.

Parameters:

state_dict – Optimizer state. It should be an object returned from state_dict().

Return type:

None

state_dict()

Returns the state of the optimizer as a dict.

Returns:

The state dict of the optimizer, which contains two entries: 1). state - a dict holding current optimization state. Its content differs between optimizer classes. 2). param_groups - a list containing all parameter groups where each parameter group is a dict

Return type:

state_dict

step(closure=None)

Performs a single optimization step (parameter update).

Parameters:

closure (Optional[Callable]) – A closure that reevaluates the model and returns the loss. Optional for most optimizers. Refer to the torch.optim.Optimizer.step() docstring for more details.

Return type:

None

zero_grad(set_to_none=True)

Sets the gradients of all optimized torch.Tensor to zero.

Parameters:

set_to_none (bool) – Instead of setting to zero, set the grads to None. Refer to the torch.optim.Optimizer.zero_grad() docstring for more details.

Return type:

None

pypots.optim.sgd

The optimizer wrapper for PyTorch SGD torch.optim.SGD.

class pypots.optim.sgd.SGD(lr=0.001, momentum=0, weight_decay=0, dampening=0, nesterov=False, lr_scheduler=None)

Bases: Optimizer

The optimizer wrapper for PyTorch SGD torch.optim.SGD.

Parameters:

• lr (float) – The learning rate of the optimizer.

• momentum (float) – Momentum factor.

• weight_decay (float) – Weight decay (L2 penalty).

• dampening (float) – Dampening for momentum.

• nesterov (bool) – Whether to enable Nesterov momentum.

• lr_scheduler (pypots.optim.lr_scheduler.base.LRScheduler) – The learning rate scheduler of the optimizer.

init_optimizer(params)

Initialize the torch optimizer wrapped by this class.

Parameters:

params (Iterable) – An iterable of torch.Tensor or dict. Specifies what Tensors should be optimized.

Return type:

None

add_param_group(param_group)

Add a param group to the optimizer param_groups.

Parameters:

param_group (dict) – Specifies the parameters to be optimized and group-specific optimization options.

Return type:

None

load_state_dict(state_dict)

Loads the optimizer state.

Parameters:

state_dict – Optimizer state. It should be an object returned from state_dict().

Return type:

None

state_dict()

Returns the state of the optimizer as a dict.

Returns:

The state dict of the optimizer, which contains two entries: 1). state - a dict holding current optimization state. Its content differs between optimizer classes. 2). param_groups - a list containing all parameter groups where each parameter group is a dict

Return type:

state_dict

step(closure=None)

Performs a single optimization step (parameter update).

Parameters:

closure (Optional[Callable]) – A closure that reevaluates the model and returns the loss. Optional for most optimizers. Refer to the torch.optim.Optimizer.step() docstring for more details.

Return type:

None

zero_grad(set_to_none=True)

Sets the gradients of all optimized torch.Tensor to zero.

Parameters:

set_to_none (bool) – Instead of setting to zero, set the grads to None. Refer to the torch.optim.Optimizer.zero_grad() docstring for more details.

Return type:

None

pypots.optim.base

The base wrapper for PyTorch optimizers (https://pytorch.org/docs/stable/optim.html#algorithms), also is the base class for all optimizers in pypots.optim.

The optimizers in pypots.optim are all wrappers for PyTorch optimizers. pypots.optim.optimizers inherent all functionalities from torch.optim.optimizers (so you can see many docstrings are copied from torch), but are more powerful. So far, they are designed to:

1). separate the hyperparameters of models and optimizers in PyPOTS, so that users don’t have to put all hyperparameters in one place, which could result in a mess and be not readable;

2). provide additional functionalities, such as learning rate scheduling, etc.;

class pypots.optim.base.Optimizer(lr, lr_scheduler=None)

Bases: ABC

The base wrapper for PyTorch optimizers, also is the base class for all optimizers in PyPOTS.

Parameters:

• lr (float) – The learning rate of the optimizer.

• lr_scheduler (pypots.optim.lr_scheduler.base.LRScheduler) – The learning rate scheduler of the optimizer.

Attributes:

torch_optimizer – The torch optimizer wrapped by this class.

abstract init_optimizer(params)

Initialize the torch optimizer wrapped by this class.

Parameters:

params (Iterable) – An iterable of torch.Tensor or dict. Specifies what Tensors should be optimized.

Return type:

None

add_param_group(param_group)

Add a param group to the optimizer param_groups.

Parameters:

param_group (dict) – Specifies the parameters to be optimized and group-specific optimization options.

Return type:

None

load_state_dict(state_dict)

Loads the optimizer state.

Parameters:

state_dict – Optimizer state. It should be an object returned from state_dict().

Return type:

None

state_dict()

Returns the state of the optimizer as a dict.

Returns:

The state dict of the optimizer, which contains two entries: 1). state - a dict holding current optimization state. Its content differs between optimizer classes. 2). param_groups - a list containing all parameter groups where each parameter group is a dict

Return type:

state_dict

step(closure=None)

Performs a single optimization step (parameter update).

Parameters:

closure (Optional[Callable]) – A closure that reevaluates the model and returns the loss. Optional for most optimizers. Refer to the torch.optim.Optimizer.step() docstring for more details.

Return type:

None

zero_grad(set_to_none=True)

Sets the gradients of all optimized torch.Tensor to zero.

Parameters:

set_to_none (bool) – Instead of setting to zero, set the grads to None. Refer to the torch.optim.Optimizer.zero_grad() docstring for more details.

Return type:

None

pypots.optim.lr_scheduler

Learning rate schedulers available in PyPOTS. Their functionalities are the same with those in PyTorch, the only difference that is also why we implement them is that you don’t have to pass according optimizers into them immediately while initializing them. Instead, you can pass them into pypots.optim.base.Optimizer after initialization and call their init_scheduler() method in pypots.optim.base.Optimizer.init_optimizer() to initialize schedulers together with optimizers.

class pypots.optim.lr_scheduler.LambdaLR(lr_lambda, last_epoch=-1, verbose=False)

Bases: LRScheduler

Sets the learning rate of each parameter group to the initial lr times a given function. When last_epoch=-1, sets initial lr as lr.

Parameters:

• lr_lambda (Callable or list,) – A function which computes a multiplicative factor given an integer parameter epoch, or a list of such functions, one for each group in optimizer.param_groups.

• last_epoch (int,) – The index of last epoch. Default: -1.

• verbose (bool,) – If True, prints a message to stdout for each update. Default: False.

Notes

This class works the same with torch.optim.lr_scheduler.LambdaLR. The only difference that is also why we implement them is that you don’t have to pass according optimizers into them immediately while initializing them.

Example

>>> lambda1 = lambda epoch: epoch // 30
>>> scheduler = LambdaLR(lr_lambda=lambda1)
>>> adam = pypots.optim.Adam(lr=1e-3, lr_scheduler=scheduler)

init_scheduler(optimizer)

Initialize the scheduler. This method should be called in pypots.optim.base.Optimizer.init_optimizer() to initialize the scheduler together with the optimizer.

Parameters:

optimizer (torch.optim.Optimizer) – The optimizer to be scheduled.

get_lr()

Compute learning rate.

get_last_lr()

Return last computed learning rate by current scheduler.

static print_lr(is_verbose, group, lr)

Display the current learning rate.

step()

Step could be called after every batch update. This should be called in pypots.optim.base.Optimizer.step() after pypots.optim.base.Optimizer.torch_optimizer.step().

class pypots.optim.lr_scheduler.MultiplicativeLR(lr_lambda, last_epoch=-1, verbose=False)

Bases: LRScheduler

Multiply the learning rate of each parameter group by the factor given in the specified function. When last_epoch=-1, sets initial lr as lr.

Parameters:

• lr_lambda (Callable or list,) – A function which computes a multiplicative factor given an integer parameter epoch, or a list of such functions, one for each group in optimizer.param_groups.

• last_epoch (int,) – The index of last epoch. Default: -1.

• verbose (bool,) – If True, prints a message to stdout for each update. Default: False.

Notes

This class works the same with torch.optim.lr_scheduler.MultiplicativeLR. The only difference that is also why we implement them is that you don’t have to pass according optimizers into them immediately while initializing them.

Example

>>> lmbda = lambda epoch: 0.95
>>> # xdoctest: +SKIP
>>> scheduler = MultiplicativeLR(lr_lambda=lmbda)
>>> adam = pypots.optim.Adam(lr=1e-3, lr_scheduler=scheduler)

init_scheduler(optimizer)

Initialize the scheduler. This method should be called in pypots.optim.base.Optimizer.init_optimizer() to initialize the scheduler together with the optimizer.

Parameters:

optimizer (torch.optim.Optimizer) – The optimizer to be scheduled.

get_lr()

Compute learning rate.

get_last_lr()

Return last computed learning rate by current scheduler.

static print_lr(is_verbose, group, lr)

Display the current learning rate.

step()

Step could be called after every batch update. This should be called in pypots.optim.base.Optimizer.step() after pypots.optim.base.Optimizer.torch_optimizer.step().

class pypots.optim.lr_scheduler.StepLR(step_size, gamma=0.1, last_epoch=-1, verbose=False)

Bases: LRScheduler

Decays the learning rate of each parameter group by gamma every step_size epochs. Notice that such decay can happen simultaneously with other changes to the learning rate from outside this scheduler. When last_epoch=-1, sets initial lr as lr.

Parameters:

• step_size (int,) – Period of learning rate decay.

• gamma (float, default=0.1,) – Multiplicative factor of learning rate decay.

• last_epoch (int) – The index of last epoch. Default: -1.

• verbose (bool) – If True, prints a message to stdout for each update. Default: False.

Notes

This class works the same with torch.optim.lr_scheduler.StepLR. The only difference that is also why we implement them is that you don’t have to pass according optimizers into them immediately while initializing them.

Example

>>> # Assuming optimizer uses lr = 0.05 for all groups
>>> # lr = 0.05     if epoch < 30
>>> # lr = 0.005    if 30 <= epoch < 60
>>> # lr = 0.0005   if 60 <= epoch < 90
>>> # ...
>>> # xdoctest: +SKIP
>>> scheduler = StepLR(step_size=30, gamma=0.1)
>>> adam = pypots.optim.Adam(lr=1e-3, lr_scheduler=scheduler)

get_lr()

Compute learning rate.

get_last_lr()

Return last computed learning rate by current scheduler.

init_scheduler(optimizer)

Initialize the scheduler. This method should be called in pypots.optim.base.Optimizer.init_optimizer() to initialize the scheduler together with the optimizer.

Parameters:

optimizer (torch.optim.Optimizer) – The optimizer to be scheduled.

static print_lr(is_verbose, group, lr)

Display the current learning rate.

step()

Step could be called after every batch update. This should be called in pypots.optim.base.Optimizer.step() after pypots.optim.base.Optimizer.torch_optimizer.step().

class pypots.optim.lr_scheduler.MultiStepLR(milestones, gamma=0.1, last_epoch=-1, verbose=False)

Bases: LRScheduler

Decays the learning rate of each parameter group by gamma once the number of epoch reaches one of the milestones. Notice that such decay can happen simultaneously with other changes to the learning rate from outside this scheduler. When last_epoch=-1, sets initial lr as lr.

Parameters:

• milestones (list,) – List of epoch indices. Must be increasing.

• gamma (float, default=0.1,) – Multiplicative factor of learning rate decay.

• last_epoch (int) – The index of last epoch. Default: -1.

• verbose (bool) – If True, prints a message to stdout for each update. Default: False.

Notes

This class works the same with torch.optim.lr_scheduler.MultiStepLR. The only difference that is also why we implement them is that you don’t have to pass according optimizers into them immediately while initializing them.

Example

>>> # Assuming optimizer uses lr = 0.05 for all groups
>>> # lr = 0.05     if epoch < 30
>>> # lr = 0.005    if 30 <= epoch < 80
>>> # lr = 0.0005   if epoch >= 80
>>> # xdoctest: +SKIP
>>> scheduler = MultiStepLR(milestones=[30,80], gamma=0.1)
>>> adam = pypots.optim.Adam(lr=1e-3, lr_scheduler=scheduler)

get_lr()

Compute learning rate.

get_last_lr()

Return last computed learning rate by current scheduler.

init_scheduler(optimizer)

Initialize the scheduler. This method should be called in pypots.optim.base.Optimizer.init_optimizer() to initialize the scheduler together with the optimizer.

Parameters:

optimizer (torch.optim.Optimizer) – The optimizer to be scheduled.

static print_lr(is_verbose, group, lr)

Display the current learning rate.

step()

Step could be called after every batch update. This should be called in pypots.optim.base.Optimizer.step() after pypots.optim.base.Optimizer.torch_optimizer.step().

class pypots.optim.lr_scheduler.ConstantLR(factor=0.3333333333333333, total_iters=5, last_epoch=-1, verbose=False)

Bases: LRScheduler

Decays the learning rate of each parameter group by a small constant factor until the number of epoch reaches a pre-defined milestone: total_iters. Notice that such decay can happen simultaneously with other changes to the learning rate from outside this scheduler. When last_epoch=-1, sets initial lr as lr.

Parameters:

• factor (float, default=1./3.) – The number we multiply learning rate until the milestone.

• total_iters (int, default=5,) – The number of steps that the scheduler decays the learning rate.

• last_epoch (int) – The index of last epoch. Default: -1.

• verbose (bool) – If True, prints a message to stdout for each update. Default: False.

Notes

This class works the same with torch.optim.lr_scheduler.ConstantLR. The only difference that is also why we implement them is that you don’t have to pass according optimizers into them immediately while initializing them.

Example

>>> # Assuming optimizer uses lr = 0.05 for all groups
>>> # lr = 0.025   if epoch == 0
>>> # lr = 0.025   if epoch == 1
>>> # lr = 0.025   if epoch == 2
>>> # lr = 0.025   if epoch == 3
>>> # lr = 0.05    if epoch >= 4
>>> # xdoctest: +SKIP
>>> scheduler = ConstantLR(factor=0.5, total_iters=4)
>>> adam = pypots.optim.Adam(lr=1e-3, lr_scheduler=scheduler)

get_lr()

Compute learning rate.

get_last_lr()

Return last computed learning rate by current scheduler.

init_scheduler(optimizer)

Initialize the scheduler. This method should be called in pypots.optim.base.Optimizer.init_optimizer() to initialize the scheduler together with the optimizer.

Parameters:

optimizer (torch.optim.Optimizer) – The optimizer to be scheduled.

static print_lr(is_verbose, group, lr)

Display the current learning rate.

step()

Step could be called after every batch update. This should be called in pypots.optim.base.Optimizer.step() after pypots.optim.base.Optimizer.torch_optimizer.step().

class pypots.optim.lr_scheduler.ExponentialLR(gamma, last_epoch=-1, verbose=False)

Bases: LRScheduler

Decays the learning rate of each parameter group by gamma every epoch. When last_epoch=-1, sets initial lr as lr.

Parameters:

• gamma (float,) – Multiplicative factor of learning rate decay.

• last_epoch (int) – The index of last epoch. Default: -1.

• verbose (bool) – If True, prints a message to stdout for each update. Default: False.

Notes

This class works the same with torch.optim.lr_scheduler.ExponentialLR. The only difference that is also why we implement them is that you don’t have to pass according optimizers into them immediately while initializing them.

Example

>>> scheduler = ExponentialLR(gamma=0.1)
>>> adam = pypots.optim.Adam(lr=1e-3, lr_scheduler=scheduler)

get_lr()

Compute learning rate.

get_last_lr()

Return last computed learning rate by current scheduler.

init_scheduler(optimizer)

Initialize the scheduler. This method should be called in pypots.optim.base.Optimizer.init_optimizer() to initialize the scheduler together with the optimizer.

Parameters:

optimizer (torch.optim.Optimizer) – The optimizer to be scheduled.

static print_lr(is_verbose, group, lr)

Display the current learning rate.

step()

Step could be called after every batch update. This should be called in pypots.optim.base.Optimizer.step() after pypots.optim.base.Optimizer.torch_optimizer.step().

class pypots.optim.lr_scheduler.LinearLR(start_factor=0.3333333333333333, end_factor=1.0, total_iters=5, last_epoch=-1, verbose=False)

Bases: LRScheduler

Decays the learning rate of each parameter group by linearly changing small multiplicative factor until the number of epoch reaches a pre-defined milestone: total_iters. Notice that such decay can happen simultaneously with other changes to the learning rate from outside this scheduler. When last_epoch=-1, sets initial lr as lr.

Parameters:

• start_factor (float, default=1.0 / 3,) – The number we multiply learning rate in the first epoch. The multiplication factor changes towards end_factor in the following epochs.

• end_factor (float, default=1.0,) – The number we multiply learning rate at the end of linear changing process.

• total_iters (int, default=5,) – The number of iterations that multiplicative factor reaches to 1.

• last_epoch (int) – The index of last epoch. Default: -1.

• verbose (bool) – If True, prints a message to stdout for each update. Default: False.

Notes

This class works the same with torch.optim.lr_scheduler.LinearLR. The only difference that is also why we implement them is that you don’t have to pass according optimizers into them immediately while initializing them.

Example

>>> # Assuming optimizer uses lr = 0.05 for all groups
>>> # lr = 0.025    if epoch == 0
>>> # lr = 0.03125  if epoch == 1
>>> # lr = 0.0375   if epoch == 2
>>> # lr = 0.04375  if epoch == 3
>>> # lr = 0.05    if epoch >= 4
>>> # xdoctest: +SKIP
>>> scheduler = LinearLR(start_factor=0.5, total_iters=4)
>>> adam = pypots.optim.Adam(lr=1e-3, lr_scheduler=scheduler)

get_lr()

Compute learning rate.

get_last_lr()

Return last computed learning rate by current scheduler.

init_scheduler(optimizer)

Initialize the scheduler. This method should be called in pypots.optim.base.Optimizer.init_optimizer() to initialize the scheduler together with the optimizer.

Parameters:

optimizer (torch.optim.Optimizer) – The optimizer to be scheduled.

static print_lr(is_verbose, group, lr)

Display the current learning rate.

step()

Step could be called after every batch update. This should be called in pypots.optim.base.Optimizer.step() after pypots.optim.base.Optimizer.torch_optimizer.step().