chemicalchecker.tool.siamese.callbacks.CyclicLR

class CyclicLR(base_lr=0.001, max_lr=0.006, step_size=2000.0, mode='triangular', gamma=1.0, scale_fn=None, scale_mode='cycle')[source]

Bases: Callback

This callback implements a cyclical learning rate policy (CLR). The method cycles the learning rate between two boundaries with some constant frequency, as detailed in this paper (https://arxiv.org/abs/1506.01186). The amplitude of the cycle can be scaled on a per-iteration or per-cycle basis. This class has three built-in policies, as put forth in the paper. “triangular”:

A basic triangular cycle w/ no amplitude scaling.

“triangular2”:: A basic triangular cycle that scales initial amplitude by half each cycle.
“exp_range”:: A cycle that scales initial amplitude by gamma**(cycle iterations) at each cycle iteration.

For more detail, please see paper.

# Example

```python

clr = CyclicLR(base_lr=0.001, max_lr=0.006,: step_size=2000., mode=’triangular’)

model.fit(X_train, Y_train, callbacks=[clr])

```

Class also supports custom scaling functions:

```python

clr_fn = lambda x: 0.5*(1+np.sin(x*np.pi/2.)) clr = CyclicLR(base_lr=0.001, max_lr=0.006,

step_size=2000., scale_fn=clr_fn, scale_mode=’cycle’)

model.fit(X_train, Y_train, callbacks=[clr])

```

# Arguments

base_lr: initial learning rate which is the: lower boundary in the cycle.
max_lr: upper boundary in the cycle. Functionally,: it defines the cycle amplitude (max_lr - base_lr). The lr at any cycle is the sum of base_lr and some scaling of the amplitude; therefore max_lr may not actually be reached depending on scaling function.
step_size: number of training iterations per: half cycle. Authors suggest setting step_size 2-8 x training iterations in epoch.
mode: one of {triangular, triangular2, exp_range}.: Default ‘triangular’. Values correspond to policies detailed above. If scale_fn is not None, this argument is ignored.
gamma: constant in ‘exp_range’ scaling function:: gamma**(cycle iterations)
scale_fn: Custom scaling policy defined by a single: argument lambda function, where 0 <= scale_fn(x) <= 1 for all x >= 0. mode paramater is ignored
scale_mode: {‘cycle’, ‘iterations’}.: Defines whether scale_fn is evaluated on cycle number or cycle iterations (training iterations since start of cycle). Default is ‘cycle’.

Methods

`clr`
`on_batch_begin`	A backwards compatibility alias for on_train_batch_begin.
`on_batch_end`	A backwards compatibility alias for on_train_batch_end.
`on_epoch_begin`	Called at the start of an epoch.
`on_epoch_end`	Called at the end of an epoch.
`on_predict_batch_begin`	Called at the beginning of a batch in predict methods.
`on_predict_batch_end`	Called at the end of a batch in predict methods.
`on_predict_begin`	Called at the beginning of prediction.
`on_predict_end`	Called at the end of prediction.
`on_test_batch_begin`	Called at the beginning of a batch in evaluate methods.
`on_test_batch_end`	Called at the end of a batch in evaluate methods.
`on_test_begin`	Called at the beginning of evaluation or validation.
`on_test_end`	Called at the end of evaluation or validation.
`on_train_batch_begin`	Called at the beginning of a training batch in fit methods.
`on_train_batch_end`	Called at the end of a training batch in fit methods.
`on_train_begin`	Called at the beginning of training.
`on_train_end`	Called at the end of training.
`set_model`
`set_params`

on_batch_begin(batch, logs=None): A backwards compatibility alias for on_train_batch_begin.

on_batch_end(epoch, logs=None)[source]: A backwards compatibility alias for on_train_batch_end.

on_epoch_begin(epoch, logs=None)

Called at the start of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Parameters:

epoch – Integer, index of epoch.
logs – Dict. Currently no data is passed to this argument for this method but that may change in the future.

on_epoch_end(epoch, logs=None)

Called at the end of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Parameters:

epoch – Integer, index of epoch.
logs – Dict, metric results for this training epoch, and for the validation epoch if validation is performed. Validation result keys are prefixed with val_. For training epoch, the values of the Model’s metrics are returned. Example: {‘loss’: 0.2, ‘accuracy’: 0.7}.

on_predict_batch_begin(batch, logs=None)

Called at the beginning of a batch in predict methods.

Subclasses should override for any actions to run.

Note that if the steps_per_execution argument to compile in tf.keras.Model is set to N, this method will only be called every N batches.

Parameters:

batch – Integer, index of batch within the current epoch.
logs – Dict. Currently no data is passed to this argument for this method but that may change in the future.

on_predict_batch_end(batch, logs=None)

Called at the end of a batch in predict methods.

Subclasses should override for any actions to run.

Note that if the steps_per_execution argument to compile in tf.keras.Model is set to N, this method will only be called every N batches.

Parameters:

batch – Integer, index of batch within the current epoch.
logs – Dict. Aggregated metric results up until this batch.

on_predict_begin(logs=None)

Called at the beginning of prediction.

Subclasses should override for any actions to run.

Parameters:: logs – Dict. Currently no data is passed to this argument for this method but that may change in the future.

on_predict_end(logs=None)

Called at the end of prediction.

Subclasses should override for any actions to run.

Parameters:: logs – Dict. Currently no data is passed to this argument for this method but that may change in the future.

on_test_batch_begin(batch, logs=None)

Called at the beginning of a batch in evaluate methods.

Also called at the beginning of a validation batch in the fit methods, if validation data is provided.

Subclasses should override for any actions to run.

Note that if the steps_per_execution argument to compile in tf.keras.Model is set to N, this method will only be called every N batches.

Parameters:

batch – Integer, index of batch within the current epoch.
logs – Dict. Currently no data is passed to this argument for this method but that may change in the future.

on_test_batch_end(batch, logs=None)

Called at the end of a batch in evaluate methods.

Also called at the end of a validation batch in the fit methods, if validation data is provided.

Subclasses should override for any actions to run.

Note that if the steps_per_execution argument to compile in tf.keras.Model is set to N, this method will only be called every N batches.

Parameters:

batch – Integer, index of batch within the current epoch.
logs – Dict. Aggregated metric results up until this batch.

on_test_begin(logs=None)

Called at the beginning of evaluation or validation.

Subclasses should override for any actions to run.

Parameters:: logs – Dict. Currently no data is passed to this argument for this method but that may change in the future.

on_test_end(logs=None)

Called at the end of evaluation or validation.

Subclasses should override for any actions to run.

Parameters:: logs – Dict. Currently the output of the last call to on_test_batch_end() is passed to this argument for this method but that may change in the future.

on_train_batch_begin(batch, logs=None)

Called at the beginning of a training batch in fit methods.

Subclasses should override for any actions to run.

Note that if the steps_per_execution argument to compile in tf.keras.Model is set to N, this method will only be called every N batches.

Parameters:

batch – Integer, index of batch within the current epoch.
logs – Dict. Currently no data is passed to this argument for this method but that may change in the future.

on_train_batch_end(batch, logs=None)

Called at the end of a training batch in fit methods.

Subclasses should override for any actions to run.

Note that if the steps_per_execution argument to compile in tf.keras.Model is set to N, this method will only be called every N batches.

Parameters:

batch – Integer, index of batch within the current epoch.
logs – Dict. Aggregated metric results up until this batch.

on_train_begin(logs={})[source]

Called at the beginning of training.

Subclasses should override for any actions to run.

Parameters:: logs – Dict. Currently no data is passed to this argument for this method but that may change in the future.

on_train_end(logs=None)

Called at the end of training.

Subclasses should override for any actions to run.

Parameters:: logs – Dict. Currently the output of the last call to on_epoch_end() is passed to this argument for this method but that may change in the future.