pypots.anomaly_detection

pypots.anomaly_detection.saits

The package of the partially-observed time-series anomaly detection model SAITS.

Refer to the paper Wenjie Du, David Cote, and Yan Liu. SAITS: Self-Attention-based Imputation for Time Series. Expert Systems with Applications, 219:119619, 2023.

Notes

This implementation is inspired by the official one https://github.com/WenjieDu/SAITS

class pypots.anomaly_detection.saits.SAITS(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_k, d_v, d_ffn, dropout=0, attn_dropout=0, diagonal_attention_mask=True, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the SAITS model [1] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • n_layers (int) – The number of layers in the 1st and 2nd DMSA blocks in the SAITS model.

  • d_model (int) – The dimension of the model’s backbone. It is the input dimension of the multi-head DMSA layers.

  • n_heads (int) – The number of heads in the multi-head DMSA mechanism. d_model must be divisible by n_heads, and the result should be equal to d_k.

  • d_k (int) – The dimension of the keys (K) and the queries (Q) in the DMSA mechanism. d_k should be the result of d_model divided by n_heads. Although d_k can be directly calculated with given d_model and n_heads, we want it be explicitly given together with d_v by users to ensure users be aware of them and to avoid any potential mistakes.

  • d_v (int) – The dimension of the values (V) in the DMSA mechanism.

  • d_ffn (int) – The dimension of the layer in the Feed-Forward Networks (FFN).

  • dropout (float) – The dropout rate for all fully-connected layers in the model.

  • attn_dropout (float) – The dropout rate for DMSA.

  • diagonal_attention_mask (bool) – Whether to apply a diagonal attention mask to the self-attention mechanism. If so, the attention layers will use DMSA. Otherwise, the attention layers will use the original.

  • ORT_weight (int) – The weight for the ORT loss.

  • MIT_weight (int) – The weight for the MIT loss.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.timemixerpp

The package of the partially-observed time-series anomaly detection model TimeMixer++.

Refer to the paper Shiyu Wang, Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Ju Shengtong, Zhixuan Chu, Ming Jin. “TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis”. ICLR 2025.

Notes

This implementation is inspired by the official one https://anonymous.4open.science/r/TimeMixerPP

class pypots.anomaly_detection.timemixerpp.TimeMixerPP(n_steps, n_features, anomaly_rate, n_layers, d_model, d_ffn, top_k, n_heads, n_kernels, dropout=0, channel_mixing=True, channel_independence=True, downsampling_layers=3, downsampling_window=2, apply_nonstationary_norm=False, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the TimeMixer++ model [3] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • n_layers (int) – The number of layers in the TimeMixer++ model.

  • d_model (int) – The dimension of the model.

  • d_ffn (int) – The dimension of the feed-forward network.

  • top_k (int) – The number of top-k amplitude values to be selected to obtain the most significant frequencies.

  • n_heads (int) – The head number of full attention in the model. Only work if channel_mixing is True.

  • n_kernels (int) – num_kernels for Inception module.

  • dropout (float) – The dropout rate for the model.

  • channel_mixing (bool) – Whether to apply channel mixing in the model.

  • channel_independence (bool) – Whether to use channel independence in the model.

  • downsampling_layers (int) – The number of downsampling layers in the model.

  • downsampling_window (int) – The window size for downsampling.

  • apply_nonstationary_norm (bool) – Whether to apply non-stationary normalization to the input data for TimeMixer++. Please refer to [27] for details about non-stationary normalization, which is not the idea of the original TimeMixer++ paper. Hence, we make it optional and default not to use here.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.tefn

The implementation of TEFN for the partially-observed time-series anomaly detection task.

Refer to the paper Tianxiang Zhan, Yuanpeng He, Yong Deng, Zhen Li, Wenjie Du, and Qingsong Wen. Time Evidence Fusion Network: Multi-source View in Long-Term Time Series Forecasting. In Arxiv, 2024.

Notes

This implementation is transferred from the official one https://github.com/ztxtech/Time-Evidence-Fusion-Network

class pypots.anomaly_detection.tefn.TEFN(n_steps, n_features, anomaly_rate, n_fod=2, apply_nonstationary_norm=True, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the TEFN model [7] on the anomaly detection task. :type n_steps: int :param n_steps: The number of time steps in the time-series data sample. :type n_features: int :param n_features: The number of features in the time-series data sample. :type anomaly_rate: float :param anomaly_rate: The rate of anomalies in the data, should be in the range (0, 1). :type n_fod: int :param n_fod: The number of FOD (frame of discernment) in the TEFN model. :type apply_nonstationary_norm: bool :param apply_nonstationary_norm: Whether to apply non-stationary normalization to the input data for TimesNet.

Please refer to [27] for details about non-stationary normalization.

Parameters:

  • ORT_weight (int) – The weight for the ORT loss, the same as SAITS.

  • MIT_weight (int) – The weight for the MIT loss, the same as SAITS.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5')[source]

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.imputeformer

The package of the partially-observed time-series anomaly detection model ImputeFormer.

Refer to the papers Tong Nie, Guoyang Qin, Wei Ma, Yuewen Mei, Jian Sun. “ImputeFormer: Low Rankness-Induced Transformers for Generalizable Spatiotemporal Imputation” KDD 2024.

Notes

This implementation is inspired by the official one https://github.com/tongnie/ImputeFormer

class pypots.anomaly_detection.imputeformer.ImputeFormer(n_steps, n_features, anomaly_rate, n_layers, d_input_embed, d_learnable_embed, d_proj, d_ffn, n_temporal_heads, dropout=0.0, input_dim=1, output_dim=1, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the ImputeFormer model [12] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • n_layers (int) – The number of layers in the ImputeFormer model.

  • d_input_embed (int) – The dimension of the input embedding. It is the input dimension of the input embedding layer.

  • d_learnable_embed (int) – The dimension of the learnable node embedding. It is the dimension of the learnable node embedding (spatial positional embedding) used in spatial attention layers.

  • d_proj (int) – The dimension of the learnable projector. It is the dimension of the learnable projector used in temporal attention layers.

  • d_ffn (int) – The dimension of the layer in the Feed-Forward Networks (FFN).

  • dropout (float) – The dropout rate for all fully-connected layers in the model.

  • n_temporal_heads (int) – The number of attention heads in temporal attention layers.

  • input_dim (int) – The dimension of the input feature dimension, default is 1.

  • output_dim (int) – The dimension of the output feature dimension, default is 1.

  • ORT_weight (float) – The weight for the ORT loss.

  • MIT_weight (float) – The weight for the MIT loss.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.patchtst

The package of the partially-observed time-series anomaly detection model PatchTST.

Refer to the paper Wenjie Du, David Cote, and Yan Liu. PatchTST: Self-Attention-based Imputation for Time Series. Expert Systems with Applications, 219:119619, 2023.

Notes

This implementation is inspired by the official one https://github.com/WenjieDu/PatchTST

class pypots.anomaly_detection.patchtst.PatchTST(n_steps, n_features, anomaly_rate, patch_size, patch_stride, n_layers, d_model, n_heads, d_k, d_v, d_ffn, dropout, attn_dropout, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the PatchTST model [19] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • patch_size (int) – The patch length for patch embedding.

  • patch_stride (int) – The stride for patch embedding.

  • n_layers (int) – The number of layers in the PatchTST model.

  • d_model (int) – The dimension of the model.

  • n_heads (int) – The number of heads in each layer of PatchTST.

  • d_k (int) – The dimension of the keys (K) and the queries (Q) in the DMSA mechanism. d_k should be the result of d_model divided by n_heads. Although d_k can be directly calculated with given d_model and n_heads, we want it be explicitly given together with d_v by users to ensure users be aware of them and to avoid any potential mistakes.

  • d_v (int) – The dimension of the values (V) in the DMSA mechanism.

  • d_ffn (int) – The dimension of the feed-forward network.

  • dropout (float) – The dropout rate for the model.

  • ORT_weight (float) – The weight for the ORT loss, the same as SAITS.

  • MIT_weight (float) – The weight for the MIT loss, the same as SAITS.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.itransformer

The package of the partially-observed time-series anomaly detection model iTransformer.

Refer to the papers Liu, Yong, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. “iTransformer: Inverted transformers are effective for time series forecasting.” ICLR 2024.

Notes

This implementation is inspired by the official one https://github.com/thuml/iTransformer

class pypots.anomaly_detection.itransformer.iTransformer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_k, d_v, d_ffn, dropout=0, attn_dropout=0, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True, ORT_weight=1, MIT_weight=1)[source]

Bases: BaseNNDetector

The PyTorch implementation of the iTransformer model [10] for the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The estimated anomaly rate in the dataset, within the range (0, 1). Used for thresholding.

  • n_layers (int) – The number of layers in the iTransformer model.

  • d_model (int) – The dimensionality of the model input and output features.

  • n_heads (int) – The number of heads in the multi-head self-attention mechanism.

  • d_k (int) – The dimensionality of keys and queries in the self-attention mechanism.

  • d_v (int) – The dimensionality of values in the self-attention mechanism.

  • d_ffn (int) – The dimensionality of the feed-forward network within each block.

  • dropout (float, optional) – Dropout rate applied to all fully-connected layers. Default is 0.

  • attn_dropout (float, optional) – Dropout rate used in the attention layers. Default is 0.

  • batch_size (int, optional) – Number of samples per batch during training and evaluation.

  • epochs (int, optional) – Total number of training epochs.

  • patience (int, optional) – Number of epochs to wait for improvement before triggering early stopping. Disabled if None.

  • training_loss (Criterion or type, optional) – Loss function used during training. Defaults to MAE.

  • validation_metric (Criterion or type, optional) – Metric used during validation. Defaults to MSE.

  • optimizer (Optimizer or type, optional) – Optimizer used for training. Defaults to custom Adam optimizer.

  • num_workers (int, optional) – Number of subprocesses used for data loading.

  • device (str, torch.device, or list, optional) – Device(s) used for model training and inference. Supports multi-GPU training.

  • saving_path (str, optional) – Path to save model checkpoints and training logs. No saving if None.

  • model_saving_strategy (str or None, optional) – Strategy to save models: one of {None, “best”, “better”, “all”}.

  • verbose (bool, optional) – Whether to print training logs during execution.

  • ORT_weight (float, optional) – The weight for the ORT loss.

  • MIT_weight (float, optional) – The weight for the MIT loss.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Trains the model on the given dataset.

Parameters:

  • train_set (dict or str) – Training dataset or its file path.

  • val_set (dict or str, optional) – Validation dataset. Must contain ‘X_ori’.

  • file_type (str, optional) – File type if data is loaded from disk. Default is “hdf5”.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.crossformer

The package of the partially-observed time-series anoamly detection model Crossformer.

Refer to the paper Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The 11th ICLR, 2023.

Notes

This implementation is inspired by the official one https://github.com/Thinklab-SJTU/Crossformer

class pypots.anomaly_detection.crossformer.Crossformer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_ffn, factor, seg_len, win_size, dropout=0, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the Crossformer model [17] for the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The estimated anomaly rate in the dataset, within the range (0, 1). Used for thresholding.

  • n_layers (int) – The number of transformer layers in Crossformer.

  • d_model (int) – Dimensionality of the model input and output features.

  • n_heads (int) – Number of heads in the multi-head attention mechanism.

  • d_ffn (int) – Dimensionality of the feed-forward network.

  • factor (int) – Factor used in Cross-Dimension stage (number of routers in TSA block).

  • seg_len (int) – The length of the input segments.

  • win_size (int) – Window size for merging segment outputs.

  • dropout (float, optional) – Dropout rate used throughout the model. Default is 0.

  • ORT_weight (float, optional) – Weight for the ORT loss component. Default is 1.

  • MIT_weight (float, optional) – Weight for the MIT loss component. Default is 1.

  • batch_size (int, optional) – The number of samples per batch during training and evaluation.

  • epochs (int, optional) – Total number of training epochs.

  • patience (int, optional) – Number of epochs to wait for improvement before triggering early stopping. Disabled if None.

  • training_loss (Criterion or type, optional) – Loss function used during training. Defaults to MAE.

  • validation_metric (Criterion or type, optional) – Metric used during validation. Defaults to MSE.

  • optimizer (Optimizer or type, optional) – Optimizer used for training. Defaults to custom Adam optimizer.

  • num_workers (int, optional) – Number of subprocesses used for data loading.

  • device (str, torch.device, or list, optional) – Device(s) used for model training and inference. Supports multi-GPU training.

  • saving_path (str, optional) – Path to save model checkpoints and training logs. No saving if None.

  • model_saving_strategy (str or None, optional) – Strategy to save models: one of {None, “best”, “better”, “all”}.

  • verbose (bool, optional) – Whether to print training logs during execution.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Trains the model on the given dataset.

Parameters:

  • train_set (dict or str) – Training dataset or its file path.

  • val_set (dict or str, optional) – Validation dataset. Must contain ‘X_ori’.

  • file_type (str, optional) – File type if data is loaded from disk. Default is “hdf5”.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.pyraformer

The package of the partially-observed time-series anomaly detection model Pyraformer.

Refer to the paper Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar. “Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting”. International Conference on Learning Representations. 2022.

Notes

This implementation is inspired by the official one https://github.com/ant-research/Pyraformer

class pypots.anomaly_detection.pyraformer.Pyraformer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_ffn, window_size, inner_size, dropout=0, attn_dropout=0, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the Pyraformer model for the anomaly detection task. Pyraformer is originally proposed by Liu et al. in [30].

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The estimated anomaly rate in the dataset, within the range (0, 1). Used for thresholding.

  • n_layers (int) – The number of layers in the Pyraformer model.

  • d_model (int) – The dimension of the model.

  • n_heads (int) – The number of heads in each layer of Pyraformer.

  • d_ffn (int) – The dimension of the feed-forward network.

  • window_size (list) – The downsample window size in pyramidal attention.

  • inner_size (int) – The size of neighbor attention.

  • dropout (float, optional) – The dropout rate for the model. Default is 0.

  • attn_dropout (float, optional) – The dropout rate for the attention mechanism. Default is 0.

  • ORT_weight (float, optional) – The weight for the ORT loss. Default is 1.

  • MIT_weight (float, optional) – The weight for the MIT loss. Default is 1.

  • batch_size (int, optional) – The number of samples per batch during training and evaluation.

  • epochs (int, optional) – Total number of training epochs.

  • patience (int, optional) – Number of epochs to wait for improvement before triggering early stopping. Disabled if None.

  • training_loss (Criterion or type, optional) – Loss function used during training. Defaults to MAE.

  • validation_metric (Criterion or type, optional) – Metric used during validation. Defaults to MSE.

  • optimizer (Optimizer or type, optional) – Optimizer used for training. Defaults to custom Adam optimizer.

  • num_workers (int, optional) – Number of subprocesses used for data loading.

  • device (str, torch.device, or list, optional) – Device(s) used for model training and inference. Supports multi-GPU training.

  • saving_path (str, optional) – Path to save model checkpoints and training logs. No saving if None.

  • model_saving_strategy (str or None, optional) – Strategy to save models: one of {None, “best”, “better”, “all”}.

  • verbose (bool, optional) – Whether to print training logs during execution.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Trains the model on the given dataset.

Parameters:

  • train_set (dict or str) – Training dataset or its file path.

  • val_set (dict or str, optional) – Validation dataset. Must contain ‘X_ori’.

  • file_type (str, optional) – File type if data is loaded from disk. Default is “hdf5”.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.fedformer

The package of the partially-observed time-series anomaly detectoin model FEDformer.

Refer to the paper Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML, volume 162 of Proceedings of Machine Learning Research, pages 27268–27286. PMLR, 17–23 Jul 2022.

Notes

This implementation is inspired by the official one https://github.com/MAZiqing/FEDformer

class pypots.anomaly_detection.fedformer.FEDformer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_ffn, moving_avg_window_size, dropout=0, version='Fourier', modes=32, mode_select='random', ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the FEDformer model for the anomaly detection task.

FEDformer was originally proposed by Zhou et al. in [32].

Parameters:

  • n_steps (int) – Number of time steps in the time-series data sample.

  • n_features (int) – Number of features in the time-series data sample.

  • anomaly_rate (float) – Estimated anomaly rate in the dataset, within (0, 1). Used for thresholding.

  • n_layers (int) – Number of layers in the FEDformer.

  • n_heads (int) – Number of heads in the multi-head attention mechanism.

  • d_model (int) – Dimension of model input and output features.

  • d_ffn (int) – Dimension of feed-forward network.

  • moving_avg_window_size (int) – Window size of moving average module.

  • dropout (float, optional) – Dropout rate. Default is 0.

  • version (str, optional) – FEDformer version: “Wavelets” or “Fourier”. Default is “Fourier”.

  • modes (int, optional) – Number of frequency modes. Default is 32.

  • mode_select (str, optional) – Mode selection strategy: “random” or “low”. Default is “random”.

  • ORT_weight (float, optional) – Weight for ORT loss term.

  • MIT_weight (float, optional) – Weight for MIT loss term.

  • batch_size (int, optional) – Training batch size. Default is 32.

  • epochs (int, optional) – Total training epochs. Default is 100.

  • patience (int, optional) – Patience for early stopping. Disabled if None.

  • training_loss (Criterion or type, optional) – Loss function during training. Default is MAE.

  • validation_metric (Criterion or type, optional) – Metric function during validation. Default is MSE.

  • optimizer (Optimizer or type, optional) – Optimizer. Default is custom Adam optimizer.

  • num_workers (int, optional) – Number of subprocesses for data loading.

  • device (str, torch.device, or list, optional) – Device(s) for training and inference.

  • saving_path (str, optional) – Path to save model checkpoints and logs.

  • model_saving_strategy (str or None, optional) – Checkpoint saving strategy: {None, “best”, “better”, “all”}.

  • verbose (bool, optional) – Whether to print training logs.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Trains the model on the given dataset.

Parameters:

  • train_set (dict or str) – Training dataset or its file path.

  • val_set (dict or str, optional) – Validation dataset. Must contain ‘X_ori’.

  • file_type (str, optional) – File type if data is loaded from disk. Default is “hdf5”.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.informer

The package of the partially-observed time-series anomaly detection model Informer.

Refer to the paper Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021.

Notes

This implementation is inspired by the official one https://github.com/zhouhaoyi/Informer2020

class pypots.anomaly_detection.informer.Informer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_ffn, factor, dropout=0, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the Informer model :cite:zhou2021informer for the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The estimated anomaly rate in the dataset, within the range (0, 1). Used for thresholding.

  • n_layers (int) – The number of layers in the Informer model.

  • d_model (int) – The dimensionality of input and output feature vectors.

  • n_heads (int) – The number of attention heads in each Transformer block.

  • d_ffn (int) – The dimensionality of the feed-forward network hidden layer.

  • factor (int) – The factor for controlling sparsity in the ProbSparse attention mechanism.

  • dropout (float, optional) – Dropout rate applied throughout the model. Default is 0.

  • ORT_weight (float, optional) – The weight for the ORT loss term during training. Default is 1.

  • MIT_weight (float, optional) – The weight for the MIT loss term during training. Default is 1.

  • batch_size (int, optional) – The number of samples per batch during training and evaluation. Default is 32.

  • epochs (int, optional) – The total number of training epochs. Default is 100.

  • patience (int, optional) – Number of epochs to wait for early stopping if no improvement. If None, early stopping is disabled.

  • training_loss (Criterion or type, optional) – Loss function used during training. Defaults to MAE.

  • validation_metric (Criterion or type, optional) – Metric function used for validation during training. Defaults to MSE.

  • optimizer (Optimizer or type, optional) – The optimizer instance or optimizer class used for model training. Defaults to custom Adam optimizer.

  • num_workers (int, optional) – The number of subprocesses to use for data loading. Default is 0.

  • device (str, torch.device, or list, optional) – The device(s) on which the model will be trained or evaluated. Supports CPU, CUDA, and multi-GPU.

  • saving_path (str, optional) – The path where model checkpoints and training logs are saved. No saving if None.

  • model_saving_strategy (str or None, optional) – Strategy to save models during training: one of {None, “best”, “better”, “all”}.

  • verbose (bool, optional) – Whether to print detailed logs during the training process. Default is True.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the Informer model on the provided datasets.

Parameters:

  • train_set (dict or str) – The training dataset or file path.

  • val_set (dict or str, optional) – The validation dataset or file path. Must contain ‘X_ori’ if provided.

  • file_type (str, optional) – File type if data is loaded from disk. Default is “hdf5”.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.transformer

The package of the partially-observed time-series anomaly detection model Transformer.

Refer to the papers Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. and Wenjie Du, David Cote, and Yan Liu. SAITS: Self-Attention-based Imputation for Time Series. Expert Systems with Applications, 219:119619, 2023.

Notes

This implementation is inspired by https://github.com/WenjieDu/SAITS

class pypots.anomaly_detection.transformer.Transformer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_k, d_v, d_ffn, dropout=0, attn_dropout=0, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the Transformer model for the anomaly detection task.

Transformer is originally proposed by Vaswani et al. in [47], and gets re-implemented for partially-observed time-series modeling by Du et al. in [1]. Here we adapt it specifically for anomaly detection tasks.

Parameters:

  • n_steps (int) – The number of time steps in each input time-series sample.

  • n_features (int) – The number of features (dimensions) in each input time-series sample.

  • anomaly_rate (float) – The expected anomaly rate within the dataset, between (0, 1). Used to determine detection thresholds.

  • n_layers (int) – The number of stacked Transformer encoder layers.

  • d_model (int) – The dimensionality of inputs and outputs inside the model’s backbone. It is also the input dimension to the multi-head self-attention blocks.

  • n_heads (int) – The number of parallel heads used in multi-head self-attention mechanisms.

  • d_k (int) – The dimensionality of key and query vectors in the attention mechanism. Must satisfy d_model = n_heads * d_k.

  • d_v (int) – The dimensionality of value vectors in the attention mechanism.

  • d_ffn (int) – The dimensionality of the hidden layer inside the position-wise Feed-Forward Network (FFN).

  • dropout (float, optional) – Dropout probability applied across fully connected layers. Default is 0.

  • attn_dropout (float, optional) – Dropout probability applied inside attention mechanisms. Default is 0.

  • ORT_weight (int, optional) – Weight coefficient for the ORT (Observation Reconstruction Task) loss component.

  • MIT_weight (int, optional) – Weight coefficient for the MIT (Missingness Imputation Task) loss component.

  • batch_size (int, optional) – Number of samples in each training batch. Default is 32.

  • epochs (int, optional) – Maximum number of epochs to train the model. Default is 100.

  • patience (int, optional) – Number of epochs to wait without improvement before early stopping is triggered. If None, early stopping is disabled.

  • training_loss (Criterion or type, optional) – Loss function used for training. If not specified, defaults to Mean Absolute Error (MAE).

  • validation_metric (Criterion or type, optional) – Metric used to evaluate model performance on validation set. Defaults to Mean Squared Error (MSE).

  • optimizer (Optimizer or type, optional) – Optimizer used for training the model. Defaults to a custom implementation of Adam.

  • num_workers (int, optional) – Number of worker subprocesses to use for data loading. 0 means no subprocesses (i.e., main process only).

  • device (str, torch.device, or list, optional) – Device(s) on which the model runs, e.g., ‘cuda:0’, ‘cpu’, or list of CUDA devices for multi-GPU training. If None, the model automatically selects GPU if available, otherwise CPU.

  • saving_path (str, optional) – Directory path for saving trained model checkpoints and TensorBoard logs. No saving if None.

  • model_saving_strategy (str or None, optional) – Strategy for saving model checkpoints: - None: Do not save any model. - “best”: Save only the best-performing model. - “better”: Save model when validation performance improves. - “all”: Save model at every epoch.

  • verbose (bool, optional) – Whether to print detailed training logs during model training. Default is True.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the Transformer model for anomaly detection.

Parameters:

  • train_set (dict or str) – Training dataset or path to it.

  • val_set (dict or str, optional) – Validation dataset or path to it. Must include ‘X_ori’.

  • file_type (str, optional) – File type if loading from disk. Default is “hdf5”.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.efsformer

pypots.anomaly_detection.timemixer

The package of the partially-observed time-series anomaly detection model TimeMixer.

Refer to the paper Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, and Jun Zhou. “TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting”. In ICLR 2024.

Notes

This implementation is inspired by the official one https://github.com/kwuking/TimeMixer

class pypots.anomaly_detection.timemixer.TimeMixer(n_steps, n_features, anomaly_rate, n_layers, d_model, d_ffn, top_k, dropout=0, channel_independence=False, decomp_method='moving_avg', moving_avg=5, downsampling_layers=3, downsampling_window=2, apply_nonstationary_norm=False, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the TimeMixer model for the anomaly detection task. TimeMixer is originally proposed by Wang et al. in [9].

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The estimated anomaly rate in the dataset, within the range (0, 1). Used for thresholding.

  • n_layers (int) – The number of layers in the TimeMixer model.

  • d_model (int) – The dimension of the model.

  • d_ffn (int) – The dimension of the feed-forward network.

  • top_k (int) – The number of top-k frequency amplitudes selected in frequency domain operations.

  • dropout (float, optional) – The dropout rate for the model. Default is 0.

  • channel_independence (bool, optional) – Whether to use channel independence in the model.

  • decomp_method (str, optional) – The decomposition method for the model. Must be one of [‘moving_avg’, ‘dft_decomp’].

  • moving_avg (int, optional) – The window size for moving average decomposition.

  • downsampling_layers (int, optional) – The number of downsampling layers in the model.

  • downsampling_window (int, optional) – The window size for downsampling.

  • apply_nonstationary_norm (bool, optional) – Whether to apply non-stationary normalization as described in [27].

  • batch_size (int, optional) – The number of samples per batch during training and evaluation.

  • epochs (int, optional) – Total number of training epochs.

  • patience (int, optional) – Number of epochs to wait for improvement before triggering early stopping. Disabled if None.

  • training_loss (Criterion or type, optional) – Loss function used during training. Defaults to MAE.

  • validation_metric (Criterion or type, optional) – Metric used during validation. Defaults to MSE.

  • optimizer (Optimizer or type, optional) – Optimizer used for training. Defaults to custom Adam optimizer.

  • num_workers (int, optional) – Number of subprocesses used for data loading.

  • device (str, torch.device, or list, optional) – Device(s) used for model training and inference. Supports multi-GPU training.

  • saving_path (str, optional) – Path to save model checkpoints and training logs. No saving if None.

  • model_saving_strategy (str or None, optional) – Strategy to save models: one of {None, “best”, “better”, “all”}.

  • verbose (bool, optional) – Whether to print training logs during execution.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Trains the model on the given dataset.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.nonstationary_transformer

The package of the partially-observed time-series anomaly detection model Nonstationary-Transformer.

Refer to the paper Yong Liu, Haixu Wu, Jianmin Wang, Mingsheng Long. Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting. Advances in Neural Information Processing Systems 35 (2022): 9881-9893.

Notes

This implementation is inspired by the official one https://github.com/thuml/Nonstationary_Transformers

class pypots.anomaly_detection.nonstationary_transformer.NonstationaryTransformer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_ffn, d_projector_hidden, n_projector_hidden_layers, dropout=0, attn_dropout=0, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the Nonstationary-Transformer model for the anomaly detection task. Originally proposed by Liu et al. in [27].

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The estimated anomaly rate in the dataset, within (0, 1). Used for thresholding.

  • n_layers (int) – The number of layers in the NonstationaryTransformer model.

  • d_model (int) – The dimension of the model.

  • n_heads (int) – The number of attention heads.

  • d_ffn (int) – The dimension of the feed-forward network.

  • d_projector_hidden (list) – Dimensions of hidden layers in MLP projectors.

  • n_projector_hidden_layers (int) – Number of hidden layers in MLP projectors.

  • dropout (float, optional) – Dropout rate for the model.

  • attn_dropout (float, optional) – Dropout rate in the attention mechanism.

  • ORT_weight (float, optional) – Weight for ORT loss.

  • MIT_weight (float, optional) – Weight for MIT loss.

  • batch_size (int, optional) – Batch size for training and evaluation.

  • epochs (int, optional) – Total number of training epochs.

  • patience (int, optional) – Early stopping patience. Disabled if None.

  • training_loss (Criterion or type, optional) – Loss function for training. Defaults to MAE.

  • validation_metric (Criterion or type, optional) – Metric for validation. Defaults to MSE.

  • optimizer (Optimizer or type, optional) – Optimizer class or instance.

  • num_workers (int, optional) – Number of subprocesses for data loading.

  • device (str, torch.device, or list, optional) – Device(s) to run the model.

  • saving_path (str, optional) – Path to save model checkpoints and logs.

  • model_saving_strategy (str or None, optional) – Saving strategy: None, “best”, “better”, or “all”.

  • verbose (bool, optional) – Whether to print training logs.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Trains the model on the given dataset.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.film

The package of the partially-observed time-series anomaly detection model FiLM.

Refer to the paper Zhou, Tian, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, and Rong Jin. “FiLM: Frequency improved legendre memory model for long-term time series forecasting.” In Advances in Neural Information Processing Systems 35 (2022): 12677-12690.

Notes

This implementation is inspired by the official one https://github.com/tianzhou2011/FiLM

class pypots.anomaly_detection.film.FiLM(n_steps, n_features, anomaly_rate, window_size, multiscale, modes1=32, dropout=0.5, mode_type=0, d_model=128, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the FiLM model for the anomaly detection task. FiLM is originally proposed by Zhou et al. in [28].

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The estimated anomaly rate in the dataset, within the range (0, 1). Used for thresholding.

  • window_size (list) – A list including the window sizes for the HiPPO projection layers.

  • multiscale (list) – A list including the multiscale factors for the HiPPO projection layers.

  • modes1 (int, optional) – The number of Fourier modes used. Default is 32.

  • dropout (float, optional) – The dropout ratio for the HiPPO projection layers. Default is 0.5.

  • mode_type (int, optional) – The mode type of the SpectralConv1d layers. Must be one of {0, 1, 2}.

  • d_model (int, optional) – The dimension of the model. Default is 128.

  • ORT_weight (float, optional) – The weight for the ORT loss. Default is 1.

  • MIT_weight (float, optional) – The weight for the MIT loss. Default is 1.

  • batch_size (int, optional) – The number of samples per batch during training and evaluation.

  • epochs (int, optional) – Total number of training epochs.

  • patience (int, optional) – Number of epochs to wait for improvement before triggering early stopping. Disabled if None.

  • training_loss (Criterion or type, optional) – Loss function used during training. Defaults to MAE.

  • validation_metric (Criterion or type, optional) – Metric used during validation. Defaults to MSE.

  • optimizer (Optimizer or type, optional) – Optimizer used for training. Defaults to custom Adam optimizer.

  • num_workers (int, optional) – Number of subprocesses used for data loading.

  • device (str, torch.device, or list, optional) – Device(s) used for model training and inference.

  • saving_path (str, optional) – Path to save model checkpoints and training logs.

  • model_saving_strategy (str or None, optional) – Strategy to save models: one of {None, “best”, “better”, “all”}.

  • verbose (bool, optional) – Whether to print training logs during execution.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Trains the FiLM model on the given dataset for anomaly detection.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.timesnet

The package of the partially-observed time-series anomaly detection model TimesNet.

Refer to the paper Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In ICLR, 2023.

Notes

This implementation is inspired by the official one https://github.com/thuml/TimesNet

class pypots.anomaly_detection.timesnet.TimesNet(n_steps, n_features, anomaly_rate, n_layers, top_k, d_model, d_ffn, n_kernels, dropout=0, apply_nonstationary_norm=False, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the TimesNet model [18] for the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The estimated anomaly rate in the dataset, within the range (0, 1). Used for thresholding.

  • n_layers (int) – The number of layers in the TimesNet model.

  • top_k (int) – The number of top-k frequency amplitudes selected in frequency domain operations.

  • d_model (int) – The dimensionality of the model input and output features.

  • d_ffn (int) – The dimensionality of the feed-forward network within each block.

  • n_kernels (int) – The number of 2D convolution kernels in the Inception block.

  • dropout (float, optional) – Dropout rate used throughout the model. Default is 0.

  • apply_nonstationary_norm (bool, optional) – Whether to apply non-stationary normalization as described in [27].

  • batch_size (int, optional) – The number of samples per batch during training and evaluation.

  • epochs (int, optional) – Total number of training epochs.

  • patience (int, optional) – Number of epochs to wait for improvement before triggering early stopping. Disabled if None.

  • training_loss (Criterion or type, optional) – Loss function used during training. Defaults to MAE.

  • validation_metric (Criterion or type, optional) – Metric used during validation. Defaults to MSE.

  • optimizer (Optimizer or type, optional) – Optimizer used for training. Defaults to custom Adam optimizer.

  • num_workers (int, optional) – Number of subprocesses used for data loading.

  • device (str, torch.device, or list, optional) – Device(s) used for model training and inference. Supports multi-GPU training.

  • saving_path (str, optional) – Path to save model checkpoints and training logs. No saving if None.

  • model_saving_strategy (str or None, optional) – Strategy to save models: one of {None, “best”, “better”, “all”}.

  • verbose (bool, optional) – Whether to print training logs during execution.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Trains the model on the given dataset.

Parameters:

  • train_set (dict or str) – Training dataset or its file path.

  • val_set (dict or str, optional) – Validation dataset. Must contain ‘X_ori’.

  • file_type (str, optional) – File type if data is loaded from disk. Default is “hdf5”.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.dlinear

The package of the partially-observed time-series anomaly detection model DLinear.

Refer to the paper Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI, volume 37, pages 11121–11128, Jun. 2023.

Notes

This implementation is inspired by the official one https://github.com/cure-lab/LTSF-Linear

class pypots.anomaly_detection.dlinear.DLinear(n_steps, n_features, anomaly_rate, moving_avg_window_size, individual=False, d_model=None, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the DLinear model [22] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • moving_avg_window_size (int) – The window size of moving average.

  • individual (bool) – Whether to make a linear layer for each variate/channel/feature individually.

  • d_model (Optional[int]) – The dimension of the space in which the time-series data will be embedded and modeled. It is necessary only for DLinear in the non-individual mode.

  • ORT_weight (float) – The weight for the ORT loss, the same as SAITS.

  • MIT_weight (float) – The weight for the MIT loss, the same as SAITS.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.segrnn

The package of the partially-observed time-series anomaly detection model SegRNN.

Refer to the paper Wenjie Du, David Cote, and Yan Liu. SegRNN: Self-Attention-based Imputation for Time Series. Expert Systems with Applications, 219:119619, 2023.

Notes

This implementation is inspired by the official one https://github.com/WenjieDu/SegRNN

class pypots.anomaly_detection.segrnn.SegRNN(n_steps, n_features, anomaly_rate, seg_len=24, d_model=512, dropout=0.5, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the SegRNN model [25] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • seg_len (int) – The segment length for input of RNN.

  • d_model (int) – The dimension of RNN cell.

  • dropout (float) – The dropout rate of the output layer of SegRNN.

  • ORT_weight (float) – The weight for the ORT loss, the same as SAITS.

  • MIT_weight (float) – The weight for the MIT loss, the same as SAITS.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.scinet

The package of the partially-observed time-series anomaly detection model SCINet.

Refer to the paper Minhao LIU, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia LAI, Lingna Ma, and Qiang Xu. “SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction”. In Advances in Neural Information Processing Systems, 2022.

Notes

This implementation is inspired by the official one https://github.com/cure-lab/SCINet

class pypots.anomaly_detection.scinet.SCINet(n_steps, n_features, anomaly_rate, n_stacks, n_levels, n_groups, n_decoder_layers, d_hidden, kernel_size=3, concat_len=0, dropout=0.5, pos_enc=False, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the SCINet model [26] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • n_stacks (int) – The number of stacks in the model.

  • n_levels (int) – The number of levels in the model.

  • n_groups (int) – The number of groups in the model.

  • n_decoder_layers (int) – The number of decoder layers in the model.

  • d_hidden (int) – The hidden dimension for the model.

  • kernel_size (int) – The kernel size for the model.

  • dropout (float) – The dropout rate for the model.

  • concat_len (int) – The length for concatenation.

  • pos_enc (bool) – Whether to use positional encoding in the model.

  • ORT_weight (float) – The weight for the ORT loss, the same as SAITS.

  • MIT_weight (float) – The weight for the MIT loss, the same as SAITS.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.autoformer

The package of the partially-observed time-series anomaly detection model Autoformer.

Refer to the paper Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with autocorrelation for long-term series forecasting. In Advances in Neural Information Processing Systems, volume 34, pages 22419–22430. Curran Associates, Inc., 2021.

Notes

This implementation is inspired by the official one https://github.com/thuml/Autoformer

class pypots.anomaly_detection.autoformer.Autoformer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, d_ffn, factor, moving_avg_window_size, dropout=0, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the Autoformer model [33] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • n_layers (int) – The number of layers in the Autoformer model.

  • d_model (int) – The dimension of the model.

  • n_heads (int) – The number of heads in each layer of Autoformer.

  • d_ffn (int) – The dimension of the feed-forward network.

  • factor (int) – The factor of the auto correlation mechanism for the Autoformer model.

  • moving_avg_window_size (int) – The window size of moving average.

  • dropout (float) – The dropout rate for the model.

  • ORT_weight (float) – The weight for the ORT loss, the same as SAITS.

  • MIT_weight (float) – The weight for the MIT loss, the same as SAITS.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None

pypots.anomaly_detection.reformer

The package of the partially-observed time-series anomaly detection model Reformer.

Refer to the paper Kitaev, Nikita, Łukasz Kaiser, and Anselm Levskaya. Reformer: The Efficient Transformer. International Conference on Learning Representations, 2020.

Notes

This implementation is inspired by the official one https://github.com/google/trax/tree/master/trax/models/reformer and https://github.com/lucidrains/reformer-pytorch

class pypots.anomaly_detection.reformer.Reformer(n_steps, n_features, anomaly_rate, n_layers, d_model, n_heads, bucket_size, n_hashes, causal, d_ffn, dropout=0, ORT_weight=1, MIT_weight=1, batch_size=32, epochs=100, patience=None, training_loss=<class 'pypots.nn.modules.loss.MAE'>, validation_metric=<class 'pypots.nn.modules.loss.MSE'>, optimizer=<class 'pypots.optim.adam.Adam'>, num_workers=0, device=None, saving_path=None, model_saving_strategy='best', verbose=True)[source]

Bases: BaseNNDetector

The PyTorch implementation of the Reformer model [40] on the anomaly detection task.

Parameters:

  • n_steps (int) – The number of time steps in the time-series data sample.

  • n_features (int) – The number of features in the time-series data sample.

  • anomaly_rate (float) – The rate of anomalies in the data, should be in the range (0, 1).

  • n_layers (int) – The number of layers in the Reformer model.

  • d_model (int) – The dimension of the model.

  • n_heads (int) – The number of heads in each layer of Reformer.

  • bucket_size (int) – Average size of qk per bucket, 64 was recommended in paper.

  • n_hashes (int) – 4 is permissible per author, 8 is the best but slower.

  • causal (bool) – Auto-regressive or not.

  • d_ffn (int) – The dimension of the feed-forward network. The window size of moving average.

  • dropout (float) – The dropout rate for the model.

  • ORT_weight (float) – The weight for the ORT loss, the same as SAITS.

  • MIT_weight (float) – The weight for the MIT loss, the same as SAITS.

  • batch_size (int) – The batch size for training and evaluating the model.

  • epochs (int) – The number of epochs for training the model.

  • patience (Optional[int]) – The patience for the early-stopping mechanism. Given a positive integer, the training process will be stopped when the model does not perform better after that number of epochs. Leaving it default as None will disable the early-stopping.

  • training_loss (Union[Criterion, type]) – The customized loss function designed by users for training the model. If not given, will use the default loss as claimed in the original paper.

  • validation_metric (Union[Criterion, type]) – The customized metric function designed by users for validating the model. If not given, will use the default MSE metric.

  • optimizer (Union[Optimizer, type]) – The optimizer for model training. If not given, will use a default Adam optimizer.

  • num_workers (int) – The number of subprocesses to use for data loading. 0 means data loading will be in the main process, i.e. there won’t be subprocesses.

  • device (Union[str, device, list, None]) – The device for the model to run on. It can be a string, a torch.device object, or a list of them. If not given, will try to use CUDA devices first (will use the default CUDA device if there are multiple), then CPUs, considering CUDA and CPU are so far the main devices for people to train ML models. If given a list of devices, e.g. [‘cuda:0’, ‘cuda:1’], or [torch.device(‘cuda:0’), torch.device(‘cuda:1’)] , the model will be parallely trained on the multiple devices (so far only support parallel training on CUDA devices). Other devices like Google TPU and Apple Silicon accelerator MPS may be added in the future.

  • saving_path (Optional[str]) – The path for automatically saving model checkpoints and tensorboard files (i.e. loss values recorded during training into a tensorboard file). Will not save if not given.

  • model_saving_strategy (Optional[str]) – The strategy to save model checkpoints. It has to be one of [None, “best”, “better”, “all”]. No model will be saved when it is set as None. The “best” strategy will only automatically save the best model after the training finished. The “better” strategy will automatically save the model during training whenever the model performs better than in previous epochs. The “all” strategy will save every model after each epoch training.

  • verbose (bool) – Whether to print out the training logs during the training process.

detect(test_set, file_type='hdf5', **kwargs)

Detect anomalies in the given data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The data samples for testing, should be array-like with shape [n_samples, n_steps, n_features], or a path string locating a data file, e.g. h5 file.

  • file_type (str) – The type of the given file if X is a path string.

Returns:

Anomaly detection results of the given samples.

Return type:

results

fit(train_set, val_set=None, file_type='hdf5')[source]

Train the detector on the given data.

Parameters:

  • train_set (Union[dict, str]) – The dataset for model training, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for training, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • val_set (Union[dict, str, None]) – The dataset for model validating, should be a dictionary including the key ‘X’, or a path string locating a data file. If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is time-series data for validating, can contain missing values. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include the key ‘X’.

  • file_type (str) – The type of the given file if train_set and val_set are path strings.

Return type:

None

load(path)

Load the saved model from a disk file.

Parameters:

path (str) – The local path to a disk file saving the trained model.

Return type:

None

Notes

If the training environment and the deploying/test environment use the same type of device (GPU/CPU), you can load the model directly with torch.load(model_path).

predict(test_set, file_type='hdf5', **kwargs)

Make predictions for the input data with the trained model.

Parameters:

  • test_set (Union[dict, str]) – The test dataset for model to process, should be a dictionary including keys as ‘X’, or a path string locating a data file supported by PyPOTS (e.g. h5 file). If it is a dict, X should be array-like with shape [n_samples, n_steps, n_features], which is the time-series data for processing. If it is a path string, the path should point to a data file, e.g. a h5 file, which contains key-value pairs like a dict, and it has to include ‘X’ key.

  • file_type (str) – The type of the given file if test_set is a path string.

Returns:

The dictionary containing the anomaly detection results as key ‘anomaly_detection’ and latent variables if necessary.

Return type:

result_dict

save(saving_path, overwrite=False)

Save the model with current parameters to a disk file.

A .pypots extension will be appended to the filename if it does not already have one. Please note that such an extension is not necessary, but to indicate the saved model is from PyPOTS framework so people can distinguish.

Parameters:

  • saving_path (str) – The given path to save the model. The directory will be created if it does not exist.

  • overwrite (bool) – Whether to overwrite the model file if the path already exists.

Return type:

None

to(device)

Move the model to the given device.

Parameters:

device (Union[str, device]) – The device to move the model to. It can be a string or a torch.device object.

Return type:

None