deepod.models.DevNetTS

class deepod.models.DevNetTS(epochs=100, batch_size=64, lr=0.001, network='Transformer', seq_len=100, stride=1, rep_dim=128, hidden_dims='100,50', act='ReLU', bias=False, n_heads=8, d_model=512, attn='self_attn', pos_encoding='fixed', norm='LayerNorm', margin=5.0, l=5000, epoch_steps=-1, prt_steps=10, device='cuda', verbose=2, random_state=42)

Deviation Networks for Weakly-supervised Anomaly Detection (KDD’19) [BPSvdH19]

Deviation Networks (DevNet) designed for weakly-supervised anomaly detection. This implementation is based on the architecture presented in the KDD’19 paper: “Deviation Networks for Weakly-supervised Anomaly Detection” by Pang et al.

Parameters:

• hidden_dims (Union[list, str, int], optional) – The dimensions for the hidden layers. Can be a list of integers, a string of comma-separated integers, or a single integer. - If list, each item is a layer - If str, neural units of hidden layers are split by comma - If int, number of neural units of single hidden layer - Defaults to ‘100,50’.

• act (str, optional) – Activation function to use. Choices include ‘ReLU’, ‘LeakyReLU’, ‘Sigmoid’, ‘Tanh’. Default is ‘ReLU’.

• bias (bool, optional) – Whether to include a bias term in the linear layers. Default is False.

• n_heads (int, optional) – Number of heads in multi-head attention. Only used when network is ‘transformer’. Default is 8.

• pos_encoding (str, optional) – The type of positional encoding to use. Only relevant when network is ‘transformer’. Choices are ‘fixed’ or ‘learnable’. Default is ‘fixed’.

• norm (str, optional) – Normalization method in the Transformer. Only relevant when network is ‘transformer’. Choices are ‘LayerNorm’ or ‘BatchNorm’. Default is ‘LayerNorm’.

• epochs (int, optional) – Number of training epochs. Default is 100.

• batch_size (int, optional) – Batch size for training. Default is 64.

• lr (float, optional) – Learning rate for the optimizer. Default is 1e-3.

• network (str, optional) – Type of network architecture to use. Default is ‘Transformer’.

• seq_len (int, optional) – Length of input sequences for models that require it. Default is 100.

• stride (int, optional) – Stride of the convolutional layers. Default is 1.

• rep_dim (int, optional) – The representation dimension. Unused in this model but kept for consistency. Default is 128.

• d_model (int, optional) – The number of expected features in the transformer model. Only used when network is ‘transformer’. Default is 512.

• attn (str, optional) – Type of attention to use. Only used when network is ‘transformer’. Default is ‘self_attn’.

• margin (float, optional) – Margin for the deviation loss function. Default is 5.

• l (int, optional) – The size of the sample for the Gaussian distribution in the deviation loss function. Default is 5000.

• epoch_steps (int, optional) – Maximum number of steps per epoch. If -1, all batches will be processed. Default is -1.

• prt_steps (int, optional) – Number of epoch intervals for printing during training. Default is 10.

• device (str, optional) – The device to use for training (‘cuda’ or ‘cpu’). Default is ‘cuda’.

• verbose (int, optional) – Verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch. Default is 2.

• random_state (int, optional) – Seed for the random number generator for reproducibility. Default is 42.

Methods

__init__([epochs, batch_size, lr, network, ...])	Initialize the DevNetTS.
decision_function(X[, return_rep])	Predict raw anomaly scores of X using the fitted detector.
decision_function_update(z, scores)	for any updating operation after decision function
epoch_update()	for any updating operation after each training epoch
fit(X[, y])	Fit detector.
fit_auto_hyper(X[, y, X_test, y_test, ...])	Fit detector.
inference_forward(batch_x, net, criterion)	Performs a forward pass during inference.
inference_prepare(X)	Prepares the data for inference.
load_model(path)
load_ray_checkpoint(best_config, best_checkpoint)
predict(X[, return_confidence])	Predict if a particular sample is an outlier or not.
save_model(path)
set_seed(seed)
set_tuned_net(config)
set_tuned_params()
training_forward(batch_x, net, criterion)	Performs a forward pass during training.
training_prepare(X, y)	Prepares the data and model for training by creating a balanced data loader, initializing the network, and setting up the loss criterion.

decision_function(X, return_rep=False)

Predict raw anomaly scores of X using the fitted detector.

The anomaly score of an input sample is computed based on the fitted detector. For consistency, outliers are assigned with higher anomaly scores.

Parameters:

• X (numpy array of shape (n_samples, n_features)) – The input samples. Sparse matrices are accepted only if they are supported by the base estimator.

• return_rep (boolean, optional, default=False) – whether return representations

Returns:

anomaly_scores – The anomaly score of the input samples.

Return type:

numpy array of shape (n_samples,)

decision_function_update(z, scores)

for any updating operation after decision function

epoch_update()

for any updating operation after each training epoch

fit(X, y=None)

Fit detector. y is ignored in unsupervised methods.

Parameters:

• X (numpy array of shape (n_samples, n_features)) – The input samples.

• y (numpy array of shape (n_samples, )) – Not used in unsupervised methods, present for API consistency by convention. used in (semi-/weakly-) supervised methods

Returns:

self – Fitted estimator.

Return type:

object

fit_auto_hyper(X, y=None, X_test=None, y_test=None, n_ray_samples=5, time_budget_s=None)

Fit detector. y is ignored in unsupervised methods.

Parameters:

• X (numpy array of shape (n_samples, n_features)) – The input samples.

• y (numpy array of shape (n_samples, )) – Not used in unsupervised methods, present for API consistency by convention. used in (semi-/weakly-) supervised methods

• X_test (numpy array of shape (n_samples, n_features), default=None) – The input testing samples for hyper-parameter tuning.

• y_test (numpy array of shape (n_samples, ), default=None) – Label of input testing samples for hyper-parameter tuning.

• n_ray_samples (int, default=5) – Number of times to sample from the hyperparameter space

• time_budget_s (int, default=None) – Global time budget in seconds after which all trials of Ray are stopped.

Returns:

config – tuned hyper-parameter

Return type:

dict

inference_forward(batch_x, net, criterion)

Performs a forward pass during inference.

Parameters:

• batch_x (Tensor) – A batch of input features.

• net (nn.Module) – The neural network model.

• criterion (Loss) – The loss function used during training. Not used.

Returns:

The batch of input features (unmodified).

s (Tensor):

The computed scores for the batch.

Return type:

batch_z (Tensor)

inference_prepare(X)

Prepares the data for inference.

Parameters:

X (Tensor) – The input features for inference.

Returns:

A DataLoader for inference.

Return type:

test_loader (DataLoader)

predict(X, return_confidence=False)

Predict if a particular sample is an outlier or not.

Parameters:

• X (numpy array of shape (n_samples, n_features)) – The input samples.

• return_confidence (boolean, optional(default=False)) – If True, also return the confidence of prediction.

Returns:

• outlier_labels (numpy array of shape (n_samples,)) – For each observation, tells whether it should be considered as an outlier according to the fitted model. 0 stands for inliers and 1 for outliers.

• confidence (numpy array of shape (n_samples,).) – Only if return_confidence is set to True.

training_forward(batch_x, net, criterion)

Performs a forward pass during training.

Parameters:

• batch_x (tuple) – A batch of input features and target labels.

• net (nn.Module) – The neural network model.

• criterion (Loss) – The loss function used during training.

Returns:

The computed loss for the batch.

Return type:

loss (Tensor)

training_prepare(X, y)

Prepares the data and model for training by creating a balanced data loader, initializing the network, and setting up the loss criterion.

Parameters:

• X (np.ndarray) – The input features for training.

• y (np.ndarray) – The target labels for training, where 1 indicates an anomaly.

Returns:

A DataLoader with balanced mini-batches for training.

net (nn.Module):

The initialized neural network model.

criterion (Loss):

The loss function used during training.

Return type:

train_loader (DataLoader)