Thresholding Module#

Turning a continuous error signal into alarms, without labels.

A detector emits a score per timestep; operations need a boolean. The selectors here choose that cutoff from the score distribution alone, so no anomaly labels are required at deployment time. threshold_for_budget caps the fraction of points flagged, which is the control operations can actually state; dynamic_threshold needs no budget chosen in advance.

For which selector to use and how the defaults here differ from the published telemanom method, see Scoring and Thresholding.

Label-free thresholding of an anomaly error signal.

Every threshold this toolkit produced previously came from one of two places: a percentile of the training scores, or a sweep chosen against the test labels. The first ignores the shape of the error distribution; the second is an oracle and cannot be deployed. This module implements the nonparametric dynamic thresholding and pruning of Hundman et al. (2018), which select an operating point from the error signal alone.

The method has two stages.

Threshold selection. Candidate thresholds are drawn from eps = mu + z * sigma over a range of z. The chosen candidate maximises

\[\frac{\Delta\mu/\mu + \Delta\sigma/\sigma}{|e_a| + |E_{seq}|^2}\]

where e_a are the errors above the candidate, E_seq the contiguous runs they form, and delta_mu, delta_sigma the reductions in the mean and standard deviation of the error signal once those errors are removed.

The numerator asks how much calmer the signal becomes when the flagged points are taken out: a large reduction means they really were outliers rather than part of the bulk. The denominator prices that reduction. |e_a| penalises flagging many points, and |E_seq|**2 penalises fragmentation quadratically, so a few coherent events are strongly preferred over the same number of points scattered across the series. The ratio is therefore a signal-to-cost trade resolved without reference to any label.

Pruning. Candidate sequences are ranked by their peak error and a sequence is kept only while consecutive peaks fall off gradually. A large relative drop marks the boundary between genuine anomalies and the nominal tail, and everything below the last such drop is reclassified as nominal. The nominal maximum is appended to the ranking so that a single candidate still has something to be compared against.

All functions here operate on a plain error array, so the caller decides which errors to pass: training errors for a fixed operating point to deploy, or a trailing window for an adaptive one.

Choosing between the two selectors: threshold_for_budget() is exactly controllable and suits a deployed trigger, where operations can state an alarm rate but cannot state a recall they have no way to observe. dynamic_threshold() needs no budget chosen in advance.

Two defaults depart from the published method, both because the error signals differ: the candidate range starts lower (see dynamic_threshold()), and pruning is off. One interaction is worth knowing before deploying a budget: at small budgets the flagged points are scattered singletons that never form a run of two, so filter_sequences() with its default minimum length discards all of them.

Measured results on SMAP, and how this module compares with the published implementation, are recorded in the anomaly scoring page of the documentation rather than here, so that they can be revised as the detectors change.

telemetry_anomdet.thresholding.anomalous_sequences(errors: Sequence[float], threshold: float) → list[tuple[int, int]][source]#

Contiguous runs of errors strictly above a threshold.

Parameters:

  • errors – Error signal, one value per timestep.

  • threshold – Cutoff; a value must exceed it to be included.

Returns:

(start, end) index pairs, inclusive of both ends.

Return type:

list

telemetry_anomdet.thresholding.dynamic_threshold(errors: Sequence[float], strategy: str = 'sigma', n_candidates: int = 120, z_range: tuple[float, float] = (1.0, 12.0), q_range: tuple[float, float] = (0.8, 0.9999)) → dict[source]#

Choose a threshold from the error signal alone, with no labels.

Each candidate is scored by the objective described in the module docstring: the reduction it produces in the mean and standard deviation of the error signal, divided by the number of points and the square of the number of sequences it flags.

Two candidate sets are available.

"sigma" (default)

mu + z * sigma for z spanning z_range, as published.

"quantile"

Evenly spaced upper quantiles of the errors, spanning q_range.

Which set is better depends on the signal, and the measured difference is small, so the published choice is the default.

The lower end of z_range matters more, and the default of 1.0 departs from the published 2.5 deliberately. Deviation scores here are already normalised by each node’s training median and IQR, so their distribution is much tighter than the smoothed prediction errors the original method was tuned on, and a floor of 2.5 sits past the region where useful thresholds lie. The choice was validated on held-out channels; see the anomaly scoring page of the documentation.

Parameters:

  • errors – Error signal, one value per timestep.

  • strategy – "quantile" or "sigma".

  • n_candidates – How many candidates to evaluate.

  • z_range – Range of z for the sigma strategy.

  • q_range – Range of quantiles for the quantile strategy.

Returns:

threshold, the objective score, the n_above and n_sequences it flags, and the strategy used. When no candidate flags anything, threshold is the maximum error so that nothing exceeds it and score is 0.0.

Return type:

dict

telemetry_anomdet.thresholding.threshold_for_budget(errors: Sequence[float], budget: float = 0.01) → dict[source]#

Threshold that flags approximately budget of the signal.

The alarm budget is the operationally meaningful control. Operations can say how often a trigger may fire; they cannot say what recall they want, because recall is unobservable without labels. The budget needs no search: it is a quantile of the errors.

When anomalies are a small part of the signal, the flagged fraction tracks the false alarm rate closely, so choosing a budget sets the false alarm rate directly and without labels.

The fraction achieved is approximate rather than bounded, and can exceed the budget slightly. Read flagged in the result for the fraction actually reached rather than assuming the budget was met exactly.

Parameters:

  • errors – Error signal, one value per timestep.

  • budget – Target fraction of points to flag, in (0, 1).

Returns:

threshold, the flagged fraction actually achieved, and the n_above and n_sequences it produces.

Return type:

dict

telemetry_anomdet.thresholding.filter_sequences(sequences: list[tuple[int, int]], min_length: int = 2, ignore_before: int = 0) → list[tuple[int, int]][source]#

Drop predictions that the detection protocol treats as unusable.

Two filters, both taken from telemanom, applied to sequences before they are scored or acted on.

min_length discards runs shorter than the given length. A single isolated sample above the threshold is a spike in the error signal rather than an event, and telemanom never promotes one to a sequence.

ignore_before discards sequences ending before the given index. A forecaster has no history at the start of a stream, so its errors there reflect the cold start rather than the data. telemanom skips the opening 2 * l_s samples, halving that for shorter streams and skipping nothing for very short ones.

Parameters:

  • sequences – Predicted (start, end) ranges, inclusive.

  • min_length – Shortest run to keep, in samples.

  • ignore_before – Index before which sequences are discarded.

Returns:

The retained subset, in the original order.

Return type:

list

telemetry_anomdet.thresholding.prune_sequences(errors: Sequence[float], sequences: list[tuple[int, int]], threshold: float, min_decrease: float = 0.13) → list[tuple[int, int]][source]#

Drop candidate sequences that are not clearly separated from the nominal tail.

Sequences are ranked by peak error, the nominal maximum is appended, and the relative drop between consecutive peaks is examined. Everything below the last drop exceeding min_decrease is reclassified as nominal: a gradual decline means the remaining candidates are part of the same population, whereas a sharp fall marks a real boundary.

Parameters:

  • errors – Error signal the sequences were found in.

  • sequences – Candidate (start, end) pairs from anomalous_sequences().

  • threshold – The threshold used to find them, needed to identify the nominal population.

  • min_decrease – Relative drop that counts as a boundary, as a fraction.

Returns:

The retained subset of sequences, in their original order.

Return type:

list

telemetry_anomdet.thresholding.detect_anomalies(errors: Sequence[float], threshold: float | None = None, min_decrease: float = 0.13, prune: bool = True, **threshold_kwargs) → dict[source]#

Full label-free detection: choose a threshold, find sequences, prune them.

Parameters:

  • errors – Error signal, one value per timestep.

  • threshold – Use this cutoff instead of selecting one. Useful for applying an operating point derived from training errors to new data.

  • min_decrease – Passed to prune_sequences().

  • prune – Set False to keep every sequence above the threshold.

  • **threshold_kwargs – Passed to dynamic_threshold().

Returns:

mask (boolean, one entry per timestep), threshold, sequences retained, and n_pruned sequences discarded.

Return type:

dict