StandardScaler

The StandardScaler algorithm uses the scikit-learn StandardScaler algorithm to standardize data fields by scaling their mean and standard deviation to 0 and 1, respectively. This preprocessing step helps to avoid dominance of one or more fields over others in subsequent machine learning algorithms. This step is practically required for some algorithms, such as KernelPCA and SVM. This algorithm supports incremental fit.

Parameters

• The with_mean and with_std parameters specify if the fields should be standardized with respect to their mean and standard deviation.
• The partial_fit parameter controls whether an existing model should be incrementally updated or not. This allows you to update an existing model using only new data without having to retrain it on the full training data set. The default is False.

Syntax

fit StandardScaler <fields> [into <model name>] [with_mean=<true|false>] [with_std=<true|false>] [partial_fit=<true|false>]

You can save StandardScaler models using the into keyword and apply new data later using the apply command.

... | apply scaling_model

You can inspect the statistics extracted by StandardScaler with the summary command.

...| summary scaling_model

Syntax constraints

• Using partial_fit=true on an existing model ignores the newly supplied parameters. The parameters supplied at model creation are used instead. If partial_fit=false or partial_fit is not specified (default is false), the model specified is created and replaces the pre-trained model if one exists.
• If My_Incremental_Model does not exist, the command saves the model data under the model name My_Incremental_Model.
• If My_Incremental_Model exists and was trained using StandardScaler, the command updates the existing model with the new input.
• If My_Incremental_Model exists but was not trained by StandardScaler, an error message is thrown.

Examples

The following example uses StandardScaler on a test set.

... | fit StandardScaler *  | ...

The following example includes the partial_fit parameter.

| inputlookup track_day.csv | fit StandardScaler "batteryVoltage", "engineCoolantTemperature", "engineSpeed" partial_fit=true into My_Incremental_Model

Local availability Permalink to this section#

• Local class: StandardScaler
• Source file: Splunk_ML_Toolkit/bin/algos/StandardScaler.py (in-repo path Splunk_ML_Toolkit/bin/algos/StandardScaler.py)
• algos.conf stanza: [StandardScaler]
• Class bases: TransformerMixin, BaseAlgo

Source Permalink to this section#

Adapted from the Splunk AI Toolkit 5.6.4 documentation at /en/splunk-cloud-platform/apply-machine-learning/use-ai-toolkit/5.6.4/algorithms-and-scoring-metrics-in-the-ai-toolkit/algorithms-in-the-ai-toolkit (section: preprocessor).