AI Toolkit deep dives overview
The AI Toolkit lets you create, validate, manage, and operationalize machine learning models through a guided user interface. If you're unsure where to get started with the AI Toolkit you can use this series of deep dives to get a walk-t…
Deep dive: Create a data ingest anomaly detection dashboard using ML-SPL commands
Your data ingest pipelines can be impacted by traffic spikes, network issues, misconfiguration issues, and even bugs. These issues can cause unexpected downtime and negatively impact your organization. One option for accessing real-time…
Getting personalized AI output with AWS Knowledge Base RAG
AWS Bedrock LLM users can enhance the `ai` command with AWS Knowledge Base (KB) Retrieval-Augmented Generation (RAG). This feature transforms generic `ai` command responses into specific, actionable instructions, based on your environmen…
Custom visualizations in the AI Toolkit
The AI Toolkit includes several reusable custom visualizations that you can use in your own dashboards. Each visualization expects data in a certain format with certain fields, that you can see in the syntax portion of the visualization…
Deep dive: Using ML to detect outliers in error message rates
The goal of this deep dive is to identify when there are unusual spikes in the number of error messages generated by web applications. Being able to spot user experience issues as early as possible can help mitigate any potential loss of…
Learn more about the AI Toolkit
There are several ways to learn more about the AI Toolkit:
Deep dive: Using ML to identify network traffic anomalies
The goal of this deep dive is to identify periods of time when there is unusual data transfer traffic on your network. Spotting outliers in data transfer traffic data can help identify a multitude of issues ranging from the benign, to pe…
Deep dive: Inference externally trained ONNX models with the AI Toolkit
In version 5.4.0 and higher of the AI Toolkit, you can upload and inference Open Neural Network Exchange (ONNX) models pre-trained in your preferred third-party tool to the AI Toolkit. To learn how to use pre-trained ONNX models in your…
Deep dive: Using ML to detect outliers in server response time
The goal of this deep dive is to identify periods of time where web applications have unusually slow response times.
Share data in the AI Toolkit
When the AI Toolkit is deployed on Splunk Enterprise, the Splunk platform sends aggregated usage data to Splunk Inc. (\"Splunk\") to help improve the AI Toolkit in future releases. For information about how to opt in or out, and how the da…
Troubleshooting the deep dives
Use the following prompts to troubleshoot the deep dives.
AI Toolkit REST endpoints
The Splunk platform REST API gives you access to the same information and functionality available to core system software and Splunk Web. You can access the Splunk REST endpoints through both the `services` endpoint and the `servicesNS`…
Deep dive: Using ML to identify user access anomalies
The goal of this deep dive is to identify when there are unusual volumes of failed logons as compared to the historical volume of failed logins in your environment.
The AI Toolkit lets you create, validate, manage, and operationalize machine learning models through a guided user interface. If you're unsure where to get started with the AI Toolkit you can use this series of deep dives to get a walk-through of implementing the machine learning (ML) search commands that ship with the AI Toolkit for specific ML goals.
You can follow each deep dive from start to finish and implement the same operational outcomes in your own Splunk platform environment. Each deep dive consists of some example data sources, sample SPL code, and instructions for implementing the analytic.
Note: You might need to tune or modify these examples to work properly on your data. SPL knowledge is valuable when trying to implement these deep dives in your own environment.
What makes ML different from other analytics in Splunk products? Permalink to this section
Most analytics in the Splunk platform revolve around hard-to-find types of searches, where you are trying to spot a particular event or set of events that make up something of interest. For example, looking for memory errors on a server, or looking for a user running a process that is known to be malicious.
These types of analytics can usually be implemented with a single SPL search, whereas with ML you almost always need to run two searches: one to train a model, using the fit command, and one to apply a model, using the apply command.
The fit command is similar to the outputlookup command, and the apply is similar to the lookup. The apply stage is usually analogous with the hard-too-find detection search, but the training of models can seem unusual if you are new to machine learning. To learn more about how the fit and apply commands behave, see About the fit and apply commands.
Available deep dives Permalink to this section
The following deep dives are available:
- Deep dive: Using ML to detect user access anomalies
- Deep dive: Using ML to detect outliers in error message rates
- Deep dive: Using ML to detect outliers in server response time
- Deep dive: Using ML to detect network traffic anomalies
- Deep dive: Create a data ingest anomaly detection dashboard using ML-SPL commands
- Deep dive: Inference externally trained ONNX models with the AI Toolkit
Note: If you encounter questions while working on these deep dives, see Troubleshooting the deep dives.
See also Permalink to this section
See the following resources to learn more about the AI Toolkit:
- What is the AI Toolkit process?
- Preparing your data for machine learning
- Smart Assistants overview
- Configure algorithm performance costs
See the following resources to learn about the dedicated ML training course, our .conf archives, and numerous blog posts on the subject of machine learning and the AI Toolkit:
- Splunk 8.0 for Analytics and Data Science
- How Israel's Ministry of Energy applies Machine Learning to protect their Critical Infrastructure and OT Operations
- Augment your Security Monitoring Use Cases with MLTK's Machine Learning
- Anomaly Detection, Sealed with a KISS
- Cyclical Statistical Forecasts and Anomalies - Part 1
- Cyclical Statistical Forecasts and Anomalies - Part 2
- Cyclical Statistical Forecasts and Anomalies - Part 3
- Cyclical Statistical Forecasts and Anomalies - Part 4
- Cyclical Statistical Forecasts and Anomalies - Part 5
- Building Machine Learning Models with DensityFunction
- Anomalies Are Like a Gallon of Neapolitan Ice Cream - Part 1
- Anomalies Are Like a Gallon of Neapolitan Ice Cream - Part 2
Source: /en/splunk-cloud-platform/apply-machine-learning/use-ai-toolkit/5.7.3/ai-toolkit-deep-dives-library/ai-toolkit-deep-dives-overview (upstream Splunk AITK 5.7.3 docs)