# Spyderbat Product Docs

Everything you need to know about our cloud-native runtime security platform that will provide you with continuous security monitoring, improved observability and timely alerting.

## Getting Started with Spyderbat

<table data-view="cards"><thead><tr><th></th><th></th><th></th></tr></thead><tbody><tr><td><a href="https://www.spyderbat.com/start-free?hsCtaTracking=9eec012f-6c31-45d2-bbf2-a59ac6842c8c%7C1a5474ef-1ffa-4700-a467-fad7a10bbd29">Try Spyderbat Community Edition for Free</a></td><td>Set up your test organization and deploy up to 5 Nano Agents</td><td></td></tr><tr><td><a href="/pages/iJKdd9ICuxb2IzNJazFP">Spyderbat System Requirements</a></td><td>Learn about infrastructure prerequisites and supported OS types before deploying Spyderbat</td><td></td></tr><tr><td><a href="/pages/iJKdd9ICuxb2IzNJazFP">How to Install a Spyderbat Nano Agent onto a K8s Cluster</a></td><td>How-To guide and a 6-minute video to get you started</td><td></td></tr></tbody></table>

## Navigating Spyderbat UI

<table data-view="cards"><thead><tr><th></th><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="/pages/Yn1kZI34qXIj8PUjXcE9">How to navigate Spyderbat Investigation</a></td><td>Get started with monitoring your environment at runtime</td><td></td><td></td></tr><tr><td><a href="/pages/1bWkiiV1C7lVwGuqgMJg">Spyderbat Dashboards from A to Z</a></td><td>Latest runtime security findings at your fingertips</td><td></td><td></td></tr><tr><td><a href="/pages/F1cU8pyxw1JlxXCsUtqD">Invite your team to experience Spyderbat potential</a></td><td>Manage your users and their access permissions</td><td></td><td></td></tr></tbody></table>

## Enhancing Your Spyderbat Experience

<table data-view="cards"><thead><tr><th></th><th></th><th></th></tr></thead><tbody><tr><td><a href="https://api.prod.spyderbat.com/openapi.pdf">Spyderbat API Integrations</a></td><td>View complete list of integrations</td><td></td></tr></tbody></table>


# Getting Started

Introduction to essential features and functionality of the Spyderbat console.

* [Sign Up and Create an Organization](/getting-started/create-an-organization)
* [Install the Nano Agent](/getting-started/install-the-nano-agent)
* [Spyderbat User and Role Management Overview](/getting-started/spyderbat-user-and-role-management-overview)
* [Three Things to Try with Spyderbat Community Edition](/getting-started/three-things-to-try-with-spyderbat-community-edition)
* [Help and Support](/getting-started/help-and-support)


# Create an Organization

Spyderbat Accounts are tied to one or more Organizations. Membership to an Organization is a requirement for installing the Spyderbat Nano Agent and fully utilizing Spyderbat as a whole.

All of the data generated by Spyderbat is tied directly to an Organization. Users can be members of multiple organizations, and may have different RBAC permissions for each org.

## Community Organizations

Spyderbat's free-tier is called Community, and all it takes to create a Community Spyderbat Organization is to create a Spyderbat Account. Receive the account creation instructions once you fill out a short [contact form](https://www.spyderbat.com/start-free).

## Professional/Enterprise Organizations

For those interested in trying out the professional and enterprise solutions, [contact sales](https://www.spyderbat.com/start-free). Our team will ensure will set one or more Spyderbat Organization(s) to suit your specific needs.


# Install the Nano Agent

Once you have successfully logged into your organization, the next step is to install the Nano Agent

Installing the Spyderbat Nano Agent is a requirement for using the multitude of security and operations features that Spyderbat has to offer.

#### Prerequisites

* Identify the Linux system(s) or Kubernetes Cluster you wish to secure with the Nano Agent.
* [View the supported operating systems](/installation/spyderbat-nano-agent) to see if your environment is compatible.

#### Installation

1. [Install the Nano Agent on a Standalone VM](/installation/spyderbat-nano-agent/linux-vm) - This installation path is tailored for setting up the Spyderbat Nano Agent on a standalone Virtual Machine (VM). Follow the comprehensive steps provided in the link. This approach is ideal for environments with one or more persistent VMs requiring the security and visibility offered by the Nano Agent.
2. [Install the Nano Agent across a Kubernetes Cluster](/installation/spyderbat-nano-agent/kubernetes) - If you are working with a Kubernetes Cluster and wish to deploy the Nano Agent across the entire cluster, refer to this installation path. This installation path is designed for environments utilizing Kubernetes orchestration, allowing for the automatic deployment and management of the Nano Agent across multiple nodes within the cluster.

For organizations with both type of environments, there is no issue installing some Nano Agents via option 1 and some via option 2.


# Three Things to Try with Spyderbat Community Edition

Review your security monitoring scope, trace your own activity at runtime, and validate detected suspicious activity via Spyderbat flags.

<mark style="color:blue;">Published: August 22, 2021</mark>

OK, you installed your first Spyderbat Nano Agent ([How-to Install the Spyderbat Nano Agent](/installation/spyderbat-nano-agent)). Now what?

### 1) Look at the last hour of activity

If you just installed the Spyderbat Nano Agent, you will see the system as a source on the **Sources** screen.

Click **Open In Search** on the source to query activity from that system, then search for Spydertraces.

### 2) View Your Own Activity

Do you still have a terminal open from when you installed the agent? If not, log back into the system you installed the agent on.

Run some simple Linux commands;

```
>clear
>id
>ls -la
> cat .profile
> whoami
> exit
```

Let's jump back to the Spyderbat console.

Under **Search**, click on the **End Time**, select the ‘**Now**‘ button to update the End Time to the current time, and then select the ‘**Run New Search**‘ button.

That query brought in records for the requested time period as a new Data Layer.

Look in the **Records** table for your recent activity. Click the **star icon** to the right of a record to add it to the **Causal Tree**, or click **Add All** to add everything.

Find your session in the **Records** table — it will show your bash shell and login user. Right-click on the bash shell process node in the **Causal Tree** and select **Search For Child Processes** to load the commands you ran.

For more details on using the Investigation view, see [Investigations](/concepts/flashback/investigations).

The Causal Tree updates to displays all commands (and processes) that are immediately causally connected to the bash shell. I also see the processes selected in the records table when I view the **Records** table **Process** tab.

By selecting the ‘cat’ node in the **Causal Tree** or process name in the **Records** table, the **Details** panel provides additional details such as the filename, the working directory, environment variables, and more!

### 3) View Your First Flag

Do you recall running the ‘whoami’ command? In our **Causal Tree**, it is annotated with a little flag.

Select the ‘whoami’ node in your **Causal Tre**e to view more information from the **Details** panel.

Flags are not the same as alerts. Flags color your Causal Tree with interesting information. The source of a Flag can be third-party alerts as well as other context sources. Spyderbat continuously overlays key security and other context as Flags as they occur.

A single Flag with no causal outcomes is a characteristic of a false positive. A trace of interest will usually include multiple Flags and multiple layers of activity. By viewing alerts and context as Flags, the **Causal Tree** shows you exactly how they are related, the sequence of activities, and any other activity causally connected.

### Other Things to Try

Here are some other great things to try with your Spyderbat Community Edition:

* Have a colleague do some basic admin tasks on a system that has the Spyderbat Nano Agent installed, see if you can figure out what they did in Spyderbat and compare notes with them.
* Install Spyderbat on a Vulnhub VM from [vulnhub.com](http://vulnhub.com/) and hack it, and see what Spyderbat shows. Many of the vulnhub images have walkthroughs if you are not an experienced pentester.
* Stand up a honeypot or similar system on the internet that can be easily exploited to see what Spyderbat captures!
* Want to bring in the rest of the team? Try a red team/blue team exercise where the red team attacks a set of Linux systems, and the blue team defends using Spyderbat!

Thank you and happy tracing!


# Manage Users and Roles

Manage team members and control their access levels in your Spyderbat organization.

Organization Admins can manage team members and their access levels from the **Admin** section of the Spyderbat Console, located at the bottom of the left-hand navigation panel.

<figure><img src="/files/7TbQfiFZwkZS7VQgCfyJ" alt="The Admin navigation item in the Spyderbat Console sidebar"><figcaption></figcaption></figure>

{% hint style="info" %}
Only users with the **Admin** role can see the Admin section and manage other users.
{% endhint %}

## Adding a user

1. Navigate to **Admin > Organization Management**.
2. Enter the user's email address.
3. Select a role from the **Roles** drop-down.
4. Click **Add User**.

<figure><img src="/files/UwV1Ze9xBHGqc80P7Bcw" alt="The Add User form with email and role fields"><figcaption></figcaption></figure>

The user appears in the Accounts list and receives an email invitation to join your organization.

## Removing a user

Removing a user revokes their access to your organization. It does not delete their account — user accounts belong to the individual and can be members of multiple organizations.

1. In **Admin > User Management**, hover over the user's row.
2. Click the **delete** icon.
3. Confirm the removal.

{% hint style="info" %}
Removed users do not receive a notification email. If they have an active session, they are immediately logged out.
{% endhint %}

## Changing a user's role

1. In **Admin > Organization Management**, find the user in the Accounts list.
2. Click the **Roles** drop-down on that user's row.
3. Select the new role.
4. Deselect the old role.
5. Click **Save**.

<figure><img src="/files/CwIkC1vegul1bB851XxP" alt="The role change dropdown showing available roles and Save button"><figcaption></figcaption></figure>

{% hint style="info" %}
If you don't deselect the old role, the user has both roles assigned. The higher-permission role applies, which may grant more access than intended.
{% endhint %}

## Roles and permissions

Spyderbat has five organization-level roles. The table below summarizes what each role can access:

<figure><img src="/files/BpdPjGBMptRRnyKPL18H" alt="The role dropdown showing all five available roles"><figcaption></figcaption></figure>

| Capability                               | **Admin**   | **Power User** | **Non-API User** | **Agent Deployment** | **Read Only** |
| ---------------------------------------- | ----------- | -------------- | ---------------- | -------------------- | ------------- |
| Organization and user management         | Full access | No access      | No access        | No access            | No access     |
| Source management and agent installation | Full access | Full access    | Full access      | View and add         | View only     |
| Agent health monitoring                  | Full access | Full access    | Full access      | View only            | View only     |
| Dashboards (view and create)             | Full access | Full access    | Full access      | No access            | View only     |
| Search queries                           | Full access | Full access    | Full access      | No access            | View only     |
| Investigations (process and K8s)         | Full access | Full access    | Full access      | No access            | View only     |
| Notification setup                       | Full access | No access      | No access        | No access            | No access     |
| Guardian policy management               | Full access | Full access    | Full access      | No access            | View only     |
| API key creation                         | Full access | Full access    | No access        | Full access          | No access     |

### Admin

Full access to all console features. Admins can invite and remove users, change roles, manage sources, configure notifications, and perform all investigative and administrative actions.

{% hint style="warning" %}
Limit the number of Admin users in your organization. One or two Admins is typically sufficient.
{% endhint %}

### Power User

Full access to all console features except organization and user management. Power Users can manage sources, create dashboards, run searches, conduct investigations, and create API keys.

### Non-API User

Same access as Power User but cannot create API keys. Use this role for team members who need full console access without programmatic API access.

### Agent Deployment

A limited role for onboarding engineers responsible for installing Spyderbat Nano Agents. These users can view and add sources, access agent install scripts, and monitor agent health. They cannot access dashboards, search, or investigations.

### Read Only

View-only access to most console features — dashboards, sources, agent health, investigations, and search. Read Only users cannot modify data (e.g., rename sources, archive machines) and cannot access the Admin section.


# Help and Support

How to get help, report issues, and find the right contact for your question.

## Who to Contact

Not every issue requires Spyderbat support. Some problems are best resolved by your organization's Spyderbat administrator.

### Your Spyderbat Admin

Your Spyderbat admin is the person in your organization with the Admin role. They manage user accounts, organization settings, and access configuration. Contact them for:

* Adding or removing users from your organization
* Changing user roles and permissions
* Sign-in and access issues specific to your organization
* Organization-level configuration changes

{% hint style="info" %}
Not sure who your Spyderbat admin is? Check with your IT or security team. The admin is typically the person who originally created your Spyderbat organization.
{% endhint %}

### Spyderbat Support

For platform bugs, feature questions, agent issues, API help, or account and billing questions, reach out through any of these channels:

| Channel                                                                         | When to use                                                                                                                         |
| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| **Email**: <help@spyderbat.com>                                                 | Bug reports, account issues, questions about platform behavior                                                                      |
| **Slack**: [spyderbatcommunity.slack.com](https://spyderbatcommunity.slack.com) | Community discussion, quick questions, feature requests. Request an invite from Spyderbat support if you don't already have access. |
| **Contact form**: [spyderbat.com/contact](https://www.spyderbat.com/contact)    | Sales inquiries, partnership requests, demo scheduling                                                                              |

## Reporting an Issue

When you contact support, include the following to help us resolve your issue quickly:

* **What you were doing** when the problem occurred
* **What you expected** to happen vs. what actually happened
* **Screenshots** of any error messages or unexpected behavior
* **Browser and OS** (e.g., Chrome 120 on macOS 14)

{% hint style="info" %}
If the Spyderbat Console displays an error screen, it may include a reference code, timestamp, and error details. Use the **Copy to Clipboard** button on the error screen to capture this information, then paste it into your support email or Slack message.
{% endhint %}

## Troubleshooting Common Issues

### Unable to Sign In

* **Retry the sign-in.** Close the tab and navigate to the Spyderbat Console again.
* **Clear your browser cache and cookies**, then try again.
* **Check with your Spyderbat admin** to confirm your account is active and properly configured.
* If the issue persists, contact [Spyderbat support](mailto:help@spyderbat.com) and include any error details shown on screen.

### No Organization Access

If you can sign in but cannot access any organization, your account has not been added to one or was removed. Contact your Spyderbat admin to verify your account status and get an invitation.

## Related Documentation

* [Manage Users and Roles](/getting-started/spyderbat-user-and-role-management-overview) -- adding users, assigning roles, and managing permissions
* [Create an Organization](/getting-started/create-an-organization) -- initial setup of your Spyderbat organization
* [How to Create and Use a Spyderbat API Key](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api) -- API access for integrations


# Installation

Spyderbat Nano Agent installation and deployment configuration options.

***


# Spyderbat Nano Agent

Nano Agent operational principles, compatibility, network requirements and proxy support, general FAQ

### How does Spyderbat collect data?

Spyderbat collects data by deploying a lightweight “Nano Agent” for Linux based systems. The agent leverages [eBPF](https://ebpf.io/) (“extended Berkeley Packet Filter”) filters to build a continuous map of activity within and across systems.

### Why do I need to install an agent?

Existing endpoint agents and system logs do not include the necessary information required by Spyderbat to build a complete, living map of causal activity within and across systems. Spyderbat’s Nano Agent is optimized to collect this information so that analysts can see the complete causal attack picture across systems, users, and time.

### What is the impact of the Spyderbat Nano Agent on the system?

Spyderbat has observed minimal impact on system resources (CPU, memory), and minimal network bandwidth impact due to heavy compression.

### What operating systems are currently supported?

Spyderbat currently supports the following Linux systems:

| Linux Version                                | Architecture        |
| -------------------------------------------- | ------------------- |
| AlmaLinux 9                                  | x86\_64 / Power64le |
| Amazon Linux 2                               | x86\_64 / ARM64     |
| Amazon Linux 2022                            | x86\_64 / ARM64     |
| Amazon Linux 2023                            | x86\_64             |
| Amazon Linux Bottlerocket                    | x86\_64             |
| CentOS 7 up to 7.6 (with El Repo LT)         | x86\_64             |
| CentOS 7.6+ (with Kernel 3.10.0-957+)        | x86\_64             |
| CentOS 8                                     | x86\_64             |
| Oracle Linux 8                               | x86\_64 / ARM64     |
| Oracle Linux 9                               | x86\_64 / ARM64     |
| Debian 11                                    | x86\_64             |
| Debian 12                                    | x86\_64 / ARM64     |
| Debian 13                                    | x86\_64 / ARM64     |
| Flatcar Container Linux (3227.2.1; 3374.2.3) | x86\_64             |
| Google Container-Optimized OS (GCOS)         | x86\_64             |
| Kali 2021.2                                  | x86\_64             |
| RHEL 7.6+ (with Kernel 3.10.0-957+)          | x86\_64             |
| RHEL 8                                       | x86\_64 / Power64le |
| RHEL 9                                       | x86\_64 / Power64le |
| Rocky Linux 8                                | x86\_64 / Power64le |
| Rocky Linux 9                                | x86\_64 / Power64le |
| Sangoma 16 (with El Repo LT)                 | x86\_64             |
| SLES                                         | x86\_64 / Power64le |
| Ubuntu 18.04. LTS                            | x86\_64             |
| Ubuntu 20 Desktop                            | x86\_64             |
| Ubuntu 20.04 LTS                             | x86\_64 / ARM64     |
| Ubuntu 20.10                                 | x86\_64             |
| Ubuntu 22.04                                 | x86\_64             |
| Ubuntu 24.04                                 | x86\_64             |

### What K8s Distributions are currently supported?

Spyderbat Nano Agents can be currently installed on the K8s clusters utilizing the following distributions:

| K8s Distribution     | Node Operating System                 | Container Runtime    |
| -------------------- | ------------------------------------- | -------------------- |
| EKS                  | <p>Amazon Linux 2<br>Bottlerocket</p> | containerd or Docker |
| GKE                  | <p>Ubuntu<br>GCOS</p>                 | containerd           |
| Red Hat OpenShift    | V4.xx Power64le                       | containerd or Docker |
| Rancher RKE and RKE2 | Ubuntu 20 LTS                         | containerd or Docker |
| MicroK8s             | Ubuntu 22 LTS                         | containerd           |
| K3s                  | Ubuntu 22 LTS                         | containerd           |
| aks                  | Ubuntu 22 LTS                         | containerd           |
| robin.io             | RockyLinux 8 or 9                     | containerd           |

Spyderbat Nano Agents use only standard Kubernetes APIs and standard Kubernetes resources. It should run on most Kubernetes clusters.

### What are the Nano Agent’s network requirements?

Ensure that the systems running the Nano Agent have outbound access on port 443 to <https://orc.spyderbat.com>.

### Does the Nano Agent support network proxies?

Yes. If you have a proxy configured and you have Linux environment variables like:

```
    https_proxy=:port
```

The installation script will automatically grab the environment variables from your terminal using the “-E” flag and pass those to the agent as required.

### Is information sent securely from the Nano Agent?

Yes. Spyderbat securely encrypts information sent by the Nano Agent to the Spyderbat backend using TLS.

### Does the Nano Agent support systems hosted in AWS?

The Nano Agent can be installed on any of the supported systems listed above as virtual or physical machines. Additionally, the Nano Agent collects metadata from AWS instances such as Cloud Tags, Region, Zone etc. To collect this metadata, ensure your AWS instances have an appropriate IAM (read only) role assigned to them such as “AmazonEC2ReadOnlyAccess”, see <https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/security-iam-awsmanpol.html>

### How do I start and stop the Nano Agent from the command line?

To start the Nano Agent:

```
  sudo systemctl stop nano_agent.service
```

To stop the Nano Agent:

```
  sudo systemctl start nano_agent.service
```


# Kubernetes

Nano Agent install via public or locally hosted Helm Chart or manually via daemonset; configuring parameters (memory and CPU resources, priority class), and validating install into a K8s cluster.

<mark style="color:blue;">Published: October 11, 2022</mark>

The Spyderbat Nano Agent in a containerized environment can be deployed via a Kubernetes Daemonset to a target Kubernetes Cluster. To guarantee proper coverage, it is important to ensure that a single instance of the Spyderbat Nano Agent runs on every cluster node (and is optionally deployed to API server Control Plane nodes for self-managed clusters).

Spyderbat offers a simple deployment approach via Helm Chart, which is a package manager tool for Kubernetes that creates the necessary pods, permissions, network rules, etc. Instructions are also provided below for cases where the target cluster does not have internet access to the necessary artifacts, and the deployment is executed with a simple Kubernetes Daemonset manifest.

### Infrastructure Prerequisites

The Spyderbat Nano Agent leverages eBPF technology on Linux systems to gather data and forward it to the Spyderbat backend. A full list of supported Linux OS can be found on our website [here](/installation/spyderbat-nano-agent) (paragraph 4).

Successful Spyderbat Nano Agent install and new source registration in the Spyderbat UI require that the agent has outbound access on port 443 to `https://orc.spyderbat.com`, so that the Nano Agent could successfully pull all needed updates and register with the Spyderbat backend. This means that the pod running the Nano Agent should have outbound access from the Kubernetes Cluster and target namespace to the target port and domain above.

In order to verify successful agent installation, the person installing the Spyderbat agent should also ideally have a Spyderbat Admin account in their Spyderbat organization and should be able to access their organization in the Spyderbat UI at [https://app.spyderbat.com](https://app.spyderbat.com/)

### Public Helm Chart Deployment: Clone Repo, Update and Install

Below is the set of deployment instructions for your K8 Kubernetes cluster, which is available in the Spyderbat UI Under Sources -> Add New Source. This deployment will run with all default settings for the parameters referenced above, which have been pre-populated.

```
helm repo add nanoagent https://spyderbat.github.io/nanoagent_helm/
helm repo update
helm install nanoagent nanoagent/nanoagent  
--set nanoagent.agentRegistrationCode=<agent registration code> 
--set nanoagent.orcurl=https://orc.spyderbat.com/
```

The agent registration code is specific to your organization (see below), and the ORC Url is the endpoint where your Nano Agents will register / communicate to the Spyderbat backend.

If you wish to store your Agent Registration Code in the AWS Secrets Manager, please [refer to this article](/installation/spyderbat-nano-agent/aws-unattended-install/aws-secrets-manager-overview) for more information on how to set it up.

The Helm installation commands specific to your organization can be found in the Spyderbat UI:

1. Navigate to **Sources** in the left-hand navigation
2. Click **New Source**
3. Select **Kubernetes** from the installation options
4. Copy the pre-populated Helm commands shown in the wizard

<figure><img src="/files/eITP3yoyKQttI0USyyth" alt="Sources page with Add Source menu"><figcaption></figcaption></figure>

<figure><img src="/files/O6gEfVG5Ok3eNgs5axxi" alt="Kubernetes install wizard with Helm commands"><figcaption></figcaption></figure>

### Customizing the Helm Chart Values

To get the Helm Chart source, you may clone the repo by running the following command:

```
git clone https://github.com/spyderbat/nanoagent_helmchart.git
```

The Spyderbat Helm Chart includes a set of yaml files and configurable parameters that can be optionally modified by the user before running the Helm Chart on a target Kubernetes cluster.

### Resources

The user will have the ability to specify the resource request for containers in a pod, which will enable kube-scheduler to decide which node to place the Pod on. The user will also be able to specify the resource limit for a container, so the kubelet will enforce those limits so that the running container is not allowed to use more of that resource than the limit set. The kubelet also reserves at least the requested amount of that system resource specifically for that container to use.

There are two resource types to configure: CPU and memory.

CPU is specified in units of Kubernetes CPUs, where 1 CPU unit is equivalent to 1 physical CPU core, or 1 virtual core. For CPU resource the expression 0.1 is equivalent to 100m, which can be read as “one hundred millicpu” or “one hundred millicores”.

The Memory is specified in units of bytes using either an integer format or a power-of-two equivalent. For example, 2048 Mi is the equivalent of 2048 mebibytes or MiB.

By default the resource requests are set to the following values:

* CPU at 100m = 0.1 of a single CPU core (physical or virtual)
* Memory at 512Mi = 512 MB

And the resource limits are set to the following values:

* CPU resources are hard capped at 6 CPU cores
* Memory resources are hard-capped at 10 GB of RAM

### Priority Class

This is a non-namespaced object that defines a mapping from a priority class name to the integer value of the priority: the higher the value, the higher the priority. A PriorityClass object can have any 32-bit integer value smaller than or equal to 1 billion. By default, the priority will be set to lowest.

Once the priority class is set, within the Customer’s priority scale, the agent will be installed on every node in the cluster as to their priority. If the priority class is set too low, then the pods could be preempted or evicted, so if the user wants to ensure that there is an agent installed on every node in the cluster when the pod is created, then the priority should be set accordingly.

For example:

* 100 – 1000 – low priority
* 100K+ – ultra-high priority

By default, the priority class is disabled. But if it is enabled, then the default value will auto-set to 1000.

It is important to keep in mind that if the Priority Class remains disabled then the Spyderbat Nano Agent may never get installed in the event there is no capacity.

### Namespace

Namespaces provide a mechanism for isolating groups of resources within a single cluster.\
If the namespace parameter is set to false and the agent installer is run, a single pod will be created in the default namespace.

Once this parameter is enabled (set to “true”), the “create namespace” argument will be used to create the “Spyderbat” namespace as part of the deployment.

### Service Account

During Spyderbat Nano Agent deployment into the Kubernetes cluster, the daemon set puts an agent on every node in the cluster. ClusterMonitor creates a special agent that monitors the Kubernetes cluster itself. It is the ClusterMonitor that needs service account permissions to enable such monitoring. The name of the service account can be changed in values.yaml, it defaults to “spyderbat-serviceaccount”.

The service account uses a “clusterrolebinding” of cluster-admin which allows it to read all the cluster configuration and gives it the ability to terminate pods to stop attacks.

If you do not wish to use preventive actions, the cluster role can be altered in values.yaml to only have “ReadOnly” and “Watch” permissions.

To update desired parameters via the Command Line prompt, use the following command sequence:

```
helm install nanoagent nanoagent/nanoagent  
--set nanoagent.agentRegistrationCode=<agent registration code> 
--set nanoagent.orcurl=https://orc.spyderbat.com/ 
--set-string resources.requests.cpu=1000m 
--set priorityClassDefault.value=10000
```

This would replace the resources.requests.cpu with 1000m instead of the default 100m. For numeric settings use `--set` instead of `--set-string`

Below is the summary table with all the defaults for your reference:

| Parameter               | Parameter from values.yaml | Default State  | Default Value (if enabled)                                                                                                                                                                                       |
| ----------------------- | -------------------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CPU resource request    | requests: cpu              | N/A            | 100m                                                                                                                                                                                                             |
| CPU resource limit      | limits: cpu                | N/A            | 6000m                                                                                                                                                                                                            |
| Memory resource request | requests: memory           | N/A            | 512Mi                                                                                                                                                                                                            |
| Memory resource limit   | limits: memory             | N/A            | 10240Mi                                                                                                                                                                                                          |
| Priority class          | priorityClassDefault       | enabled: false | 1000                                                                                                                                                                                                             |
| Namespace               | namespaceSpyderbat         | enabled: true  | spyderbat                                                                                                                                                                                                        |
| Omit Environment        | OMITENVIRONMENT            | "no"           | "no" emit all environment variables. "everything" omits all environment variables and "allbutredacted" uses our rules to encrypt variables that look like they contain secrets and emit only those for analysis. |

To configure access via a proxy you can add additional parameters to the Helm command line:

```
--set nanoagent.httpproxy=http://123.123.123.123:2/ 
--set nanoagent.httpsproxy=http://123.123.123.123:2/
```

To set the resource limits additional parameters like there can be added to the Helm command line. We recommend 3-5% of the resources on a node as a limit

```
--set resources.limits.cpu=2000m 
--set resources.limits.memory=8192M
```

### Helm Chart Package Contents

The Helm Chart packages the following installer files:

* Nanoagent.yaml file: used to ensure a copy of the pod is created on every node.
* Serviceaccount.yaml: creates service account as part of the deployment to allow leveraging K8 API’s
* Namespace.yaml: creates Spyderbat namespace for resource management
* Priority.yaml: sets priority for Spyderbat pod deployment on all nodes in the K8 cluster
* Clustermonitor.yaml: creates a ClusterMonitor Nano Agent that collects information from the K8s API
* Rolebinding.yaml: defines the service account cluster role binding for the Spyderbat service account
* Values.yaml: contains the user configurable parameters for the Helm Chart install

### Deployment via Self-Hosted Helm Chart and Docker Container Image

In the scenario, where you want to host the Helm Chart and Container Image locally, you may leverage the following instructions. Note that the pod running the Spyderbat Nano Agent still requires outbound internet access to `https://orc.spyderbat.com` on port 443.

On a machine with internet access, you can pull the Spyderbat container image into your local docker system with the following command:

```
docker pull public.ecr.aws/a6j2k0g1/nano-agent:latest
```

To see the image id and that it is local:

```
docker image ls
```

Which will return results looking like the following:

```
REPOSITORY                           TAG       IMAGE ID       CREATED         SIZE
<none>                               <none>    72bb338b2313   2 minutes ago   151MB
ubuntu                               latest    27941809078c   3 weeks ago     77.8MB
public.ecr.aws/a6j2k0g1/nano-agent   latest    dde533638cf2   2 months ago    148MB
```

You can export the image with:

```
docker image save dde533638cf2 > docker.image.nano_agent
```

The file docker.image.nano\_agent can be imported into your local repository.

Or alternatively, you may download a compressed image like this:

```
curl https://spyderbat.github.io/nanoagent_helm/docker.image.nano_agent.gz 
--output agentimage.tar.gz
```

This image is gzip compressed but can be installed into your registry or repository.

To get the Helm Chart for internal hosting:

```
curl https://spyderbat.github.io/nanoagent_helm/agent_helm.tar 
--output nano_agent_helmchart.tar
```

You can unpack the Helm Chart with:

```
tar xvf nano_agent_helmchart.tar
```

In `nanoagent/values.yaml`, edit the image section to point to your new image registry:

```yaml
image:
  repository: your-registry.example.com/nano-agent
  tag: latest
  pullPolicy: IfNotPresent
```

The Helm chart can be used locally, or you can host it.

### Deployment via a Daemonset

Should it be required to manually run the Spyderbat Nano Agent install, yaml files can be extracted and run one by one in a very controlled fashion.

To extract the files from the Helm Chart available in a public GitHub repository run the following command using your organization registration code (see [Public Helm Chart Deployment: Clone Repo, Update and Install](#public-helm-chart-deployment-clone-repo-update-and-install) section for detail on how to find your agent registration code):

<pre><code><strong>helm template nanoagent nanoagent/nanoagent 
</strong><strong>--set nanoagent.agentRegistrationCode=&#x3C;agent registration code> 
</strong><strong>--set nanoagent.orcurl=https://orc.spyderbat.com/ 
</strong><strong>--create-namespace 
</strong><strong>--set spyderbat_tags='CLUSTER_NAME=mycluster:environment=dev'
</strong></code></pre>

Once run, this command will produce the following YAML manifests:

* ServiceAccount
* ClusterRole and ClusterRoleBinding
* Namespace (if `--create-namespace` is set)
* PriorityClass (if enabled)
* DaemonSet for the nano agent
* Deployment for the cluster monitor

You can then proceed to modify the desired parameters in the respective files as noted above and run individual files one by one to complete the Spyderbat Nano Agent installation.

### Validation

If the installation proceeded correctly, you should see output confirming the Helm release was deployed successfully.

Once the agents register with Spyderbat's backend, you will see active sources in the Spyderbat UI (one per cluster node) under **Sources**, each with a recent registration timestamp.

<figure><img src="/files/Xgcgt7m7h0LjfAhexRei" alt="Spyderbat UI Sources page showing registered nodes"><figcaption></figcaption></figure>

Once the Spyderbat Nano Agents have been installed, you can validate the pods are running with the following command:

```
kubectl get pods -n spyderbat
```

You should see output similar to this, with one pod per cluster node:

```
NAME                      READY   STATUS    RESTARTS   AGE
nanoagent-abcd1           1/1     Running   0          2m
nanoagent-efgh2           1/1     Running   0          2m
nanoagent-ijkl3           1/1     Running   0          2m
```

Note that the free Spyderbat Community account allows you to monitor up to 5 nodes, i.e. register up to 5 sources in the Spyderbat UI. If you have a cluster that contains more than 5 nodes or anticipate scaling up in the near future, please visit <https://www.spyderbat.com/pricing/> to sign up for our Professional tier.


# Pre Deployment Environment Data Collection Script

Optimize your Helm Chart values to ensure proper sizing of the Spyderbat Nano Agent parameters for your K8s environment.

### What is Pre-Deployment Collection Script and How It Works

To optimally configure and size the Spyderbat Nano agent and backend to support your Kubernetes cluster, we have created a script that will collect some useful data and metrics that Spyderbat can review to optimize the Helm installation or our agents and size Spyderbat backend appropriately.

### Script Output Details

The script collects the following data in your environment:

1. Summary metrics about the number of nodes, pods, deployments, replicasets, daemonsets, services and namespaces, which helps us assess the size and load on your cluster.
2. Information about the nodes of the cluster, including their provisioned capacity and any taints applied to the nodes, which helps us understand the headroom available in your cluster to add our agents, and helps us pro-actively recommend configuring tolerations on our agents to ensure visibility on all nodes.
3. Cumulative metrics about what resource requests currently running pods are requesting (CPU, memory), which helps us understand the headroom available in your cluster to add our agents.
4. The name and namespaces of the deployments, daemonsets and services running on your cluster, which helps us assess if any other daemonsets or deployments could interfere with our agents and helps us discover if your cluster has node-auto-scaling configured.
5. PriorityClasses currently present for the cluster which helps us assess whether our agent will have sufficient priority to get scheduled on any new nodes being added to the cluster.

The script does NOT collect any of the following:

* Implementation and status details in the 'spec' and 'status' sections of the pods, deployments or daemonsets.
* Any sensitive data that might be present in these sections of the k8s resources (environment variables, configs)

### Script Execution Prerequisites

Spyderbat Pre-Deployment Collection script should be run from a machine you currently use to manage your cluster from.

Below are the requirements for the script to run successfully:

1. python3 <https://www.python.org/downloads/>
2. kubectl and a valid kube config file <https://kubernetes.io/docs/tasks/tools/>

   The script will call on the kubectl command to collect cluster information. The cluster(s) to install Spyderbat on should be one of the contexts configured in the kube config file.

### Script Execution Steps

First you will need to download the cluster\_collect.py script from [this public repository](https://github.com/spyderbat/deployment_tools).

After installing the script run it as

*`./cluster_collect.py -h`*

OR

*`python3 cluster_collect.py -h`*

For usage info run

*`usage: cluster_collect.py [-h] [-c CONTEXT] [-o OUTPUT]`*

Here are available options:

* `-h, --help` show this help message and exit
* `-c CONTEXT, --context CONTEXT` kubectl context to pull from (if none provided, all contexts in the kubectl config will be analyzed)
* `-o OUTPUT, --output OUTPUT` output file (default is Spyderbat-clusterinfo.json.gz)

By default, the script will collect information for all clusters configured in your kubeconfig file.

If you want to collect only for one cluster, use the -c CONTEXT flag, with the name of the context (as available in kubectl config get-contexts) to collect for.

For example:

*`./cluster_collect.py -c qacluster1`*

By default the output will go into a file called **spyderbat-clusterinfo.json.gz.** You can use the -o flag to use another filename.

### Output Delivery and Review

If the script ran successfully, please send the output file back to Spyderbat. We will review the findings with you to discuss the next steps for your deployment and provide recommendations on how to best configure your deployment parameters to ensure that all Spyderbat Nano Agents come online, initialize fully, and successfully register with the Spyderbat backend.

Here is an example of the output file data:

<figure><img src="/files/McpNSUWgUSOk0fl2OV6t" alt=""><figcaption><p>Click to enlarge</p></figcaption></figure>

If you would like to review an example of a full file output, please [Contact US](mailto:support@spyderbat.com).


# Linux Standalone

This section covers: adding a source to your monitoring scope in Spyderbat console, generating installation scripts/commands, and running the agent installer on the target machine.

<mark style="color:blue;">Published: August 23, 2021</mark>

The Spyderbat Nano Agent is an extremely lightweight collector to access unprecedented insight from Linux systems and their causal activity and relationships. The Spyderbat Nano Agent leverages proven technology, [Enhanced Berkeley Packet Filters or “eBPF”](https://ebpf.io/). The Spyderbat Nano Agent collects targeted, non-human readable data using eBPF from modern 64-bit Linux distributions for both x86 and ARM based architectures.

### Step 1

Select **Sources** from the left-navigation menu. The wizard will launch automatically if you don't have any agents installed yet. The wizard will guide you through the installation in a few simple steps.

<figure><img src="/files/eITP3yoyKQttI0USyyth" alt="Sources page with install wizard launch button"><figcaption></figcaption></figure>

### Step 2

You will see two choices regarding the target system you want to install the Nano Agent on.

* If you are installing on a virtual Linux system in AWS, select “EC2 instance” – where you’ll want to give the AWS instance an IAM (read only) role to grab metadata like Cloud Tags etc.
* If you are installing on any instance of Linux (virtual or physical) select “Standalone”.

Ensure you are installing on a Linux system supported by Spyderbat. You can view a complete list of supported versions [here](/installation/spyderbat-nano-agent).

After you have made your selection, hit “next step”.

{% hint style="warning" %}
Note the system you’re installing Spyderbat's Nano Agent should have outbound access on port 443 to <https://orc.spyderbat.com>
{% endhint %}

### Step 3

In the next step, you’ll see a command you can copy and paste into a terminal on the target system. If you do not have Curl installed on your system, select the **wget** tab to copy this command instead.

{% hint style="warning" %}
Note - You will need sudo permissions to install the Spyderbat Nano Agent
{% endhint %}

The UI provides you with feedback by displaying check marks of the install progress. Once the Spyderbat Nano Agent is installed, registers with Spyderbat, and it transmitting data, you will see that the agent was installed successfully both in your terminal and in the Spyderbat UI.

<figure><img src="/files/IdxgmgGwbECVYmConjOX" alt="Install wizard showing Linux install command"><figcaption></figcaption></figure>

Once you see every checkmark displayed, click on ‘Next Step’ to be directed to the **Sources** page. You should now see the system you just installed the Nano Agent onto.

You should see that the source is healthy, the last active time should indicate recent activity and the sparkline will start to indicate a summary of system activity over time. You can rename the Source if necessary.

<figure><img src="/files/Xgcgt7m7h0LjfAhexRei" alt="Sources page showing installed agents"><figcaption></figcaption></figure>

You are now ready to jump into an **Investigation**! Click on “**View Spydertrace**” link for the source will take you by default into the last hour of activity for that system in the “investigate” view.

[Click here to learn more about Spyderbat Investigations](/concepts/flashback/investigations)

Congratulations – you installed the Spyderbat Nano Agent!

Happy tracing!


# AWS Unattended Install

Automatic installation of the Spyderbat Nano Agent on an AWS EC2 instance with auto-scaling groups using the instance launch wizard.

<mark style="color:blue;">Published: November 19, 2021</mark>

### Introduction

In this walkthrough, we’ll show how you can install the Spyderbat Nano Agent automatically when an AWS EC2 instance is created – this can be useful particularly for ephemeral instances, such as when leveraging AWS auto scaling groups for example. We’ll walk through creating an EC2 instance in the AWS console using the instance launch wizard, and leverage the ability to pass in user data at instance creation time – for more information about user data and cloud-init, see AWS docs [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-cloud-init). For installing the Spyderbat Nano Agent in an attended fashion, see the walkthrough guide [here](/installation/spyderbat-nano-agent/linux-vm).

### Step by step guide

1\) The 1st step is to retrieve the command to install the agent for your organization – click on the “New Source” button in the sources section of the product for your organization

<figure><img src="/files/eITP3yoyKQttI0USyyth" alt="Sources page with New Source button"><figcaption></figcaption></figure>

2\) Once you click on this button, you should be launched into the agent installation wizard where you will be presented with a link to install the agent, let’s copy the “wget” version of the install command and save that to the notepad.

<figure><img src="/files/IdxgmgGwbECVYmConjOX" alt="Agent installation wizard with install command"><figcaption></figcaption></figure>

3\) Now go to the [AWS EC2 management console](https://console.aws.amazon.com/ec2/home).

4\) Go to Instances and use the Launch Instances wizard to request one or more instances.

5\) Choose the desired AMI for the new instances and click Select.

6\) Choose the desired instance type. Then click Configure Instance Details.

7\) At the bottom of the "Configure Instance Details" screen, expand the "Advanced Details" section and locate the "User data" input box.

8\) In the user data field, we will enter a shell script to run the install command we copied to our notepad, similar to the below (for RedHat family distributions):

```
#!/bin/bash
yum install - y wget lsof
wget -- quiet - O -
https://orc.spyderbat.com/v1/reg/OMJBdOBVZvzFGEMLgQSt/script | /bin/sh
```

\
The 1st line indicates this is a bash shell script, the second line ensures the ‘wget’ and ‘lsof’ utilities are installed, and the 3rd line is the install command you copied from the installation wizard. Note that we have omitted “sudo -E” from the command we copied since the user data script is run as root when the instance boots. For Debian family based distributions, the following can be used:

```
#!/bin/bash yum install - y wget lsof wget -- quiet - O - https://orc.spyderbat.com/v1/reg/OMJBdOBVZvzFGEMLgQSt/script | /bin/sh
```

\
9\) Continue with the steps in the install wizard, or jump to Review and Launch if you are done.

10\) When the instance is created in AWS, it should now download and install the agent as part of the boot sequence (for reference, the cloud-init output log file is created at /var/log/cloud-init-output.log on the created instance) – note you should ensure the instance(s) that are created have outbound access on port 443 to <https://orc.spyderbat.com>.

11\) Check the “sources” section of the Spyderbat and you should now see your new instance appear in your list of sources.

You can leverage the user data in a similar fashion when using other mechanisms to create AWS EC2 instances, for example when specifying a [launch template for an Auto Scaling group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/create-launch-template.html).

[Click here for more information about Spyderbat’s Nano Agent](/installation/spyderbat-nano-agent)


# Secure your Registration Code with AWS Secrets Manager

Learn how to leverage AWS Secrets Manager as a secret store for the Nano Agent Registration Code (There is an assumption that you are familiar with AWS, IAM, and EKS and how the 3 interact).

### Overview

Spyderbat Nano Agent registration code is a unique alpha-numeric combination that is used to associate the installed Nano Agents with your organization in Spyderbat backend and the data. This registration code can only be visible in the Spyderbat UI to the users in your organization with the relevant permissions (check out our [article on User Roles nd Permissions](/getting-started/spyderbat-user-and-role-management-overview) for more info).

You may choose to store your organization's Nano Agent Registration Code in the AWS Secrets Manager, to facilitate the use of automation for agent deployment or to adhere to required internal processes, in which case you will need to follow the steps below.

### Adding the Agent Registration Code to AWS Secrets Manager

First you will need to store the registration keys in Secrets Manager and get the ARN for it:

```
aws secretsmanager create-secret --name \<name\> --region \<region\>
- aws secretsmanager put-secret-value --secret-id \<name\> --region \<region\> --secret-string "{\"spyderbat-registration-key\":\"\<key\>\"}"
aws secretsmanager get-secret-value --secret-id \<name\> --region \<region\>

```

Next step is to create an IAM Policy that allows GetSecretValue and DescribeSecret for it. After that, add the AWS secrets store csi driver to your cluster if it is not already available.

```
helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts

helm install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver --namespace kube-system --set syncSecret.enabled=true

kubectl apply -f https://raw.githubusercontent.com/aws/secrets-store-csi-driver-provider-aws/main/deployment/aws-provider-installer.yaml
```

### Accessing the Agent Registration Code in AWS Secrets Manager

Create a role that will have access to the above mentioned policy and is federated to your eks cluster (see associate-iam-oidc-provider):

```
eksctl create iamserviceaccount --name spyderbat-serviceaccount --region="<region>" --cluster "<cluster_name>" --attach-policy-arn "<policy_arn>" --approve --namespace spyderbat

eksctl get iamserviceaccount --name spyderbat-serviceaccount --region="<region>" --cluster "<cluster_name>" --namespace spyderbat
```

### Modifying Helm Chart to Query AWS Secrets Manager

Now that you have all those values, you can run a Helm chart install of the Nano Agent to reference that secret and mount it accordingly. You could utilize your own custom values.yaml file or by overriding via --set in the Helm CLI:

```
aws:
    secretsmanager:
        enabled: true
        rolearn: "<role_arn>"
        secretarn: "<secret_arn>"
```

```
helm repo add nanoagent https://spyderbat.github.io/nanoagent_helm/
helm repo update
helm install nanoagent nanoagent/nanoagent \  --set nanoagent.orcurl="<orc_url>" \  --namespace spyderbat \  --create-namespace \  --set CLUSTER_NAME="<cluster_name>"
```

The steps above represent one of the ways this task could be accomplished. If you have any questions feel free to contact us at <support@spyderbat.com>.


# Create a Golden Image with the Nano Agent Pre-Installed

Template Spyderbat Nano Agent install via Golden Image for environments with auto-scaling and automation requirements.

<mark style="color:blue;">Published: July 25, 2022</mark>

Using a Golden Image of a virtual machine (VM) to deploy consistently creates a template that reduces errors, ensures consistency, and lowers the level of effort during deployment. The use of a Golden Image is also common in environments with autoscaling and automation.

Below are the steps that should be followed to include the Spyderbat Nano Agent into your Golden Image:

1\. Identify the VM that you want to base your Golden Image.

2\. Install the Spyderbat Nano Agent on this VM by choosing to add a new source in the Sources section of the UI.

<figure><img src="/files/X0t0aGTvjtX35y8bfCHS" alt=""><figcaption></figcaption></figure>

3\. After you have successfully installed and registered the Spyderbat Nano Agent, run the following command to stop the Nano Agent (it will take a few seconds to fully stop):

```
sudo systemctl stop nano_agent.service
```

4\. Remove the unique machine ID (MUID) that associates the Nano Agent with the specific VM it is running on, by executing the following command:

```
sudo rm /opt/spyderbat/etc/muid
```

5\. Save this VM as your Golden Image using the respective Cloud Platform or Virtual Machine functionality.

6\. The Nano Agent service will start automatically when the virtual machine is loaded and boots, at which time a new, unique MUID will be generated and associated with the specific VM.

See also[ How to perform an unattended Spyderbat Nano Agent installation on AWS](/installation/spyderbat-nano-agent/aws-unattended-install) for additional information.


# Spyderbat AWS Agent

Overview of the AWS Agent, deployment options and how to get started

The Spyderbat AWS Agent enables AWS Context Integration in the Spyderbat Platform. This integration provides a comprehensive view of cloud assets and IAM configurations, enhancing the ability to detect and investigate potential security incidents.

For an overview of the AWS Agent and its role in the Spyderbat Platform, refer to the [AWS Context integration page in the integration concepts section](/concepts/integrations/aws).

## Permissions Required by the Spyderbat AWS Agent

To function effectively, the Spyderbat AWS Agent requires specific permissions to collect data from AWS APIs. Below are the key permissions grouped by AWS services:

* **EC2**:
  * `ec2:Describe*`
* **EKS**:
  * `eks:List*`
  * `eks:Describe*`
* **IAM Roles and Policies**:
  * `iam:Get*`
  * `iam:List*`
  * `iam:Put*`
* **STS (Security Token Service)**:
  * `sts:AssumeRole`
  * `sts:AssumeRoleWithWebIdentity`

The agent also supports consuming configured secrets (registration key) in AWS Secrets Manager - which would require an extra permission to access the configured secret arn. &#x20;

Permissions are configured using a custom AWS policy attached to the IAM Role that the AWS Agent assumes. How the agent assumes this role depends on the deployment options and is discussed in the more detailed deployment guides.&#x20;

## Deployment Options for the AWS Agent

Spyderbat offers multiple deployment options for the AWS Agent to suit different environments and requirements. Below are the currently available deployment methods:

1. **Hosted on an AWS VM**: You can deploy the AWS Agent on a virtual machine within your AWS account. This option gives you full control over the agent and its environment.
2. **Hosted on a Kubernetes Cluster**: The AWS Agent can be deployed as a Kubernetes pod within a cluster. This is suitable for users who want to integrate AWS context alongside their Kubernetes workloads.

For detailed installation instructions for each deployment option, refer to the respective guides:

* [AWS VM Deployment Guide](/installation/spyderbat-aws-agent/aws-agent-vm)
* [Kubernetes Deployment Guide](/installation/spyderbat-aws-agent/aws-agent-helm)

## Getting Started with the AWS Agent

To begin using the Spyderbat AWS Agent:

1. **Choose a Deployment Method**: Decide whether to deploy the agent on an AWS VM, a Kubernetes cluster, or use the hosted option.
2. **Deploy the Agent**: Follow the instructions in the relevant deployment guide to deploy the AWS Agent.

Once deployed, the agent will start collecting cloud context and feeding it to the Spyderbat Platform, where it can be used for enhanced visibility, detection, and investigation.


# AWS Linux VM

Automatic installation of the Spyderbat AWS Agent on an AWS EC2 instance

This guide provides detailed instructions on how to deploy the Spyderbat AWS Agent on an AWS Virtual Machine (VM). The AWS Agent runs as a systemd service on the VM to ensure continuous operation. Deploying the agent on an AWS VM gives you complete control over the environment and configuration, allowing you to integrate AWS context into the Spyderbat Platform.

## Prerequisites

Before deploying the Spyderbat AWS Agent on an AWS VM, make sure you have the following prerequisites in place:

1. **sudo permissions**: You will need sudo permissions to install the Spyderbat AWS Agent.
2. **Outbound Network Access**: The system you’re installing Spyderbat's AWS Agent on should have outbound access on port 443 to `https://orc.spyderbat.com`.
3. **AWS Account**: You need an AWS account with administrative access to create and configure resources. The VM must be launched within the AWS account that you wish to monitor.
4. **VM Instance Profile with Required IAM Role**: The VM must have an instance profile attached that includes an IAM Role with the following permissions:
   * **EC2**: `ec2:Describe*`
   * **EKS**: `eks:List*`, `eks:Describe*`
   * **IAM Roles and Policies**: `iam:Get*`, `iam:List*`, `iam:Put*`
   * **ECR**: `ecr:Describe*`, `ecr:List*`, `ecr:Get*`
   * **STS**: `sts:AssumeRole`, `sts:AssumeRoleWithWebIdentity`
   * **Secrets Manager** (Optional): Access to the ARN of the configured secret for the registration key.

Here is an example permissions policy that can be used when creating the role

```json
{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": [
				"ec2:Describe*",
				"eks:List*",
				"eks:Describe*",
				"organizations:ListDelegatedAdministrators",
				"organizations:DescribeOrganization",
				"organizations:DescribeOrganizationalUnit",
				"organizations:DescribeAccount",
				"organizations:ListAWSServiceAccessForOrganization",
				"iam:Get*",
				"iam:List*",
 				"iam:Put*",
				"ecr:Describe*",
				"ecr:List*",
				"ecr:Get*"
			],
			"Resource": "*"
		}
	]
}
```

## Step-by-Step Deployment

### Step 1: Launch an AWS VM

Launch an AWS VM within the AWS account you wish to monitor. The instance should be configured with the following settings:

* **Amazon Machine Image (AMI)**: Use an AMI that supports Linux (e.g., Amazon Linux 2, Ubuntu).
* **Instance Type**: Choose an instance type suitable for your workload (e.g., `t3.medium`).
* **Network Settings**: Ensure the instance has access to the internet or appropriate VPC configuration for accessing AWS APIs.
* **IAM Role**: Attach the IAM Role created earlier with the required permissions.
* Configure storage and other instance details as needed.

### Step 2: Connect to the VM and Install Dependencies

1. Install Docker by following the official [Docker installation guide](https://docs.docker.com/get-docker/).

### Step 3: Install the Spyderbat AWS Agent

* Log in to the Spyderbat UI
* Navigate to the **Sources** menu (top left)
* Click on the **Add Source** button, and select **Install AWS Agent**

![Add AWS Agent Source](/files/aFgtLwEfteVGsnD8C3UD)

This will bring you to the following screen:

![AWS VM Curl Install](/files/pTmS5z8BsVWzIPMt8rM9)

The agent installation command is obtained from the Spyderbat UI that you can execute on the VM. Click on the tab 'curl' there, and then the command below will be provided that you can paste. If you do not have Curl installed on your system, select the 'wget' tab to copy this command instead. Then use that in the VM to install the agent.

Here's how the curl command will look like

```sh
curl --retry 5 https://orc.spyderbat.com/v1/reg/<registration-key>/script?agentType=aws_agent -o installSpyderbatAws.sh
sudo -E /bin/sh ./installSpyderbatAws.sh
```

Now execute this script on the AWS VM.

### Step 4: Verify Integration

The CLI and UI both provide you with feedback on the process. In the UI, check marks of the install progress will be displayed. Once the Spyderbat AWS Agent is installed, registers with Spyderbat, and is transmitting data, you will see that the agent was installed successfully both in your terminal and in the Spyderbat UI.

## Managing the AWS Agent Service

The Spyderbat AWS Agent runs as a systemd service (`aws_agent.service`) on the VM. You can use the following commands to manage the AWS Agent service:

* **Check Service Status**:

  ```sh
  sudo systemctl status aws_agent.service
  ```
* **Start the Service**:

  ```sh
  sudo systemctl start aws_agent.service
  ```
* **Stop the Service**:

  ```sh
  sudo systemctl stop aws_agent.service
  ```
* **Restart the Service**:

  ```sh
  sudo systemctl restart aws_agent.service
  ```
* **View Service Logs**:

  ```sh
  sudo journalctl -u aws_agent.service
  ```

## Troubleshooting

* **Agent Logs**: Check the agent logs using the following command:

  ```sh
  sudo journalctl -u aws_agent.service
  ```
* **Permission Issues**: Ensure the IAM Role attached to the VM has the correct permissions as listed in the prerequisites.
* **Network Connectivity**: Verify that the VM has access to the internet or the required VPC endpoints to communicate with AWS services.

## Next Steps

* Once the AWS Agent is successfully deployed and integrated, you can proceed to use the spyderbat platform to monitor and investigate your assets.
* The AWS Agents behavior can be customized using a configuration file. For more details on advanced configuration of the agent, consult they [Spyderbat AWS Agent Configuration Guide](/installation/spyderbat-aws-agent/aws-agent-vm-configuration)


# Kubernetes

Automatic installation of the Spyderbat AWS Agent on a Kubernetes Cluster

This guide provides detailed instructions on how to deploy the Spyderbat AWS Agent on a Kubernetes cluster. In a cluster deployment, the AWS Agent runs as a single-pod StatefulSet.

The most straightforward option is to run the AWS Agent on an EKS cluster within the AWS account that you want to monitor. In this case, the required configuration is minimal, and the deployment process is streamlined. This scenario is described in this guide.

More advanced configurations are also possible:

* The agent can poll information from another AWS account, or you can deploy a Helm chart to monitor multiple AWS accounts within a single Helm installation. This requires configuring additional IAM roles to be assumed.
* The Spyderbat registration key can be managed in AWS Secrets Manager if your organization uses Secrets Manager as the standard secret management technology for cluster workloads.

For more details on these configurations, please consult the [AWS Agent Configuration Guide for Helm](/installation/spyderbat-aws-agent/aws-agent-helm-configuration).

## Prerequisites

Before deploying the Spyderbat AWS Agent on an AWS EKS cluster, ensure you have the following prerequisites in place:

1. **Outbound Network Access**: The cluster you’re installing Spyderbat's AWS Agent on must have outbound access on port 443 to `https://orc.spyderbat.com`.
2. **Kubectl and Helm**: Install Kubectl and Helm clients, and configure Kubectl for the cluster where you want to install the agent.
3. **AWS Account**: The cluster the agent is deployed on must reside in the AWS account that you wish to monitor.
4. **IAM Role**: Create an IAM Role that will be associated with the service account used by the AWS Agent.

   * **Role Permissions**: The role must have the following permissions attached:
     * **EC2**: `ec2:Describe*`
     * **EKS**: `eks:List*`, `eks:Describe*`
     * **IAM Roles and Policies**: `iam:Get*`, `iam:List*`, `iam:Put*`
     * **STS**: `sts:AssumeRole`, `sts:AssumeRoleWithWebIdentity`
     * **Secrets Manager** (Optional): Access to the ARN of the configured secret for the registration key.
   * **Role Trust Policy**: The IAM Role for the Spyderbat AWS Agent requires a trust policy that allows the Kubernetes Service Account associated with the AWS Agent to assume the role. Below is the trust policy:

   ```json
   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Principal": {
           "Federated": "arn:aws:iam::<account-id>:oidc-provider/oidc.eks.<region>.amazonaws.com/id/<open-id-provider-id>"
         },
         "Action": "sts:AssumeRoleWithWebIdentity",
         "Condition": {
           "StringLike": {
             "oidc.eks.<region>.amazonaws.com/id/<open-id-provider-id>:aud": "sts.amazonaws.com",
             "oidc.eks.<region>.amazonaws.com/id/<open-id-provider-id>:sub": "system:serviceaccount:*:aws-agent"
           }
         }
       }
     ]
   }
   ```

   Note that `<account-id>`, `<region>`, and `<open-id-provider-id>` are dependent on your local deployment of the EKS cluster. Take note of the ARN of this role, as it will be an input for the Helm chart deployment.

   You do not need to create the Kubernetes service account associated with the role, as the Helm chart installation will handle that.

## Installation with AWS Agent Helm Chart

### Step 1 - Copy the Helm install command from the Spyderbat UI

* Log in to the Spyderbat UI
* Navigate to the **Sources** menu (top left)
* Click on the **Add Source** button, and select **Install AWS Agent**

![Add AWS Agent Source](/files/aFgtLwEfteVGsnD8C3UD)

This will bring you to the following screen where you can click on the **Helm** tab to select installation using the Helm chart.

![AWS Helm Install](/files/NnY0bvJ8gV7HD3YaKnJ4)

In the input fields, enter the following:

* **Cluster Name**: The name of the cluster you are deploying to. This will help the AWS Agent associate itself with the cluster and facilitate recognition in the Cluster Health and Sources UI. This is not required but recommended.
* **IAM Role ARN**: Enter the ARN of the role you created earlier. This is a required field.

Upon entering the information, the UI will generate a command that you can use to start the installation. Copy the command. It will be similar to the following (your registration key will differ):

```bash
helm repo add awsagent https://spyderbat.github.io/aws_agent_helmchart/
helm repo update
helm install awsagent awsagent/awsagent \
  --set credentials.spyderbat_registration_key=<registrationKey> \
  --set spyderbat_orc_url=https://orc.kangaroobat.net \
  --set serviceAccount.awsRoleArn=<AWS IAM Role ARN> \
  --namespace spyderbat \
  --create-namespace \
  --set CLUSTER_NAME=<cluster-name>
```

### Step 2 - Run the Helm Command

In your command-line shell, with Kubectl and Helm installed and configured to use the target cluster as the active context, paste the copied Helm command.

### Step 3 - Validate the Installation

Check for any reported errors during the installation, and use the following command to validate that `awsagent` is installed:

```bash
helm list
```

Then use:

```bash
kubectl get all -n spyderbat
```

You should see a StatefulSet named `awsagent-auto` and an associated pod named `awsagent-auto-0` running if the installation was successful.

To check the logs of the AWS Agent pod, use:

```bash
kubectl logs pod/awsagent-auto-0
```

## Uninstalling the AWS Agent from Your Cluster

To remove the AWS Agent, use Helm uninstall:

```bash
helm list -n spyderbat
helm uninstall awsagent -n spyderbat
```

## Advanced Configuration

There are various settings that can be customized to address specific needs. These can be achieved by using a custom `values.yaml` file or by using the `--set` option during Helm installation.

For more details on these settings, please consult the [AWS Agent Configuration Guide for Helm](/installation/spyderbat-aws-agent/aws-agent-helm-configuration).


# Configuration Guide - AWS Linux VM

Detailed configuration guide for the Spyderbat AWS Agent installed on an AWS VM

This guide explains how to configure the Spyderbat AWS Agent to collect information from an AWS account and send it to the Spyderbat platform. It provides detailed instructions for locating the configuration file, managing AWS credentials, and configuring all available settings.

***

## 1. Managing the configuration

The Spyderbat AWS Agent's configuration file is a YAML file named `aws-agent.yaml`. It is used to control the behavior of the agent, such as which AWS services to monitor, where to send data, and how to manage credentials.

### Locating the Configuration File

By default, the configuration file is located at:

```
/opt/spyderbat/etc/aws-agent.yaml
```

This file can be edited using any text editor with root privileges. For example:

```bash
sudo vi /opt/spyderbat/etc/aws-agent.yaml
```

### Applying Changes

After making changes to the configuration file, the AWS Agent service must be restarted to apply the updates. Use the following command to restart the service:

```bash
sudo systemctl restart aws_agent.service
```

### Validating Configuration

To ensure the configuration file is valid, check the service status after restarting:

```bash
sudo systemctl status aws_agent.service
aws_agent.service - Spyderbat AWS Agent Service
     Loaded: loaded (/etc/systemd/system/aws_agent.service; enabled; preset: disabled)
     Active: active (running) since Wed 2024-12-11 18:48:48 UTC; 3 weeks 6 days ago
   Main PID: 2146512 (aws_agent)
      Tasks: 8 (limit: 1112)
     Memory: 10.1M
        CPU: 4min 7.715s
     CGroup: /system.slice/aws_agent.service
             ├─2146512 /usr/bin/bash /opt/spyderbat/bin/aws_agent
             └─2146528 docker run --pull always -v /opt/spyderbat/etc:/etc/aws-config --name aws-agent public.ecr.aws/a6j2k0g1/aws-agent:latest --config /etc/aws->

Jan 08 12:24:30 ip-172-31-86-31.ec2.internal aws_agent[2146528]:  2025-01-08 12:24:30,479:INFO    :poller eks/us-west-1 got 2 records
Jan 08 12:24:30 ip-172-31-86-31.ec2.internal aws_agent[2146528]:  2025-01-08 12:24:30,755:INFO    :Sending heartbeat and stat update
Jan 08 12:24:31 ip-172-31-86-31.ec2.internal aws_agent[2146528]:  2025-01-08 12:24:31,346:INFO    :Session(region_name=None) IAM Poller got 56 roles and their inl>
```

If there are any errors, they will be displayed in the status output. Ensure the YAML syntax is correct before restarting the service again.

### Checking agent logs

The agent logs can be consulted based on the service journal:

```bash
sudo journalctl -u aws_agent.service
```

***

## 2. AWS Credentials Management

The Spyderbat AWS Agent requires access to AWS services to collect data. The agent supports multiple methods for obtaining credentials, listed below in the order of precedence:

### 1. **IAM Instance Profile (Recommended)**

If the agent is deployed on an EC2 instance with an IAM role assigned, it will automatically use the instance profile credentials. This is the most secure and recommended method. No additional configuration is required for this setup.

### 2. **Environment Variables**

You can set the following environment variables to provide credentials explicitly:

* `AWS_ACCESS_KEY_ID`
* `AWS_SECRET_ACCESS_KEY`

For example, add the variables to the environment:

```bash
export AWS_ACCESS_KEY_ID=<your_access_key_id>
export AWS_SECRET_ACCESS_KEY=<your_secret_access_key>
```

### 3. **From Files**

The agent can also read credentials from files. This is typically used when credentials are mounted as secrets in Kubernetes or other containerized environments. Place the credentials in the following files:

* `/etc/aws-config/secrets/aws_access_key_id`
* `/etc/aws-config/secrets/aws_secret_access_key`

**Note**: This method is not recommended for standalone deployments.

***

## 3. Configuration Settings

Below is a detailed explanation of each configuration setting available in the `aws-agent.yaml` file.

### `spyderbat_orc_url`

* **Description**: The URL of the Spyderbat orchestration API endpoint. This is where the agent sends the collected data.
* **Example**:

  ```yaml
  spyderbat_orc_url: https://orc.spyderbat.com
  ```
* **Default**: `https://orc.spyderbat.com`

***

### `outfile`

* **Description**: Specifies a file where the agent writes the collected data instead of sending it to the Spyderbat backend. This is primarily for debugging purposes.
* **Example**:

  ```yaml
  outfile: /tmp/out.json.gz
  ```
* **Default**: Not set.

***

### `cluster_name`

* **Description**: The name of the Kubernetes cluster, used for identification in the Spyderbat UI. This is optional for standalone deployments.
* **Example**:

  ```yaml
  cluster_name: staging-cluster-us-east-1
  ```
* **Default**: Not set.

***

### `aws_account_id`

* **Description**: Specifies the AWS account ID the agent monitors. Use `auto` for auto-discovery.
* **Example**:

  ```yaml
  aws_account_id: auto
  ```
* **Default**: `auto`

***

### `role_arn`

* **Description**: The ARN of the IAM role the agent assumes to gather information. This is useful when explicit AWS credentials are used. It should not be used if the correct role was already assumed through an EC2 IAM Instance Profile.
* **Example**:

  ```yaml
  role_arn: arn:aws:iam::123456789012:role/SpyderbatRole
  ```
* **Default**: Not set.

***

### `send_buffer_size`

* **Description**: The number of records accumulated before sending data to the Spyderbat backend.
* **Example**:

  ```yaml
  send_buffer_size: 100
  ```
* **Default**: `100`

***

### `send_buffer_records_bytes`

* **Description**: The maximum size (in bytes) of accumulated records before sending to the backend.
* **Example**:

  ```yaml
  send_buffer_records_bytes: 1000000
  ```
* **Default**: `1000000` (1 MB)

***

### `send_buffer_max_delay`

* **Description**: The maximum delay (in seconds) before sending accumulated records, even if the buffer is not full.
* **Example**:

  ```yaml
  send_buffer_max_delay: 30
  ```
* **Default**: `30`

***

### `log_level`

* **Description**: Configures the logging level for the agent.
* **Options**: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`
* **Example**:

  ```yaml
  log_level: INFO
  ```
* **Default**: `INFO`

***

### `pollers`

* **Description**: Configures the AWS services and regions to monitor. Each entry specifies a service, polling interval, and regions.
* **Example**:

  ```yaml
  pollers:
    - service: ec2
      polling_interval: 30
      regions:
        - us-east-1
        - us-west-2
    - service: eks
      polling_interval: 30
      regions:
        - us-east-1
        - us-east-2
  ```
* **Default**: Monitors all supported services and regions if not set explicitly.

Per service in the pollers section, the following properties can be set:

### `polling_interval`

* **Description**: The interval in seconds at which the agent will poll the service.
* **Example**:

  ```yaml
    - service: eks
      polling_interval: 30
  ```
* **Default**: 30

### `regions`

* **Description**: The regions that the agent will poll for the service. If not set, the agent will poll all regions.
* **Example**:

  ```yaml
    - service: eks
      regions:
        - us-east-1
        - us-east-2
  ```
* **Default**: not set (all regions)

## Example configuration file

You can find an example illustrated configuration yaml file [here](https://github.com/spyderbat/spyderbat-docs/tree/main/installation/spyderbat-aws-agent/aws-agent-example.yaml)


# Configuration Guide - Kubernetes

This guide provides a detailed explanation of the various configuration options available in the YAML configuration file for the Spyderbat AWS Agent. The configuration file allows you to control aspects of the agent's behavior, such as polling, AWS account details, and integration settings.

## Configuration Parameters Overview

Below are key configuration parameters that can be set in the `values.yaml` file of the Helm chart for the Spyderbat AWS Agent:

### Credentials

#### 1. `awsSecretsManager`

* **Description**: Configures AWS Secrets Manager integration to store the Spyderbat registration key.
* **Fields**:
  * **`enabled`**: Whether to use AWS Secrets Manager for storing the registration key.
  * **`secretArn`**: The ARN of the secret in AWS Secrets Manager containing the Spyderbat registration key.
* **Example**:

  ```yaml
  awsSecretsManager:
    enabled: false
    secretArn: <arn of the secret in secrets manager>
  ```

#### 2. `credentials`

* **Description**: Configures AWS credentials and the Spyderbat registration key.
* **Fields**:
  * **`aws_access_key_id`**: AWS access key ID (optional).
  * **`aws_secret_access_key`**: AWS secret access key (optional).
  * **`spyderbat_registration_key`**: The Spyderbat registration key.
* **Example**:

  ```yaml
  credentials:
    spyderbat_registration_key: <spyderbat registration key>
  ```

## Spyderbat Configuration Parameters

### 1. `spyderbat_orc_url`

* **Description**: URL for the Spyderbat orchestration endpoint, used by the agent to communicate with Spyderbat's backend.
* **Example**:

  ```yaml
  spyderbat_orc_url: https://orc.spyderbat.com
  ```

### 2. `cluster_name`

* **Description**: Specifies the cluster name where the AWS Agent is running. This helps in identifying the data source in the Spyderbat UI.
* **Example**:

  ```yaml
  cluster_name: my-cluster
  ```

### 3. `awsAgentsConfigs`

* **Description**: Configures the AWS accounts and services that the agent will monitor.
* **Fields**:
  * **`aws_account_id`**: AWS account ID to monitor. Set to `auto` to auto-discover the account ID.
* **Example**:

  ```yaml
  awsAgentsConfigs:
    - aws_account_id: auto
  ```

The helm chart can install one or more aws agents (one per account to be monitored). The default installation installs a single agent. You can configure multiple agents by providing multiple sections under the awsAgentsConfigs section.

For each of the section, the configuration options are the same as described in the configuration guide for the single-vm install, which you can consult [here](/installation/spyderbat-aws-agent/aws-agent-vm-configuration#config-settings).

## Managing Configuration

* **Updating Configuration**: To update the configuration, modify the `values.yaml` file and upgrade the release using:

  ```sh
  helm upgrade aws-agent spyderbat/aws-agent -f values.yaml
  ```
* **Validation**: Ensure to validate the syntax of the `values.yaml` before applying changes to avoid runtime issues.


# Install Spyctl CLI

Spyctl, an open source CLI tool, allows you to view and manage resources within your Spyderbat environment.

Source code: <https://github.com/spyderbat/spyctl-cli>

The initial step in utilizing any software package is ensuring its correct installation, so let's get started by walking through the installation process for Spyctl.

### Prerequisites

* Python 3.8 or newer

### Installation Command

{% tabs %}
{% tab title="Debian/Ubuntu" %}
To install spyctl globally requires the pipx utility.

```bash
sudo apt install pipx
```

Install spyctl using pipx.

```bash
pipx install spyctl
```

Verify the installation.

```bash
spyctl --version
```

Alternatively, you can use a virtual environment to install spyctl.

```bash
python -m venv spyctl
source spyctl/bin/activate
pip install spyctl
```

Verify the installation.

```bash
spyctl --version
```

{% hint style="info" %}
Note: depending on your system, you may need to use `python3` or instead of `python`. If you go the virtual environment route, you may need to install virtualenv first.

```
apt install python3.X-venv
```

Where `python3.X` is the version of python you have installed.
{% endhint %}
{% endtab %}

{% tab title="Windows/MacOS/Other Linux" %}
To install Spyctl, simply run this command in your terminal of choice:

```bash
pip install spyctl
```

To verify the installation:

<pre class="language-bash" data-full-width="false"><code class="lang-bash"><strong>spyctl --version
</strong></code></pre>

{% endtab %}
{% endtabs %}

### Enabling Shell Completion

To enable shell completion, follow these steps:

{% tabs %}
{% tab title="Bash" %}
{% hint style="info" %}
The default version of Bash for Mac OS X users does not support programmable shell completion. Guides like [this](https://kizu514.com/blog/install-upgraded-gnu-bash-on-a-macbook-pro/) will help you install a newer version of Bash.
{% endhint %}

Create the Spyctl directory if you haven’t already.

```bash
mkdir -p ~/.spyctl
```

Generate the shell completion script.

```bash
_SPYCTL_COMPLETE=bash_source spyctl > ~/.spyctl/spyctl-complete.bash
```

Add the following line to the end of `~/.bashrc`.

```
. ~/.spyctl/spyctl-complete.bash
```

{% endtab %}

{% tab title="Fish" %}
Generate and save the shell completion script.

```python
_SPYCTL_COMPLETE=fish_source spyctl > ~/.config/fish/completions/spyctl-complete.fish
```

{% endtab %}

{% tab title="Zsh" %}
Create the Spyctl directory if you haven’t already.

```sh
mkdir -p ~/.spyctl
```

Generate the shell completion script.

```bash
_SPYCTL_COMPLETE=zsh_source spyctl > ~/.spyctl/spyctl-complete.zsh
```

Add the following line to the end of `~/.zshrc`.

```python
. ~/.spyctl/spyctl-complete.zsh
```

{% endtab %}
{% endtabs %}

After modifying the shell config, you need to start a new shell in order for the changes to be loaded.


# Initial Configuration

Learn how to quickly configure Spyctl to interact with your Spyderbat data

### Prerequisites

* [Install Spyctl](/installation/spyctl)
* [Generate a key to access the Spyderbat API](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api)

### Initial Configuration

In this section you will learn how to configure Spyctl to enable data retrieval from across your entire organization. To do so, you must first create an APISecret and then use that APISecret to set a Context. An APISecret encapsulates your Spyderbat API credentials; the Context specifies where Spyctl should look for data when interacting with the Spyderbat API (e.g., organization, cluster, machine, service, or container image).

#### Create an APISecret

An APISecret encapsulates your Spyderbat API credentials. You must create at least one APISecret in order for Spyctl to access your data via theSpyderbat API.

To create an APISecret, use an [API key generated from the Spyderbat Console](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api).

| Region             | API URL                              |
| ------------------ | ------------------------------------ |
| United States      | <https://api.spyderbat.com>          |
| Mumbai, India      | <https://api.mum.prod.spyderbat.com> |
| Frankfurt, Germany | <https://api.deu.prod.spyderbat.com> |

{% hint style="info" %}
For most users, the API URL will be the one in the United States. If you are unsure which one applies to you, contact <support@spyderbat.com>.
{% endhint %}

Copy a generated API key and region-specific API URL into the following command:

```
spyctl config set-apisecret -k <apikey> -u <apiurl> NAME
```

For example:

```
$ spyctl config set-apisecret -k ZXlKaGJHY2lPaUpJVXpJMU5pSXNJbXRwWkNJNkluTm\
lJaXdpZEhsd0lqb2lTbGRVSW4wLmV5SmxlSEFpT2pFM01EUTVPVGM1TWpBc0ltbGhkQ0k2TVRZM\
016UTJNVGt4T1N3aWFYTnpJam9pYTJGdVoyRnliMjlpWVhRdWJtVjBJaXdpYzNWaUlqb2ljSGhX\
YjBwMlVFeElXakJIY1VJd2RXMTNTMEVpZlEuZGpxWkRCOTNuUnB4RUF0UU4yQ0ZrOU5zblQ5Z2Q\
tN0tYT081TEZBZC1GSQ== -u "https://api.spyderbat.com" my_secret

Set new apisecret 'my_secret' in '/home/demouser/.spyctl/.secrets/secrets'
```

{% hint style="info" %}
Spyctl saves APISecrets in *$HOME/.spyctl/.secrets/secrets*
{% endhint %}

#### Set a Context

Contexts will let Spyctl know where to look for data. The broadest possible Context is organization-wide. This means that when you run Spyctl commands, the Spyderbat API will return results relevant to your entire organization.

{% hint style="info" %}
For the `--org` field in the following command you may supply the name of your organization which can be found in the top right of the Spyderbat Console or the organization UID which can be found in your web browser's url when logged into the [Spyderbat Console](https://app.spyderbat.com): `https://app.spyderbat.com/app/org/UID/dashboard`.
{% endhint %}

```
spyctl config set-context --org <ORG NAME or UID> --secret <SECRET NAME> NAME
```

For example:

```
$ spyctl config set-context --org "John's Org" --secret my_secret my_context
Set new context 'my_context' in configuration file '/home/demouser/.spyctl/config'.
```

You can view your configuration by issuing the following command:

```
spyctl config view
```

You should see something like this:

```
apiVersion: spyderbat/v1
kind: Config
contexts:
- name: my_context
  secret: my_secret
  context:
    organization: John's Org
current-context: my_context
```

{% hint style="info" %}
The global configuration file is located at *$HOME/.spyctl/config*
{% endhint %}

{% hint style="info" %}
It is possible to create more specific contexts, such as a group of machines or a specific container image. You can think of the fields in your context as filters to limit your scope. Follow this link to learn more about contexts: Contexts
{% endhint %}

At this point you should now be able to run spyctl commands that utilize the Spyderbat API.


# Install Spydertop CLI

Learn how Spyderbat leverages kernel-level system monitoring and public APIs to expand HTOP functionality to allow analysts to look into system anomalies days or even months later.

<mark style="color:blue;">Published: September 13, 2022</mark>

### HTOP’s Strengths and Shortcomings

There is a program called “top” on most Linux systems for simple system monitoring. The tool lists the CPU and memory usage for the computer and each process, just as Task Manager does on Windows. [HTOP](https://htop.dev/) is a more advanced and user-friendly version of top, displaying graphs in addition to raw values and adding colors for readability. Both programs are widely used for monitoring Linux systems, allowing administrators to track processes’ resource usage or quickly get a list of running tasks.

These tools are only designed to show the state of the system at the current moment. They lack the ability to record and display information even over the last few seconds as Task Manager does. This limitation is understandable since neither program is designed to log system performance or give a historical understanding of the machine. But what if the behavior you want to profile is intermittent, and you cannot be on the machine to run top when it happens?

### Enter Spydertop

Spydertop is an open-source tool developed by Spyderbat that provides a solution for this currently unfulfilled use case. Utilizing Spyderbat's kernel-level system monitoring and public APIs, it provides the same in-depth information as HTOP, and extends these abilities historically. Spydertop allows analysts to look into system anomalies days or even months after they occur.

To get started, navigate to **Spydertop** in the left-hand navigation and select a source to explore.

<figure><img src="/files/1c8k1O6he66ZrpVYGAWp" alt="Spydertop source selection page listing available machines with Explore buttons"><figcaption><p>Select a source to explore in Spydertop.</p></figcaption></figure>

### How it works

Imagine a Kubernetes node that has the Spydertop Nano Agent installed. The agent collects the data necessary for Spydertop to function; for more details, refer to the [Installation Guide](/installation/spyderbat-nano-agent) or watch [this video](https://youtu.be/z6mGLjqysCk)

on how to get it installed. On this system, there happens to be an application with a bug that causes it to continuously use up more memory. At 2:00 in the morning, the container reaches its memory limit and automatic safeguards restart the application. It begins to function correctly afterward, showing no signs of excessive memory usage.

In the morning, an analyst sees the crash report and decides to investigate. They start Spydertop on their own machine, and it uses Spyderbat's public API to collect all the resource usage records from that early morning crash, as well as the active processes, connections, and more. Using these records, Spydertop displays the memory usage of the machine: 95% at 1:30 AM. By stepping through time, the analyst sees the memory slowly increase until the crash. Next, they sort the running processes by memory usage, find the buggy application, and can now resolve the issue.

<figure><img src="/files/aZTtsoCBbj9E8LhBI2Zj" alt="Spydertop process view showing CPU, memory, and disk usage with a sortable process table"><figcaption><p>Spydertop displays CPU, memory, disk, and network usage alongside a detailed process table. Use the timeline at the bottom to step through time.</p></figcaption></figure>

### How to use Spydertop

You can try out Spydertop by checking out the [public repository](https://github.com/spyderbat/spydertop) or running the [docker image](https://hub.docker.com/r/spyderbat/spydertop). If you don’t have an API key yet, it will guide you through setting one up. After that, it is as simple as picking a machine and what time to investigate (both of which can be passed as command-line options for convenience).

Once the necessary data has been loaded from the API, Spydertop presents a simple CLI interface. Spydertop aims to make the transition easy for users already accustomed to HTOP, so the user interface, buttons, and keyboard shortcuts are designed to be similar.

The first few lines display machine-wide resource usage information, such as CPU core usage and disk reads and writes. Taking up the rest of the screen is the process table, which shows resource usage and details for individual processes. Several other tabs are available in this table — **Process**, **System**, **Connection**, **Red Flags**, **Listening**, and **Container** — to show different views of the data. The timeline at the bottom lets you navigate to any point in time, with playback speed controls (1x, 5x, 15x, 60x).

Get started using Spydertop for free by installing the python CLI, or try it without an account by running the docker image with a set of example data:

```bash
docker run -it spyderbat/spydertop -i examples/minikube-sock-shop.json.gz
```

Press `F1` or `h` within the CLI version of Spydertop to access the built-in help menu with a full list of keyboard shortcuts and commands.


# Install the Spyderbat Event Forwarder

Install the Spyderbat Event Forwarder to deliver SIEM-forwarded events from Spyderbat to your SIEM, Splunk, or any HTTP endpoint.

{% hint style="warning" %}
The Event Forwarder requires SIEM forwarding to be enabled on at least one saved query before it will receive events. See [SIEM Forwarding](/concepts/integrations/siem-forwarding) to set up the control plane first.
{% endhint %}

The Spyderbat Event Forwarder is an open-source binary ([github.com/spyderbat/event-forwarder](https://github.com/spyderbat/event-forwarder)) that polls the Spyderbat SIEM API and delivers events to your destination. It supports files, stdout, syslog, and HTTP webhooks.

## Deployment options

| Deployment            | When to use                         | Guide                                                                                  |
| --------------------- | ----------------------------------- | -------------------------------------------------------------------------------------- |
| Helm chart            | Kubernetes environments             | [Helm Chart](/installation/spyderbat-event-forwarder/helm-chart)                       |
| Traditional installer | Linux systemd (VM, bare metal, EC2) | [Traditional Installer](/installation/spyderbat-event-forwarder/traditional-installer) |

Run only one Event Forwarder instance per organization. Multiple instances each deliver the full event stream, producing duplicates.

For architecture details, see [Spyderbat Event Forwarder](/concepts/integrations/spyderbat-event-forwarder).


# Helm Chart

Install the Spyderbat Event Forwarder on Kubernetes using the Helm chart.

{% hint style="warning" %}
Before installing the Event Forwarder, enable SIEM forwarding on at least one saved query in the console. Without this step, the forwarder polls the API and receives nothing. See [SIEM Forwarding](/concepts/integrations/siem-forwarding).
{% endhint %}

The Helm chart deploys the Event Forwarder with persistent storage, so it resumes where it left off after restarts or pod rescheduling.

{% hint style="warning" %}
Run only one Event Forwarder instance per organization. Multiple instances each receive the full event stream independently and will cause duplicate events in your SIEM.
{% endhint %}

## Prerequisites

* A Kubernetes cluster with Helm installed
* Spyderbat Nano Agents deployed on the hosts you want to monitor
* Your org UID and a Spyderbat API key (see [API Key Setup](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api))
* SIEM forwarding enabled on at least one saved query (see [SIEM Forwarding](/concepts/integrations/siem-forwarding))

## Install

Clone the event-forwarder repository and install the chart:

```bash
git clone https://github.com/spyderbat/event-forwarder.git
cd event-forwarder/helm-chart/event-forwarder
helm install <release-name> . \
  --namespace spyderbat \
  --set spyderbat.spyderbat_org_uid=YOUR-ORG-UID \
  --set spyderbat.spyderbat_secret_api_key=YOUR-API-KEY \
  --create-namespace
```

### Helm values

| Value                                | Description                          | Default                  | Required |
| ------------------------------------ | ------------------------------------ | ------------------------ | -------- |
| `spyderbat.spyderbat_org_uid`        | Your organization UID                | —                        | Yes      |
| `spyderbat.spyderbat_secret_api_key` | Your Spyderbat API key               | —                        | Yes      |
| `spyderbat.api_host`                 | API host override for non-US regions | `api.prod.spyderbat.com` | No       |
| `namespace`                          | Kubernetes namespace                 | `spyderbat`              | No       |

## Verify the installation

Check the pod logs to confirm the forwarder started and is receiving events:

```bash
kubectl logs -f statefulset.apps/sb-forwarder-event-forwarder -n spyderbat
```

{% hint style="info" %}
The StatefulSet name above assumes you used `sb-forwarder` as the Helm release name (the `<release-name>` argument in the `helm install` command). If you used a different release name, replace `sb-forwarder` with your actual release name — for example, `statefulset.apps/my-release-event-forwarder`.
{% endhint %}

The forwarder logs status as JSON. Look for lines where the `"message"` field reports new record counts:

```
{"schema":"event_forwarder:meta:1.0.0","message":"5 new records (0 invalid, 5 logged)",...}
```

For detailed verification steps and troubleshooting, see [Event Forwarder validation](/concepts/integrations/spyderbat-event-forwarder#event-forwarder-validation).

## Related pages

* [SIEM Forwarding](/concepts/integrations/siem-forwarding) — control plane setup
* [Spyderbat Event Forwarder](/concepts/integrations/spyderbat-event-forwarder) — architecture overview
* [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart) — end-to-end setup guide


# Traditional Installer

Install the Spyderbat Event Forwarder on Linux using the traditional installer.

{% hint style="warning" %}
Before installing the Event Forwarder, enable SIEM forwarding on at least one saved query in the console. Without this step, the forwarder polls the API and receives nothing. See [SIEM Forwarding](/concepts/integrations/siem-forwarding).
{% endhint %}

The traditional installer sets up the Event Forwarder as a systemd service on Linux. It creates the `sbevents` system user, installs the binary to `/opt/spyderbat-events/`, and registers the service. Supports x86\_64 and arm64.

{% hint style="warning" %}
Run only one Event Forwarder instance per organization. Multiple instances each receive the full event stream independently and will cause duplicate events in your SIEM.
{% endhint %}

## Prerequisites

* A Linux host with systemd (VM, bare metal, or EC2)
* Spyderbat Nano Agents deployed on the hosts you want to monitor
* Your org UID and a Spyderbat API key (see [API Key Setup](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api))
* SIEM forwarding enabled on at least one saved query (see [SIEM Forwarding](/concepts/integrations/siem-forwarding))
* At least 512 MB of available memory and 1 CPU core if running alongside other services

## Install

1. Download the latest release from [github.com/spyderbat/event-forwarder/releases](https://github.com/spyderbat/event-forwarder/releases).
2. Unpack the tarball:

   ```bash
   mkdir /tmp/sef
   tar xfz spyderbat-event-forwarder.*.tgz -C /tmp/sef
   ```
3. Run the installer:

   ```bash
   cd /tmp/sef
   sudo ./install.sh
   ```

   The installer prints the config file location and the commands to start the service:

   ```
   spyderbat-event-forwarder is installed!

   Please edit the config file now:
       /opt/spyderbat-events/etc/config.yaml

   To start the service, run:
       sudo systemctl start spyderbat-event-forwarder.service

   To view the service status, run:
       sudo journalctl -fu spyderbat-event-forwarder.service
   ```
4. Edit the config file:

   ```bash
   sudo vi /opt/spyderbat-events/etc/config.yaml
   ```

   **Minimal configuration (required fields only):**

   ```yaml
   spyderbat_org_uid: YOUR-ORG-UID
   spyderbat_secret_api_key: YOUR-API-KEY

   # Non-US / non-default deployments only: hostname only, no scheme
   # api_host: api.example.spyderbat.com
   ```

   The `api_host` field is only needed if you are not using the default US deployment (`api.prod.spyderbat.com`). The value must be a hostname without a scheme — `api.example.com` is correct; `https://api.example.com` will cause a startup error.

   **With webhook output (for Panther, Splunk HEC, or any HTTP endpoint):**

   ```yaml
   spyderbat_org_uid: YOUR-ORG-UID
   spyderbat_secret_api_key: YOUR-API-KEY

   webhook:
     endpoint_url: https://your-siem/ingest
     compression_algo: zstd
     max_payload_bytes: 1048576
     authentication:
       method: bearer
       parameters:
         secret_key: YOUR-BASE64-ENCODED-SECRET
   ```

   For architecture details, see [Spyderbat Event Forwarder](/concepts/integrations/spyderbat-event-forwarder).
5. Start the service:

   ```bash
   sudo systemctl start spyderbat-event-forwarder.service
   ```
6. Verify the service is enabled to start at boot (the installer does this automatically):

   ```bash
   sudo systemctl is-enabled spyderbat-event-forwarder.service
   ```

   This should print `enabled`.

## Verify the installation

Tail the service logs to confirm the forwarder started and is receiving events:

```bash
sudo journalctl -fu spyderbat-event-forwarder.service
```

The forwarder logs status as JSON. Look for lines where the `"message"` field reports new record counts, for example:

```
{"schema":"event_forwarder:meta:1.0.0","message":"5 new records (0 invalid, 5 logged)",...}
```

You can also tail the event output file directly for human-readable ndjson events:

```bash
sudo tail -f /opt/spyderbat-events/var/log/spyderbat_events.log
```

For detailed verification steps and troubleshooting, see [Event Forwarder validation](/concepts/integrations/spyderbat-event-forwarder#event-forwarder-validation).

### Troubleshooting: service fails to start (crash loop)

If the forwarder crashes repeatedly on startup, check the logs for a DNS or URL error:

```bash
sudo journalctl -u spyderbat-event-forwarder.service --no-pager -n 20
```

A common cause is an incorrect `api_host` value. The field requires a hostname only — no scheme:

* Correct: `api_host: api.example.spyderbat.com`
* Incorrect: `api_host: https://api.example.spyderbat.com`

After fixing the config, clear the failed service state before restarting:

```bash
sudo systemctl reset-failed spyderbat-event-forwarder.service
sudo systemctl start spyderbat-event-forwarder.service
```

## Forward events to Splunk with the Universal Forwarder

If you use the Splunk Universal Forwarder, point it at the event output file:

```bash
sudo splunk add monitor /opt/spyderbat-events/var/log/spyderbat_events.log
```

## Related pages

* [SIEM Forwarding](/concepts/integrations/siem-forwarding) — control plane setup
* [Spyderbat Event Forwarder](/concepts/integrations/spyderbat-event-forwarder) — architecture overview
* [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart) — end-to-end setup guide
* [Panther Webhook Configuration](/tutorials/integrations/forwarder-panther-config) — webhook setup example


# Install the Spyderbat MCP Server

Model Context Protocol (MCP) server for LLMs to interact with Spyderbat's API, enabling AI assistants to search, query, and manage Spyderbat resources.

## What is MCP?

The [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) lets AI assistants interact with external tools and data. The Spyderbat MCP server exposes tools for searching, investigating, and managing your Spyderbat environment through natural language.

## Prerequisites

Before configuring the Spyderbat MCP server, you'll need:

* A Spyderbat account with API access
* A valid API key ([How to create one](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api))
* An MCP-compatible client (Claude Code, Cursor, Windsurf, or VS Code with Cline/Continue)

{% hint style="warning" %}
**API Key Security**

The Spyderbat MCP server authenticates via API key in the `Authorization` header (OAuth is not currently supported). Keep these guidelines in mind:

* **API keys inherit the permissions of the user who created them** — treat them like passwords
* **Never commit keys to version control** — store config files outside your repo or add them to `.gitignore`
* **For Claude Code**, use `-s user` to store the configuration in `~/.claude/` rather than the project directory
* **Rotate keys regularly** via the Spyderbat console
  {% endhint %}

## Quick Start

Pick your client below, or use one-click install:

**Claude Code** (one command, no file editing):

```bash
claude mcp add --transport http -s user spyderbat https://api.spyderbat.com/mcp/v1/mcp --header "Authorization: Bearer <your-api-key>"
```

**Other clients** — see [Client Configuration](#client-configuration) for Cursor, Windsurf, VS Code, and MCP Inspector.

After setup, restart your client and try: *"List my Spyderbat organizations"*

## One-Click Install

Install directly in your editor:

[![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.png)](cursor://anysphere.cursor-deeplink/mcp/install?name=spyderbat\&config=eyJ1cmwiOiJodHRwczovL2FwaS5zcHlkZXJiYXQuY29tL21jcC92MS9tY3AiLCJ0cmFuc3BvcnQiOiJzdHJlYW1hYmxlLWh0dHAiLCJoZWFkZXJzIjp7IkF1dGhvcml6YXRpb24iOiJCZWFyZXIgJHtpbnB1dDpTUFlERVJCQVRfQVBJX0tFWX0ifX0=) [![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_MCP_Server-007ACC?style=flat\&logo=visualstudiocode\&logoColor=white)](vscode:mcp/install?%7B%22name%22%3A%22spyderbat%22%2C%22config%22%3A%7B%22type%22%3A%22http%22%2C%22url%22%3A%22https%3A%2F%2Fapi.spyderbat.com%2Fmcp%2Fv1%2Fmcp%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7Binput%3ASPYDERBAT_API_KEY%7D%22%7D%7D%7D) [![Add to LM Studio](https://files.lmstudio.ai/deeplink/mcp-install-dark.svg)](lmstudio://add_mcp?name=spyderbat\&config=eyJ1cmwiOiJodHRwczovL2FwaS5zcHlkZXJiYXQuY29tL21jcC92MS9tY3AiLCJoZWFkZXJzIjp7IkF1dGhvcml6YXRpb24iOiJCZWFyZXIgJHtpbnB1dDpTUFlERVJCQVRfQVBJX0tFWX0ifX0%3D)

> **Note:** You'll be prompted to enter your Spyderbat API key during installation. [Create an API key](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api) if you don't have one.

## MCP Server Endpoints

The Spyderbat MCP server is deployed at the following endpoints:

* **US Region**: `https://api.spyderbat.com/mcp/v1/mcp`
* **Mumbai Region**: `https://api.mum.prod.spyderbat.com/mcp/v1/mcp`

All examples below use the US endpoint. Replace the URL if your organization is in the Mumbai region.

## Client Configuration

{% tabs %}
{% tab title="Claude Code" %}
Run the following command to add the Spyderbat MCP server:

```bash
claude mcp add --transport http -s project spyderbat https://api.spyderbat.com/mcp/v1/mcp --header "Authorization: Bearer <your-api-key>"
```

Use `-s project` for project-scoped config, or `-s user` for global config stored in `~/.claude/`.

After adding the server, restart Claude Code for the changes to take effect.

**Verify the connection:**

```
> List my Spyderbat organizations
```

{% endtab %}

{% tab title="Cursor" %}
Add the following to your Cursor MCP settings (Settings > MCP):

```json
{
  "mcpServers": {
    "spyderbat": {
      "url": "https://api.spyderbat.com/mcp/v1/mcp",
      "transport": "streamable-http",
      "headers": {
        "Authorization": "Bearer <your-api-key>"
      }
    }
  }
}
```

**`transport`** must be `"streamable-http"`.

After saving, restart Cursor for the changes to take effect.
{% endtab %}

{% tab title="Windsurf" %}
Add the following to `~/.codeium/windsurf/mcp_config.json`:

```json
{
  "mcpServers": {
    "spyderbat": {
      "serverUrl": "https://api.spyderbat.com/mcp/v1/mcp",
      "transport": "streamable-http",
      "headers": {
        "Authorization": "Bearer <your-api-key>"
      }
    }
  }
}
```

> **Note:** Windsurf uses `serverUrl` instead of `url` — copy-pasting from Cursor/VS Code configs won't work.

After saving, restart Windsurf for the changes to take effect.
{% endtab %}

{% tab title="VS Code / Cline" %}
In VS Code, open Cline settings and add the MCP server configuration:

```json
{
  "cline.mcpServers": {
    "spyderbat": {
      "url": "https://api.spyderbat.com/mcp/v1/mcp",
      "transport": "streamable-http",
      "headers": {
        "Authorization": "Bearer <your-api-key>"
      }
    }
  }
}
```

After saving, reload VS Code for the changes to take effect.
{% endtab %}

{% tab title="MCP Inspector" %}
[MCP Inspector](https://github.com/modelcontextprotocol/inspector) is a graphical debugging tool for testing MCP server connections and exploring available tools.

**Setup:**

1. **Transport Type**: Select **Streamable HTTP**
2. **URL**: Enter the [endpoint](#mcp-server-endpoints) for your region
3. **Connection Type**: Select **Via Proxy**
4. **Authentication**: Enable **Custom Headers** and add:
   * **Key**: `Authorization`
   * **Value**: `Bearer <your-api-key>`
5. Click **Connect**

<figure><img src="/files/WN2sZsUH7io1kFp3uju4" alt="MCP Inspector connected to Spyderbat MCP server"><figcaption><p>MCP Inspector connected to Spyderbat MCP server</p></figcaption></figure>

Once connected, you can browse available tools, view their schemas, and test them from the interface.
{% endtab %}
{% endtabs %}

## Example Prompts

The AI uses `onboard`, `prime`, and `search_docs` to learn available tools, schemas, and query syntax automatically. You can prompt conversationally and the AI translates your intent into API calls. For search schema and operator details, see the [Search Reference](/reference/search).

Here's a realistic triage session:

1. **Orient** — *"Get up to speed on Spyderbat. What tools do you have?"* Calls `onboard` and `prime` to learn available tools and investigation workflows.
2. **Check access** — *"What organizations do I have access to?"* Calls `list_organizations`, returns org names and UIDs.
3. **Triage** — *"Find the highest-scoring Spydertraces from the last 24 hours."* Searches `model_spydertrace` for `score > 50`, sorted descending. Traces bundle related processes, connections, and red flags into scored units — your triage queue.
4. **Investigate** — *"Dig into the top trace. What processes ran, what connections were made, and what triggered it?"* Calls `get_trace_contents` to pull processes, connections, and red flags linked to that trace. Walks the process tree via `ppuid` to show parent-child relationships.
5. **Attribute** — *"Where did that external connection come from? What pod and namespace?"* Searches `model_connection` by IP, resolves `pod_uid` via `get_objects` to get pod name, namespace, and node.
6. **Resolve** — *"That's a health probe — suppress it for this cluster."* Calls `suppress_trace` with `preview=true` to generate a suppression policy, then applies it scoped to the cluster after you confirm.

### Example Conversation

```
User: Find high-severity red flags in the prod-k8s cluster from the last 24 hours

AI: I'll search event_redflag with:
    severity = "high" and cluster_name = "prod-k8s"
    start_time = "-24h"

Found 3 results:
- root_shell (MITRE TA0004.T1548) in namespace "payments" — score 82
- ssh_inside_container_detection (MITRE TA0008.T1021) in namespace "backend" — score 67
- nonk8s_container_detection (MITRE TA0005.T1610) in namespace "monitoring" — score 31

User: The nonk8s_container_detection in monitoring — what's the process tree?

AI: Process lineage via ppuid:
    systemd → containerd-shim → bash → curl 127.0.0.1:8080/healthz
    This is a Kafka health check probe hitting localhost, not a suspicious container.

User: Suppress it for the monitoring namespace.

AI: Previewing suppression policy scoped to namespace "monitoring"...
    [shows policy preview, warnings, and scope]
    Apply? (yes/no)
```

{% hint style="info" %}
**Context Window Usage**

Broad searches can return large result sets that consume context window tokens. To keep responses useful, narrow time ranges, use specific filters, and ask the AI to summarize rather than dump all results.
{% endhint %}

## Troubleshooting

### 401 Unauthorized

* Verify your API key is valid and not expired
* Check that the Authorization header format is exactly: `Bearer <your-api-key>`

### Connection Timeout

* Check that your network can reach `api.spyderbat.com`
* Verify the endpoint URL matches your organization's region

### Tools Not Appearing

* Restart your MCP client after configuration changes
* For Claude Code: exit and restart the CLI
* For Cursor: fully restart the application

### Empty Search Results

If searches return no data, walk through this debugging flow:

1. **Verify org access** — ask: *"List my organizations"* and confirm the UID matches
2. **Widen time range** — ask: *"What is the current time?"* (uses `get_current_time`) and check that your search window includes recent data
3. **Check schema** — ask: *"List available search schemas for my org"* to confirm the schema exists
4. **Check permissions** — verify your API key has access to the target organization

### Getting Help

If you encounter issues not covered here, check the [Spyderbat documentation](/) or contact support.

## Video Walkthrough

Watch a demonstration of the Spyderbat MCP server in action: <https://www.loom.com/share/1ba8c96999484d20a8f5b831c07cff6a>


# Concepts

The Concepts section helps you learn about the Spyderbat Platform. This section is designed to provide a comprehensive understanding of the fundamental ideas and principles that let you view and secur

***


# Guardian & Interceptor

The Spyderbat Guardian Feature is designed to enhance security within your Spyderbat environment. It provides a robust framework for defining and enforcing expected behavior through Guardian Policies. These policies are crucial for maintaining the integrity of your systems and ensuring that only authorized activities are permitted.

## Guardian Policies

Guardian Policies are the cornerstone of Guardian, serving as the rulebook for allowed and prohibited activity within your environment. They come in two primary forms:

### Workload Policies

Workload Policies are tailored to containers and Linux services, specifying a whitelist of permitted activities. This ensures that only known, safe operations are allowed to execute, providing a first line of defense against unauthorized or malicious behavior.

Read more about Workload Policies [here](/concepts/guardian/workload_policies).

#### Key Components:

* A comprehensive list allowed process and network activity.
* Scope: The selectors detailing the specific containers or services to which the policy applies.
* Response: The mechanism by which the policy take actions.

### Ruleset Policies

Ruleset Policies offer a more flexible approach, supporting policy-agnostic rulesets that can be applied across different environments. These rulesets contain both allow and deny rules, providing a granular level of control over the behavior within your systems.

Read more about Ruleset Policies [here](/concepts/guardian/ruleset_policies).

#### Key Components:

* Allow Rules: Explicitly permit certain actions, overriding any broader deny rules that may be in place.
* Deny Rules: Define actions that are explicitly prohibited, regardless of other allow rules.
* Reusability

### Interceptor

The Interceptor feature set allows Guardian to take response actions based on policy violations. When a policy violation occurs, Interceptor Response Actions can trigger actions such as generating alerts, or blocking the offending activity.

More details on response actions can be found [here](/reference/policies/response-actions).

## Tutorials

Tutorials detailing the creation of the various policy types can be found in the `tutorials` section of this documentation.

* [Guardian Tutorials](/tutorials/guardian)

## Conclusion

The Spyderbat Guardian Feature is a powerful tool for maintaining security and compliance in containerized and Linux service environments. By effectively utilizing Guardian Policies, you can ensure that your systems operate within the defined parameters of expected behavior, safeguarding against potential threats.

For more detailed information and advanced configurations, please refer to the [policy reference guide](/reference/policies).


# Ruleset Policies

<mark style="color:blue;">Published: April 29, 2024</mark>

## What are Ruleset Policies

Ruleset Policies are a way of defining a set of allow or deny rules for a given scope. Currently, ruleset policies are supported for the following scope(s):

* Kubernetes Clusters

Ruleset Policies, themselves are very simple. They define a scope with a selector, and they contain a list of pointers to reusable rulesets. And they also define a set of response actions when deviations occur.

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  createdBy: demo.user@spyderbat.com
  creationTimestamp: 1712787973
  lastUpdatedBy: demo.user@spyderbat.com
  lastUpdatedTimestamp: 1714417836
  name: demo-cluster-policy
  selectorHash: 66e45259eba6ed4365e28e7e673a18cf
  type: cluster
  uid: pol:xxxxxxxxxxxxxxxxxxxx
  version: 1
spec:
  clusterSelector:
    matchFields:
      name: demo-cluster
  enabled: true
  mode: audit
  rulesets:
  - demo-cluster-ruleset
  response:
    default:
    - makeRedFlag:
        severity: high
    actions: []
```

The Rulesets used by a Ruleset Policy are policy-agnostic and as such can be defined once and used across multiple policies. Rulesets contain a set of allow or deny rules. Each rule contains a target, verb, list of values, and optional selectors (for additional scoping).

* Target: what the rule is referring to within the scope of the policy.
  * ex. `container::image` this means that we are allowing or denying containers using images specified in the values field.
* Verb: The currently available verbs for ruleset rules are `allow` or `deny`. Any object matching a deny rule will generate a Deviation.
* Values: This is the set of values that are allowed or denied. If the target is `container::image` then the values should be container images that are either allowed or denied.
* Selectors: Optional selectors that further define the scope of a single rule. For instance you may want a rule that defines allowed activity in a specific namespace within a cluster.

The following is an example rule that allows containers with the images `docker.io/guyduchatelet/spyderbat-demo:1` and `docker.io/library/mongo:latest` in the namespaces `rsvp-svc-dev` and `rsvp-svc-prod`.

```yaml
namespaceSelector:
  matchExpressions:
  - {key: kubernetes.io/metadata.name, operator: In, values: [rsvp-svc-dev, rsvp-svc-prod]}
target: container::image
values:
- docker.io/guyduchatelet/spyderbat-demo:1
- docker.io/library/mongo:latest
verb: allow
```

The following rule denies the image `docker.io/guyduchatelet/spyderbat-demo:2` globally.

```yaml
target: container::image
values:
- docker.io/guyduchatelet/spyderbat-demo:2
verb: deny
```

The following is an example ruleset automatically generated from a demo cluster:

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatRuleset
metadata:
  createdBy: demo.user@spyderbat.com
  creationTimestamp: 1712787972
  lastUpdatedBy: demo.user@spyderbat.com
  lastUpdatedTimestamp: 1714162618
  name: demo-cluster-ruleset
  type: cluster
  uid: rs:xxxxxxxxxxxxxxxxxxxx
  version: 1
spec:
  rules:
  - namespaceSelector:
      matchExpressions:
      - {key: kubernetes.io/metadata.name, operator: In, values: [rsvp-svc-dev, rsvp-svc-prod]}
    target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:1
    - docker.io/library/mongo:latest
    verb: allow
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: kube-system
    target: container::image
    values:
    - 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon-k8s-cni-init:v1.10.1-eksbuild.1
    - 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon-k8s-cni:v1.10.1-eksbuild.1
    - 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/coredns:v1.8.7-eksbuild.1
    - 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/kube-proxy:v1.22.6-eksbuild.1
    - public.ecr.aws/aws-secrets-manager/secrets-store-csi-driver-provider-aws:1.0.r2-58-g4ddce6a-2024.01.31.21.42
    - registry.k8s.io/csi-secrets-store/driver:v1.4.2
    - registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.10.0
    - registry.k8s.io/sig-storage/livenessprobe:v2.12.0
    verb: allow
  - target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:2
    verb: deny
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: prometheus
    target: container::image
    values:
    - quay.io/prometheus/node-exporter:v1.7.0
    - quay.io/prometheus/pushgateway:v1.7.0
    - registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.10.1
    verb: allow
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: spyderbat
    target: container::image
    values:
    - public.ecr.aws/a6j2k0g1/aws-agent:latest
    - public.ecr.aws/a6j2k0g1/nano-agent:latest
    verb: allow
```

## How rules are evaluated

Rules are evaluated based on a specific hierarchy. Scoped rules take precedence over global rules, explicit rules take precedence over wildcarded rules, deny rules are evaluated first, and anything that matches no rules is denied by default.

### Evaluation Order

1. Scoped explicit deny
2. Scoped explicit allow
3. Scoped wildcarded deny
4. Scoped wildcarded allow
5. Global explicit deny
6. Global explicit allow
7. Global wildcarded deny
8. Global wildcarded allow
9. Default deny

|              |                                                     |
| ------------ | --------------------------------------------------- |
| \*Scoped     | the rule contains a selector                        |
| \*Explicit   | the matched value contains no wildcard characters   |
| \*Wildcarded | the matched value contains a wildcard character `*` |

### Examples

#### Scenario 1 (Global Explicit Allow):

Image: `docker.io/guyduchatelet/spyderbat-demo:1`

```yaml
spec:
  rules:
  - target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:1
    verb: allow
```

A container with the image `docker.io/guyduchatelet/spyderbat-demo:1` would be allowed globally.

#### Scenario 2 (Default Deny)

Image: `docker.io/guyduchatelet/spyderbat-demo:bad-tag`

```yaml
spec:
  rules:
  - target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:1
    verb: allow
```

A container with the image `docker.io/guyduchatelet/spyderbat-demo:bad-tag` would be denied by default.

#### Scenario 3 (Global Explicit Allow with Global Wildcard Deny):

Image 1: `docker.io/guyduchatelet/spyderbat-demo:1` Image 2: `docker.io/guyduchatelet/spyderbat-demo:bad-tag`

```yaml
spec:
  rules:
  - target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:1
    verb: allow
  - target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:*
    verb: deny
```

Global explicit allow is evaluated before global wildcarded deny so Image 1 is allowed. Image 2 is denied by the global wildcarded deny.

#### Scenario 3 (Scoped Wildcarded Allow with Global Explicit Deny):

Image 1: `docker.io/guyduchatelet/spyderbat-demo:1` Namespace labels: `{kubernetes.io/metadata.name: rsvp-demo-prod}` Image 2: `docker.io/guyduchatelet/spyderbat-demo:bad-tag` Namespace labels: `{kubernetes.io/metadata.name: rsvp-demo-prod}` Image 3: `docker.io/guyduchatelet/spyderbat-demo:bad-tag` Namespace labels: `{kubernetes.io/metadata.name: rsvp-demo-dev}`

```yaml
spec:
  rules:
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: rsvp-demo-prod
    target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:*
    verb: allow
  - target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:bad-tag
    verb: deny
```

Since the first rule has a namespace selector, that rule is *scoped*. Scoped wildcarded allow rules are evaluated before global explicit deny rules so Image 1 and Image 2 are allowed. Image 3 is denied by the global explicit deny rule.

#### Scenario 4 (Scoped Explicit Allow with Scoped Wildcarded Deny)

Image 1: `docker.io/guyduchatelet/spyderbat-demo:1` Namespace labels: `{kubernetes.io/metadata.name: rsvp-demo-prod}` Image 2: `docker.io/guyduchatelet/spyderbat-demo:bad-tag` Namespace labels: `{kubernetes.io/metadata.name: rsvp-demo-prod}` Image 3: `docker.io/guyduchatelet/spyderbat-demo:bad-tag` Namespace labels: `{kubernetes.io/metadata.name: rsvp-demo-dev}`

```yaml
spec:
  rules:
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: rsvp-demo-prod
    target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:1
    verb: allow
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: rsvp-demo-prod
    target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:*
    verb: deny
```

Both rules are *scoped* because they have a namespace selector. Scoped explicit allow rules are evaluated before scoped wildcarded deny rules so Image 1 is allowed. Image 2 is denied by the scope wildcarded deny rule. Image 3 does not match the scope of any rule so it is denied by default.

#### Scenario 5 (Scoped Explicit Allow with Scoped Explicit Deny)

Image: `docker.io/guyduchatelet/spyderbat-demo:1` Namespace labels: `{kubernetes.io/metadata.name: rsvp-demo-prod}`

```yaml
spec:
  rules:
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: rsvp-demo-prod
    target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:1
    verb: allow
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: rsvp-demo-prod
    target: container::image
    values:
    - docker.io/guyduchatelet/spyderbat-demo:1
    verb: deny
```

Scoped explicit deny rules are evaluated before scope explicit allow rules, so the image is denied by scoped explicit allow.

## Quick Start Tutorial

To quickly get started using using Cluster Ruleset Policies follow our tutorial using spyctl.

[How to Put Guardrails Around Your K8s Cluster](/tutorials/guardian/how-to-put-guardrails-around-your-k8s-cluster-spyctl)


# Workload Policies

Workload Policies define allowed process and network activity for containers and Linux services, alerting or blocking deviations from expected behavior.

## What Are Workload Policies?

Workload Policies are the most granular form of Guardian Policy. They define the allowed process and network activity for well-defined workloads. Policies are supported for two workload types:

* **Containers** — including standalone Docker containers and Kubernetes pod containers
* **Linux Services** — systemd-managed services running on monitored machines

Workload policies work best when the workload has a relatively stable set of activity. A third-party containerized application that runs a few processes and makes a few network connections is a good candidate — the activity can be baselined and the policy should stabilize quickly.

Workload policies are less effective for dynamic, constantly changing activity. A development container where engineers run unique and varied commands is a poor fit because the policy would require constant updates.

## How Workload Policies Work

### Fingerprints

As soon as the Nano Agent is installed on a machine, Spyderbat automatically gathers information about workloads running on it. The platform compiles **Fingerprints** for each Linux service and container it sees. A Fingerprint represents the observed process and network activity for a single workload instance over a time window.

To view Fingerprints, navigate to **Guardian > Fingerprints** in the console. Use the **Container** and **Linux-Service** tabs to switch between workload types.

<figure><img src="/files/z2w5tYChwv3d5ar8KwMc" alt="Guardian Fingerprints page showing the Container tab with a list of container fingerprints, faceted filtering, and columns for image name, cluster, namespace, machines, policy coverage, and fingerprint count."><figcaption><p>The Guardian Fingerprints page displays all observed container fingerprints with filtering by cluster, namespace, machine, status, image, and pod labels.</p></figcaption></figure>

The Fingerprints page shows key information for each fingerprint group:

| Column                            | Description                                     |
| --------------------------------- | ----------------------------------------------- |
| **Image Name** / **Service Name** | The container image or Linux service identifier |
| **Cluster**                       | The Kubernetes cluster (containers only)        |
| **Namespace**                     | The Kubernetes namespace (containers only)      |
| **Machines**                      | How many machines this workload runs on         |
| **Covered By Policy**             | Whether an existing policy covers this workload |
| **Number of Fingerprints**        | How many individual fingerprint instances exist |

Here is an example Fingerprint for a single container. It captures the observed process tree and network connections:

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatFingerprint
metadata:
  name: docker.io/library/nginx:latest
  type: container
  cluster_name: production-cluster
  containerName: web-proxy
  namespace: default
spec:
  containerSelector:
    image: docker.io/library/nginx:latest
    imageID: sha256:10d1f5b58f74...
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: default
  podSelector:
    matchLabels:
      app: web-proxy
  processPolicy:
  - name: nginx
    exe:
    - /usr/sbin/nginx
    euser:
    - root
    id: nginx_0
    children:
    - name: nginx
      exe:
      - /usr/sbin/nginx
      euser:
      - www-data
      id: nginx_1
  networkPolicy:
    ingress:
    - from:
      - ipBlock:
          cidr: 10.0.1.0/24
      ports:
      - port: 443
        protocol: TCP
      processes:
      - nginx_1
    egress: []
```

### From Fingerprints to Policies

When you create a policy, Spyderbat combines all related Fingerprints into a single document. If the same container image runs across 14 instances and one instance has network connections the others do not, the resulting policy includes that extra activity — producing a union of all observed behavior.

The resulting Workload Policy looks like:

```yaml
containerSelector:
  image: docker.io/library/nginx:latest
mode: audit
enabled: true
processPolicy:
- name: nginx
  exe:
  - /usr/sbin/nginx
  euser:
  - root
  id: nginx_0
  children:
  - name: nginx
    exe:
    - /usr/sbin/nginx
    euser:
    - www-data
    id: nginx_1
networkPolicy:
  ingress:
  - from:
    - ipBlock:
        cidr: 10.0.1.0/24
    - ipBlock:
        cidr: 10.0.2.0/24
    ports:
    - port: 443
      protocol: TCP
    processes:
    - nginx_1
  egress: []
response:
  default:
  - makeRedFlag:
      severity: high
  actions: []
```

Notice the key differences from the Fingerprint: the policy adds `mode`, `enabled`, and `response` fields, and the network policy now includes IP ranges from all instances.

### Creating a Policy from Fingerprints

To create a Workload Policy:

1. On the Fingerprints page, select one or more fingerprints using the checkboxes.
2. Click **Create Policy** in the action bar that appears at the bottom of the page.
3. **Step 1 — Baseline and Edit**: Spyderbat combines the selected fingerprints into a single policy document. The policy editor shows the YAML on the left and a visual Policy Trace on the right. You can edit the policy to generalize selectors (such as namespaces or clusters) or adjust process and network rules.
4. **Step 2 — Save Policy**: Name the policy and save it.

{% hint style="info" %}
While it is technically possible to define Workload Policies from scratch, it is best to use Spyderbat's assisted creation from Fingerprints. The platform handles the complex process and network tree structures automatically.
{% endhint %}

### Policy Modes

Every Workload Policy operates in one of two modes:

| Mode        | Behavior                                                                                                                                      |
| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audit**   | Monitors activity and records Deviations, but does not take enforcement actions. Use this mode when baselining a new policy.                  |
| **Enforce** | Records Deviations and executes configured Response Actions (such as killing a process or pod). Use this mode once the policy has stabilized. |

New policies default to **audit** mode. Switch to **enforce** mode only after reviewing Deviations and confirming the policy accurately reflects expected behavior.

### Deviations

Once applied, a policy continuously monitors all workloads matching its selectors. When activity occurs that falls outside the policy's allowed process or network rules, the platform generates a **Deviation**.

A Deviation is a record containing all the information needed to either:

* **Update the policy** to allow the new activity (if it is expected)
* **Investigate the activity** as a potential security concern

The Guardian Policies page shows deviation counts for each policy over the last 24 hours, including both unique and total deviation counts.

<figure><img src="/files/8FfX6Z7zP9vqS4PfOh8K" alt="Guardian Policies page showing a table of policies with columns for name, type, created date, latest timestamp, policy mode, status, unique deviations, and total deviations."><figcaption><p>The Guardian Policies page lists all workload policies with their mode, status, and recent deviation counts.</p></figcaption></figure>

### Response Actions

When a Deviation occurs, the policy executes its configured response actions. The two categories are:

* **Standard actions** — Generate flags that appear in dashboards and can trigger notifications:
  * `makeRedFlag` — Creates a security-focused flag that can trigger Spydertraces
  * `makeOpsFlag` — Creates an operations-focused flag for infrastructure issues
* **Agent actions** — Direct the Nano Agent to take action on the machine:
  * `agentKillProcess` — Terminates the deviant process
  * `agentKillProcessGroup` — Terminates the deviant process and its process group
  * `agentKillProcessTree` — Terminates the deviant process and all descendants
  * `agentKillPod` — Terminates the entire Kubernetes pod (containers only)
  * `agentReniceProcess` — Adjusts the CPU priority of the deviant process

By default, new policies are configured with a single response action: `makeRedFlag` with `severity: high`.

{% hint style="warning" %}
Agent enforcement actions (`agentKillProcess`, `agentKillPod`, etc.) actively terminate workloads. Only enable these in **enforce** mode after the policy has been thoroughly validated in **audit** mode.
{% endhint %}

For full details on response action configuration, see the [Response Actions reference](/reference/policies/response-actions).

## Policy Structure

A Workload Policy defines three main sections:

| Section            | Purpose                                                                                                                             |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------- |
| **Selectors**      | Define which workloads the policy applies to (by container image, namespace, pod labels, cluster, machine, or service name)         |
| **Process Policy** | A tree of allowed processes, including executable paths, effective users, and parent-child relationships                            |
| **Network Policy** | Allowed ingress and egress network connections, including IP ranges, ports, protocols, and which processes may make the connections |

For reference documentation on selectors, see the [Selectors reference](/reference/selectors).

## Related Pages

* [Guardian & Interceptor](/concepts/guardian) — Overview of Guardian policies and Interceptor response actions
* [Ruleset Policies](/concepts/guardian/ruleset_policies) — Cluster-level policies using reusable allow/deny rulesets
* [Response Actions](/reference/policies/response-actions) — Full reference for configuring policy response actions
* [Selectors](/reference/selectors) — Reference for policy selector types
* [How to Lock Down Your Workloads with Guardian Policies Using Spyctl](/tutorials/guardian/how-to-lock-down-your-critical-workloads-with-policies-spyctl) — Step-by-step tutorial


# Flashback (Go Back In Time)


# Investigations

Overview of Process Investigations in the Spyderbat Console — the Records Panel, Causal Tree, and Details Panel for visual causal analysis of Spydertraces.

A **Process Investigation** is Spyderbat's visual causal analysis tool. It takes a Spydertrace — a complete record of causally connected activity on a system — and lets you explore every process, connection, and event in a navigable Causal Tree. Instead of piecing together log lines, you see the full chain of cause and effect.

## Starting an Investigation

Investigations can be launched from **Search** results. Navigate to **Search Page**, run a query that returns Spydertraces, then select one or more traces using the checkboxes. A toast bar appears at the bottom of the screen with action buttons — click **Start Process Investigation**.

<figure><img src="/files/OidpNysit0NJBrNwYuoj" alt="Search results with traces selected and the action toast showing Start Process Investigation"><figcaption><p>Select one or more Spydertraces from Search, then click Start Process Investigation in the toast bar.</p></figcaption></figure>

The Investigation view opens in a new browser tab. It has three main areas: the **Records** panel (left), the **Causal Tree** (right), and the **Details** panel (below the tree).

<figure><img src="/files/TfPhR9e10WeRwD57mTwV" alt="Full investigation view showing Records panel, Causal Tree, and Details panel"><figcaption><p>The Investigation view: Records on the left, Causal Tree on the right, Details below.</p></figcaption></figure>

## Records Panel

The Records panel on the left side controls what data is available for investigation.

### Hosts and Time Range

At the top, the **Hosts** dropdown selects which machine(s) to query. Below it, the time range picker lets you adjust the window of activity. Click **Run New Search** to refresh results for a different host or time range.

### Data Layers

**Data Layers** appear below the time range. Each search or dashboard card that feeds into the investigation creates a separate data layer. You can toggle layers on or off to focus on specific subsets of data — similar to layers in a graphics editor. The layer count (e.g., "Data Layers (2)") shows how many are loaded.

### Record Type Tabs

Records are organized into tabs by type:

* **Spydertrace** — Complete traces of causally connected activity
* **Red Flags** — Security-relevant events detected by Spyderbat. See [Custom Flags](/concepts/scout/custom-flags) for creating your own detections.
* **Process** — Individual process records
* **Connection** — Network connections
* **Container** — Container-level records

Each tab shows a count of matching records. Select a tab to browse records of that type.

### Adding Records to the Causal Tree

To visualize a record in the Causal Tree:

* Click the **star icon** on an individual record row to add it to the graph
* Click **Add All** above the records table to add all visible records at once
* Use **Show in Graph** to jump to a record already displayed in the tree

## Causal Tree

The Causal Tree is the core of the investigation — a visual graph showing causal relationships between processes, connections, and systems.

### Node Types

* **S nodes** — Systems (machines or hosts)
* **P nodes** — Processes (individual commands or executables)
* **C nodes** — Connections (network activity, linked to remote IPs and ports)

Grey badges next to process nodes indicate changes in the effective user or privilege level, giving immediate visual context about who was executing what.

### Mode Toggle

At the top of the Causal Tree, switch between two interaction modes:

* **Pan & Zoom** — Click and drag to pan the view, scroll to zoom. Best for exploring large graphs.
* **Selection** — Click nodes to select them, drag to box-select multiple nodes. Best for manipulating specific parts of the graph.

### Toolbar

The toolbar provides these controls:

* **Clear All** (trash icon) — Remove all nodes from the graph
* **Undo / Redo** — Step backward or forward through graph changes
* **Auto Focus** — Automatically center and zoom to fit all nodes
* **Zoom In / Zoom Out** — Adjust zoom level (also available via scroll wheel)
* **Highlight nodes** — A dropdown that color-codes nodes by a selected attribute (e.g., Container UID). Nodes sharing the same value get the same color, making it easy to spot container or host boundaries.
* **Options** — Display toggles: Hide Threads, Show Relative Time, Hide Future Nodes, Hide Container Box, Hide Process Context Box
* **Copy Investigation Link** (top right) — Generate a permalink to the current investigation state. Links are not public — the recipient must be a member of your organization with read access.
* **Summarize** (top right) — Get an AI-generated summary of the investigation. See [Spydertrace Summarize](/concepts/summarize/spydertrace-summarize) for details.

### Node Navigation

At the bottom of the Causal Tree:

* **Previous node / Next node** — Cycle through nodes in chronological order
* **Add next N objects** — When a Spydertrace contains more than 50 objects, only the first 50 load automatically. This button appears when more objects are available — click it to load the next batch. A progress indicator (e.g., "6 / 8 Objects Displayed") shows how many are currently loaded versus the total.

### Interacting with Nodes

* **Left-click** a node to select it and view its details in the Details panel
* **Right-click** a node for a context menu with options to add or remove related nodes (children, descendants, connections, flags)

For a deeper guide on Causal Tree usage, see [How to Use the Investigations Feature in Spyderbat](/tutorials/flashback/how-to-investigate-in-spyderbat).

## Details Panel

The Details panel appears below the Causal Tree and shows metadata for the selected node or record. It is organized into expandable sections:

* **Process** — Command line, executable path (shown when a process node is selected)
* **Spydertrace** — Trace score, record counts (flags, processes, connections, systems)
* **Container** — Container name, image, and runtime details
* **Parent Process** — The parent process that spawned the selected process
* **System** — Hostname, IP, and system identifiers
* **AWS** — Cloud instance metadata (region, instance ID, etc.), when available

Click a different node in the Causal Tree or a different row in the Records table to update the Details panel with that item's metadata.


# Search

Detailed overview of the Search tab in the console and the Spyderbat search language it uses.

The Spyderbat Search Language is a tool designed to easily query and extract actionable insights from your data within the Spyderbat console. This guide is designed to introduce the language and to help you understand the search functionalities and their practical applications for your organization.

The Search section of the Spyderbat UI is located second from the top of the left hand navigation panel. When entering the page, you will be presented with a search bar, some example searches, and a list of recent searches if you have run any previously.

![Spyderbat search page](/files/FFt1i0Tk0eqZ3Ag3Cscp)

To begin searching, click on the search bar. You’ll need to pick a schema, then enter a query expression in the Spyderbat search language, and finally select a time range. All of these steps will be elaborated on below in more detail. The schema and expression can also be filled by clicking on an example or recent search from below the bar. From there, you can click on the search button to begin loading results, or see a syntax error if the query was invalid.

## Schemas

In Spyderbat, a schema represents a type of data collected by Spyderbat. For example, the schemas shown in the search UI include `Process` and `Connection`, which represent data associated to processes and network connections, respectively.

There are a number of schemas available to search on, each with their own fields that can be queried, which can be found categorized in the schema selector. These schemas are not part of the text of your query, but are selected on the search page. Additionally, a full list of them can be found in the [Search Reference](/reference/search).

![Search schemas selector](/files/k0r1o2wBWQSUVVL7kQ4B)

### The Query Builder

After selecting a schema, you can either begin to type an expression, or open the Query Builder. The Query Builder is a useful tool for crafting expressions that contains all the information about the schemas, instead of requiring you to check the reference and documentation.

It contains the full set of fields for each schema, descriptions of the fields, and matches them to comparisons for you, allowing you to easily create effective and correct searches.

![Search query builder](/files/MhkmiffavF6zGv1GIIiV)

The Query Builder will open with the same schema as you had selected in the search page, but you can also select a different schema using the dropdown next to the query preview. Additionally, clicking on the query preview will show a list of recent queries constructed using the Query Builder. Clicking on one will populate the Query Builder and allow you to modify it or send it to the search page.

To modify a query in the Query Builder, use the selector boxes to select a field and what to compare it with. The selectors also contain short descriptions of the fields and operators.

## Expressions

After selecting the schema, you’ll need to specify the expression for your search. The expression tells the query engine exactly what criteria to use for filtering data. Every schema has its own set of fields that can be used in the expression, and each field has a type such as `String` or `Number` that determines the comparisons available. In addition to these simple types, the `List` and `Map` types are also available as composite types, which are discussed below.

The simplest possible expression is the "Everything" query, represented by a single `*`. This will return every object within the given time frame.

### Comparisons

The next basic type of expression is a comparison of a field and a value. For example, given that the `Process` schema has the `Executable` field, we can use its shortened name `exe` in an expression to find all processes with the executable "/usr/bin/bash":

```txt
schema: Process
query:  exe = "/usr/bin/bash"
```

This expression uses the `=` operator, which simply checks for equality with the value. Each field type has a set of available operators in addition to `=`, allowing for more advanced searches. Additionally, inside an expression, fields must always use their shortened name, and they can only be compared with constant values in the expression itself, not the values of other fields.

A full list of the available comparison operators is available in the [Search Reference](/reference/search).

#### Pattern Matching

String fields have two unique operators: `~=` and `~~=`, which are Unix glob style pattern matching and regular expression matching, respectively. Regular expressions can be researched elsewhere, but glob pattern matching in this context refers to a string with the characters `*` and `?` used as wildcards. They can both be escaped in a pattern to match their literal characters instead of wildcards using the standard escape character `\`.

As wildcards, `*` can match any number of characters, while `?` matches exactly one unknown character. For example, we can modify the previous example to check for any bash executable, instead of a specific path:

```txt
schema: Process
query:  exe ~= "*bash"
```

String escapes are processed before regular expression evaluation, so regular expressions containing the `\` character will need to be double-escaped. To make complex expressions easier to write, raw strings can be used to avoid string processing before evaluating the regular expression:

```txt
Normal strings: args[*] ~~= "\\[[0-9]+\\]"
Raw strings: args[*] ~~= r"\[[0-9]+\]"
```

The string escape parsing logic follows the same rules as [Python strings](https://docs.python.org/3/reference/lexical_analysis.html#escape-sequences).

#### CIDR Matching

The IP address type also has a CIDR matching operator: `<<`. CIDR blocks can be researched elsewhere, but the operator is essentialy a way to compare against a range of IP addresses instead of individual ones. For example, we can check for connections from any IP address between 192.168.1.0 and 192.168.1.255:

```txt
schema: Connection
query:  local_ip << 192.168.1.0/24
```

#### Matching Empty Values

Fields that do not have a value can be matched with `null` using the special `is` and `is not` operators. For example, to find processes without any container:

```txt
schema: Process
query:  container_id is null
```

If instead the value exists, but is an empty string, we can use an empty string to match against it. For example, matching against processes that have the "DEBUG" environment variable set to the empty string:

```txt
schema: Process
query:  environ["DEBUG"] = ""
```

This query will not match processes where the DEBUG environment variable simply does not exist.

### Logic

Multiple field comparisons can be combined together using the basic logical operations `and`, `or`, and `not`. They can use any capitalization and may be combined with parentheses to specify precedence. For example:

```txt
schema: Process
query:  (auser = "root" or euser = "root") and duration > 60
```

This search uses parentheses to guarantee that the `Duration` field must always be greater than 60 seconds, regardless of the other conditions.

### Field Types

In addition to the available operators, certain types of fields have special rules. For all types, the value used with any comparisons must be of the same type, although `Number` and `Integer` are effectively the same. In addition, all `String` values must be in either single or double quotes, and may escape any ending quotes inside the string using the standard escape character `\`.

There are two outlier types that fields can have: `List` and `Map`. These types both have elements of another type, most commonly `String`. A `List` represents an ordered collections of elements, while a `Map` represents key and element pairs. Interacting with these data types is relatively straightforward and can be combined with other comparisons for more complex queries. To use a `List`, you can either access elements by index – for example, you would use `[0]` to access the first element in the list – or search for a value occurring anywhere using the `[*]` syntax. Elements in a `Map` can be queried by referencing the key in brackets or with the `:keys[*]` and `:values[*]` syntax to search for a key or value anywhere in the map, respectively.

For example, this search finds `Process` objects with a first argument of “-i”:

```txt
schema: Process
query:  args[1] = "-i"
```

`Process` objects where any argument is “-i”:

```txt
schema: Process
query:  args[*] = "-i"
```

Kubernetes `Pod` objects serving the PostgreSQL database in the "production" namespace:

```txt
schema: Pod
query:  metadata.labels["service"] = "postgres" AND metadata.namespace = "production"
```

A full list of fields and their types is available in the [Search Reference](/reference/search).

### Filtering with related objects

An advanced feature of the Spyderbat search language is related object queries - the ability to reference other objects of different schemas in addition to the main object and query fields on those objects. Every schema has a set of related objects alongside its fields, which point to specific other objects of either the same or a different schema.

After a related object, any field or even related object on the related object's new schema can be chained together using the normal query syntax. This is useful when you want to filter based on information that isn’t contained in the original schema.

The capabilities are best illustrated with an example, such as the `machine` reference in the `Process` object, which points to a related object with the `Machine` schema - in this case, it would be the machine that the process is running on. Using the `Cloud Region` field in the `Machine` schema, we can find all bash processes in the "us-east-1" cloud region, even though we do not gather any region data in the `Process` schema:

```txt
schema: Process
query:  exe = "/usr/bin/bash" and machine.cloud_region = "us-east-1"
```

In the example above, each process has exactly one associated machine. Some schemas have multiple other objects that can be associated, however, such as the children of a process. In that case, the reference must always be followed by `[*]`, similar to how any element of a `List` can be used in a comparison. For example, the mentioned `children[*]` reference in the `Process` schema can be used to find all bash processes that ran "sudo" in a child process:

```txt
schema: Process
query:  exe = "/usr/bin/bash" and children[*].exe ~= "*sudo"
```

Related objects of this type can only be queried with the `[*]` syntax, and attempting to use an index or a string key will cause a syntax error.

## Time Range

After selecting a schema and creating an expression, the last part necessary for a search is to select the time range for the query. When doing so, this will return all data that existed at any point during that time range. For example, a search with the `Process` schema in a time range of the previous 15 minutes will return all matching processes that were running in the past 15 minutes, but could have been started five minutes or a day ago.

The Spyderbat UI includes a time picker with preset relative times and a custom duration picker. Relative times become constant for a search once it has been run, as opposed to the search continuing to update as time goes by. For a search to continue reporting results, save it to a Dashboard card.

![Search time picker](/files/31VS6e1LIEctWZsZD6dS)

## Next Steps

Once you have search results, you can select one or more Spydertraces and click **Start Process Investigation** to visualize causal relationships in the [Causal Tree](/concepts/flashback/investigations). You can also save searches to [Dashboard cards](/concepts/dashboards) for ongoing monitoring.


# Saved Searches

Store and reuse search queries in Spyderbat. Enable notifications and SIEM forwarding on saved queries to automate alerting and event delivery.

Saved Searches store your queries so you don't need to rebuild them each time. You can attach notification targets (email, Slack, PagerDuty, webhook) to a saved search and receive alerts whenever new matching records appear. You can also enable SIEM Forwarding to route matching events to your SIEM.

## Create a saved search

The Saved Searches panel is accessible from **Search** in the side panel.

### Example: monitor new cron jobs

1. **Run a query** — Enter a query, such as `metadata.name ~= "*"` for cron jobs. Click **Search** to confirm it returns results.

   <figure><img src="/files/8QqumTrvQPYQnnWY7ZcB" alt="Search bar with a cron job query entered"><figcaption><p>Run the query to verify it returns results before saving.</p></figcaption></figure>
2. **Save the query** — Click **Save Search**.

   <figure><img src="/files/KAtqRWuJLIc7zmR6InNp" alt="Save Search button in the search toolbar"><figcaption><p>Click Save Search to open the saved search configuration dialog.</p></figcaption></figure>
3. **Configure notifications** — In the dialog that appears:

   * Edit the auto-generated name if needed (e.g., "New Cron Jobs").
   * Add an optional description.
   * Toggle **Notification Status** to enabled if you want alerts immediately.
   * Click **Add Target** to configure notification channels. You can add multiple targets.

   <figure><img src="/files/bUprcKH5dxsUaTK7BgWi" alt="Saved search configuration dialog showing name, description, and notification status fields"><figcaption><p>Configure the name, description, and notification status for the saved search.</p></figcaption></figure>

   Supported notification channels: Email, Slack, PagerDuty, Webhook.

   <figure><img src="/files/gHdjA0wMje8xtrDSQnsI" alt="Notification target type selection showing Email, Slack, PagerDuty, and Webhook options"><figcaption><p>Select a notification channel and configure its destination.</p></figcaption></figure>
4. **Save** — Click **Save** to finish.

After saving, you can manage the query from the Saved Searches page: edit the query, run it manually, toggle it on or off, or delete it.

***

## SIEM Forwarding

Beyond notifications, saved searches can also forward matching events to your SIEM. Enable the **SIEM Forwarding** toggle in **Additional Settings** when editing a saved search — this requires the `org:ManageSiemForwarding` permission. Once enabled, Spyderbat routes records matching the query to the SIEM forwarding API, where the Event Forwarder picks them up and delivers them to your destination. Changes take effect immediately for new matching records — forwarding is not applied retroactively to records that existed before you enabled it.

See [SIEM Forwarding](/concepts/integrations/siem-forwarding) for the full setup guide.

***

## Manage saved searches with spyctl

The spyctl CLI uses `saved-query` as the resource name (for example, `spyctl get saved-queries`). This refers to the same thing as a saved search in the console UI — the terms are interchangeable.

### Retrieve saved searches

```bash
spyctl get saved-queries
```

### Create a saved search

```bash
spyctl create saved-query --help
```

```
Options:
  -o, --output [yaml|json|ndjson|default]
  -a, --apply                     Apply the saved query during creation.
  -n, --name TEXT                 The name of the saved query.
  -q, --query TEXT                The query to be saved.
  -d, --description TEXT          A description of the saved query.
  -s, --schema TEXT               The schema of the saved query.
  -y, --yes                       Automatically answer yes to all prompts.

Usage:
  spyctl create saved-query [OPTIONS]
```

Example:

```bash
spyctl create saved-query \
  -n "Monitor Deployment with Replicas more than 5" \
  -q "spec.replicas > 5" \
  -s "Deployment"
```

To list all available schemas:

```bash
spyctl search --list-schemas
```

### Edit a saved search

```bash
spyctl edit saved-query <NAME_OR_ID>
```

Replace `<NAME_OR_ID>` with the ID or name of the saved search you want to edit. After editing the YAML and applying the change, you'll see a confirmation: `Successfully edited Saved Query 'query:id'`.


# Summarize


# Spydertrace Summarize

## Overview

Spyderbat's **Summarize** feature provides a quick, structured summary of a **Spydertrace** investigation, enabling users to understand key details without manually analyzing the trace. This feature enhances threat detection efficiency and streamlines the investigative process.

> **Note:** Summarize is available only on an opt-in basis per organization. It requires approval to send data to OpenAI. To enable the feature, navigate to **Admin → Organization Management → AI Management**. Here, you can opt in or out, track your **monthly usage quota**, and view the **Recent Summarize Usage Log**.

![](/files/J8Wpy3F2b2TJTFKuPc62)

By default, a monthly quota of 50 is provided, with each trace summary consuming one. You can contact us to request an increase.

## What is Summarize?

The **Summarize** feature in Spyderbat generates a concise summary of a **Spydertrace**, highlighting critical security insights.

Behind the scenes, it takes the Spydertrace as input, sends it to OpenAI, and generates a concise, easy-to-understand summary.

## How to Use Summarize

There are two ways to generate a summary: **Manual** and **Automatic**.

### 1. Manual Summarization

To manually summarize a **Spydertrace**, click the `Summarize` button. The summary generation process may take a few seconds.

#### Example 1: Search

* **Search** for the relevant **Spydertrace**.
* If you find a high-score **Spydertrace** in a restricted cluster, and want to quickly understand its details, click **Summarize** to generate a summary instantly.

![](/files/ApnBuVrulaWBpTzFas1R)

#### Example 2: Investigation

* Within the **Spyderbat Investigation** view, click **Summarize** on top-right to generate a summary.

![](/files/wD4fzeAqbTfSVbnqI5AK)

* Based on the insights, take immediate action as needed.

![](/files/5r7CfVRikR7QPLv7kNFt)

### 2. Automatic Summarization

Automatic summarization enables AI-powered summary generation for every **Spydertrace saved search**.

* When enabled, the system **automatically** generates structured summaries for saved **Spydertrace** investigations.

#### Example:

If you want a summary for every high-score Spydertrace (e.g., score 100), follow these steps:

* Search for the high-score Spydertrace.

![](/files/pRBV8bQneMGwwu08OliK)

* Add it to a saved search.
* Add description, target as desired.
* In Additional Settings, enable Auto AI Summarization and Save.

![](/files/UahypPFtrcoM1WOWJp4x)

Once enabled, every time a high-score Spydertrace occurs, you will receive a notification with an investigation link to review the Spydertrace. With automatic summarization, you don't have to wait for the summary to generate—it is ready instantly.

> Note: Only enable Automatic AI Summarization based on your organization's quota.

You can also view summarized traces in AI Management's Recent Logs.

### Benefits of Summarize

#### ⏳ Time Efficiency

Reduces manual effort in analyzing complex security traces.

#### ⚡ Quick Incident Response

Enables security teams to respond faster with key insights readily available.

#### 🔍 Improved Security Insights

Highlights critical security concerns such as unauthorized access, suspicious executions, and potential breaches.

#### 📑 Simplified Investigation

Provides a structured view of incidents, aiding forensic analysis and remediation planning.

## Conclusion

Spyderbat’s **Summarize** feature enhances security investigations by providing **automated, structured, and insightful** summaries of activities. By leveraging this feature, security teams can **quickly detect, understand, and mitigate** potential


# Dashboards

Detailed overview of the Dashboard section of the console, including collected types of data, data management (sorting, filtering and grouping), and shortest path to investigating suspicious activity.

<mark style="color:blue;">Published: July 20, 2023</mark>

The Dashboard section of the Spyderbat UI is located at the top of the left hand navigation panel, as shown below. If there is at least one source configured in the Spyderbat UI for your organization, you will be directed to the Dashboard homepage upon successful login into the console. If you have not yet set up any Sources (data collection) within your monitoring scope, please refer to our Documentation portal to access one of our [How-To Guides for Spyderbat Nano Agent Installation](/installation/spyderbat-nano-agent).

Dashboard section provides a consolidated at-a-glance overview of a variety of operational and security data points captured as a result of asset monitoring with active Spyderbat Nano Agents.

### **Dashboard Card Overview**

The Dashboard section comprises several default groups of dashboard cards. Each individual dashboard card represents a structured output of a Athena search query crafted using a set of criteria set forth by Spyderbat security analysts.

![Click to enlarge](/files/MlPt8T085tDC8MxmHFLL)

As you can see, all dashboard cards are of the same default height, which means that there are only so many rows that can be displayed within the card even with the scroll bar. Spyderbat dashboard cards surface the top 100 rows, and indicate the total number of rows that meet the dashboard card criteria in the dashboard card header.

![](/files/snQ6pKhxW5h6azjpjpdZ)

If you need to view or export all the data, you could do it through Search, by clicking “view all \[total number]” or “view first 10K”, if there are more than 10K of rows being returned. In the latter case, it is highly advisable to apply additional search or filtering criteria to reduce the volume of data, which we will cover here shortly.

### **Adjusting Card Display**

Within a session you can resize cards, reorder or hide columns, and filter or sort data. These changes reset on page refresh — they do not persist across sessions.

Each card also has a time range selector (default: 24 hours, options from 1 hour to 30 days).

For persistent customization, hover over a card and click **Run in Search** to open the underlying query in Search, where you can modify it and save it as a [custom dashboard card](/tutorials/how-to-create-custom-dashboards).

### **Data Grouping**

In addition to filtering and sorting the data within the card, some dashboard cards allow grouping the data into summary rows by column values. By default, several cards have been selected by Spyderbat analysts to have Grouping feature enabled and all data grouped based on the specific criteria called out in the first column:

![](/files/GCrVkBhckYy765seqczp)

You can expand a select grouping by clicking on the accordion symbol:

![](/files/B1tUT2MSb1CNC3cO9umP)

If you turn off grouping by moving the slider on the right from “Grouping Enabled” to “Grouping Disabled”, all rows will be displayed in an unsorted order.

When “Grouping Enabled” is on, you can also apply nested grouping options based on the values within other columns, by clicking the ellipsis (three vertical dots), on the column which values you wish to use for the nested rows grouping, and select “Group by \[column name]”:

![](/files/6UolwtnBeX3bUxiHEJJf)

![](/files/f0dNJNBf7HPgXYtXN9wt)

To remove nested grouping, you will have to follow the same steps and choose “Stop Grouping by \[column name]” from the drop down. To remove all grouping, just flip the “Grouping Enabled” slider to “Grouping Disabled”:

![](/files/0jN2ixO7nYMl9AnxQEm4)

### **From Dashboards to Investigation**

Besides offering you extensive observability options and holistic view of your security posture, Dashboard cards allow you to easily segway into investigating any suspicious or simply interesting activity in your monitored environment. All you need to do to start an investigation is select one or more rows in one or multiple dashboard cards and click “Start Investigation”.

![](/files/Jne8dJsRSMQSnezLXbaD)

Clicking the X in the “Start Investigation” pop-up, will automatically deselect all rows.

{% hint style="info" %}
For an overview of the Investigation view, see [Investigations](/concepts/flashback/investigations). For a hands-on guide, see the [Investigations Tutorial](/tutorials/flashback/how-to-investigate-in-spyderbat).
{% endhint %}

At any time during your investigation you can go back to the dashboards section to add more items to your existing investigation or start a brand new investigation:

![](/files/A3IOx8RpHeombGIqJTuM)

If you choose to start a new investigation, the existing open investigation will get overwritten, unless you save an Investigation Link.

If you are focusing your investigation on K8s assets and inventory, rather than processes, the system will prompt you to run a K8s investigation.

![](/files/xBEYw3rWKeiF9PmJ4RVm)

{% hint style="info" %}
To learn more about Spyderbat Investigation section and how to navigate it, please refer to our [Investigations Tutorial](/tutorials/flashback/how-to-investigate-in-spyderbat).
{% endhint %}


# Dashboard Categories

The seven built-in dashboard categories in Spyderbat: Security, User Tracking, Policy, Operations, Network, Inventory, and Kubernetes. Includes the full list of default cards in each category.

Spyderbat organizes default dashboard cards into seven categories. Each category groups cards by monitoring objective — security detections, user activity, policy violations, network traffic, and so on.

The card selection in each default category is fixed. You can't add, remove, or modify cards in built-in categories. If you need a different set of cards, create a custom dashboard. Users with [appropriate permissions](/getting-started/spyderbat-user-and-role-management-overview) can create custom dashboards and organize them into custom categories. See [how to create custom dashboards](/tutorials/how-to-create-custom-dashboards) for instructions.

<figure><img src="/files/Aq5Q62kLTBfz1ga4bR2e" alt="The Security dashboard category showing five cards: Recent Spydertraces with Score > 50, Suppressed Spydertraces, Recently Observed Listening Sockets, Recent Critical and High Severity Security Flags, and Processes Executed Out of /tmp"><figcaption></figcaption></figure>

## Security

The Security category surfaces detections and suspicious activity relevant to SecOps workflows. It contains five cards:

* **Recent Spydertraces with Score > 50** — Spydertraces are causal graphs of related activity on a monitored node or container. This card shows traces where the aggregate risk score exceeds 50, indicating a combination of detections worth investigating.
* **Suppressed Spydertraces** — Spydertraces that matched a Guardian policy and were suppressed rather than alerted on. Useful for confirming expected behavior is being correctly ignored, or for auditing suppression coverage.
* **Recently Observed Listening Sockets** — Open ports and listening sockets across monitored hosts. Exposed services are a common attack surface, particularly when misconfigured or unpatched.
* **Recent (Critical and High Severity) Security Flags** — Point-in-time detections generated from MITRE ATT\&CK scenarios, Spyderbat's analytics, and any third-party sources you've configured (such as [Falco](/tutorials/integrations/falco-connector-deployment)).
* **Processes Executed Out of /tmp** — The `/tmp` directory is world-writable and frequently targeted by malware that needs to write and execute files without requiring elevated permissions.

## User tracking

The User Tracking category focuses on interactive user activity, which is often a signal of either legitimate privileged access or an active threat. It contains four cards:

* **Interactive User Spydertraces** — Causal traces of activity started by a foreground process under user control in a terminal session.
* **Interactive User Sessions** — Interactive processes and the effective users that triggered them.
* **Interactive User Sessions with Privilege Escalation** — Interactive sessions where the effective user changed (for example, via `sudo` or `su`) at any point in the activity chain.
* **Interactive Shell Inside a Container** — An interactive shell in a container is an anti-pattern in production environments. Its presence can indicate unauthorized access or an active intrusion.

## Policy

{% hint style="info" %}
This category requires Guardian policies to be configured and active on your monitored workloads. See [Guardian policies](/concepts/guardian) for setup instructions.
{% endhint %}

The Policy category shows Guardian findings — deviations from expected behavior as defined by your applied policies. It contains three cards:

* **Container Policy Deviation Spydertraces** — Causal traces of activity triggered by a policy violation inside a monitored container.
* **Container Policy Deviation Flags** — Individual point-in-time detections from policy violations in containers.
* **Linux Service Policy Deviation Flags** — Individual detections from policy violations on Linux VMs, scoped to background services.

## Operations

The Operations category currently contains one card:

* **Operations Flags** — Infrastructure management and uptime events, such as a pod not running, memory management features being disabled, or critical resource thresholds being exceeded.

## Network

The Network category provides visibility into connection activity across monitored hosts. It contains five cards:

* **Long Lived (> 60 mins) Egress Connections** — Outbound connections that remain active for more than an hour. Persistent connections to unexpected destinations can indicate beaconing or data staging.
* **Egress Connections with Large (>1MB) Data Transfer** — Outbound connections with significant data volume. Useful for detecting accidental or intentional data exfiltration.
* **Cross Machine Connections** — Connections between monitored machines. Unusual lateral movement between hosts is a common indicator of compromise.
* **Connections to DNS** — DNS activity across monitored hosts. Anomalous DNS queries can indicate command-and-control communication, network reconnaissance, or malware downloads.
* **Connections Initiated by an SSH Process** — Outbound SSH connections. While SSH is a legitimate protocol, connections to unexpected destinations or made by unexpected processes are worth reviewing.

## Inventory

The Inventory category lists the assets currently observed in your monitoring scope. It contains five cards:

* **Recently Observed Systems** — Linux hosts running the Nano Agent.
* **Recently Observed Kubernetes Clusters** — Kubernetes clusters within your monitoring scope.
* **Recently Observed Kubernetes Nodes** — Nodes across monitored clusters.
* **Recently Observed Pods** — Pods across monitored clusters.
* **Recently Observed Containers** — Containers across monitored clusters.

Each card links to asset metadata and lets you pivot into activity for that asset within a selected time range. If you deploy the Nano Agent as part of your base image or Kubernetes automation, new machines appear here as soon as they come online. See [Nano Agent installation](/installation/spyderbat-nano-agent) for deployment options.

## Kubernetes

The Kubernetes category extends the inventory view with additional resource types and command activity. It contains ten cards:

* **Recently Observed Clusters**
* **Recently Observed Kubernetes Nodes**
* **Recently Observed Pods**
* **Recently Observed Containers**
* **Recently Observed Services**
* **Recently Observed Deployments**
* **Recently Observed Replicasets**
* **Recently Observed Daemonsets**
* **Executed "Kubectl delete" Commands** — `kubectl delete` commands run against monitored clusters within the selected time range.
* **Executed "Kubectl apply" or "Kubectl create" Commands** — `kubectl apply` and `kubectl create` commands run against monitored clusters. Review these to confirm that cluster changes are authorized and expected.


# Reports

Overview of the Reporting section of the console, including report creation, review, download and printing.

Spyderbat provides robust reporting capabilities to help monitor and analyze the operations and security of monitored Linux machines and Kubernetes clusters.

The reporting features are accessible via the "Reports" menu of the portal.

![](/files/jCSYls1dttThzbYsbp7H)

The **Reports** section contains two submenus:

* **Generated**: Review and manage reports generated based on your input parameters.
* **Create**: Create new reports based on a predefined inventory of available report types.

Reports can be customized with a variety of input parameters and exported in multiple formats, including JSON, YAML and PDF. Once generated, reports are stored in the "Generated" section for review, export, and printing.

### **Creating a Report**

To create a report:

1. Navigate to the **Reports** section of the portal and click on the **Create** menu item.
2. You will see a list of available report templates that you can select from.

![](/files/ruawXFWYVqGbGQX2b0uA)

Review the descriptions of the reports, and use the preview button to preview a sample of the report type.

3. Select the desired report type from the list.
4. Each report type has specific input parameters you must provide to customize the report for your environment, and specific UI controls to select them.

![](/files/eTUtXfpNruKmmdJybZMs)

Enter or select the required parameters, such as:

* **Cluster**: The Kubernetes cluster for which the report is generated.
* **Start Time** and **End Time**: Defines the reporting period (e.g., last 24 hours or specific time range).

You can give your report a specific name to make it easier to locate it later.

Other report types may have other selectors, such as machine selectors.

5. Click **Create** to initiate the report generation process.

Reports may take several minutes to generate, depending on the size of the system and the selected time range. The UI will display a popup at the bottom of the page titled 'Creating report,' with a 'View' link that directs you to the 'Generated' section.

### **Viewing Generated Reports**

All created reports will appear immediately in the Generated Reports section. Some reports take a while to process and render, and will not be immediately available to view. When they are ready, the 'Published' column will change from 'Scheduled' to the time and date when the report was published, and a 'View' button will be available to allow the user to view the report.

![](/files/KyAxWJbgs999K7Iaq2M2)

Select the report you want to access and click on its 'View' button. The report will render in a pop-up, like so:

![](/files/QD4NLLhgZQLMfddzZ1iX)

From here you can

1. **Inspect the report**: Scroll down to see the full report contents if needed
2. **Download Report**: Export the report in any of the available formats (e.g., JSON, YAML). By clicking on the 'Download' button in the bottom right corner.
3. **Export to PDF and print**: Use the print to PDF button to export and/or print the report in PDF format using your browser capabilities and preferred settings.


# Notifications

Get notified when Spyderbat detects operations issues or suspicious behavior at runtime in your environment.

Spyderbat's notification system has 3 main components:

* [**Notification Targets**](/concepts/notifications/notification-targets): Named destinations to where notifications can be sent.
* [**Notification Templates**](/concepts/notifications/notification-templates): Templates that define the structure and content of notifications, simplifying the setup process.

### Notifiable Objects:

Spyderbat allows users to set up Notifications for the below to stay informed about important events in their Spyderbat Organization.

Here are 3 types of notifiable objects:

#### 1. Saved Queries

**What it is**: Predefined searches to track specific patterns or behaviors in your data.

**Why it’s useful**: Automates monitoring by notifying you when new activity matches the query.

**Example**: Get notified when there’s unusual inbound connection.

#### 2. Custom Flags

**What it is**: Custom flags enable users to create tailored detection rules to monitor activities or behaviors specific to their environment.

**Why it’s useful**: Helps focus on what matters to you, like unusual commands or risky actions.

**Example**: Flag and alert when someone runs a command that requires high privileges.

#### 3. Agent Health Notifications

**What it is**: Alerts about the health and status of Spyderbat agents.

**Why it’s useful**: Ensures agents are functioning properly and sending data.

**Example**: Get notified if an agent goes "Offline" or to "Critical" state.

Note: To Learn How to Configure Notifications for Agent Health using Spyctl [Refer here](/tutorials/notifications/how-to-set-up-agent-health-notifications-in-spyctl)

### Quick Start Tutorial

To quickly get started using using Spyderbat Notifications follow our tutorial using spyctl.

[How to setup Spyderbat Notifications (Spyctl CLI)](/tutorials/notifications/how-to-set-up-spyderbat-notifications-spyctl)


# Notification Targets

## What are Notification Targets?

Notification Targets are named destinations where notifications can be sent.

You create Notification Targets to receive notifications via email, Slack, AWS SNS, and webhook. You specify which Notification Targets to use when creating Notification Configurations.

## Using Notification Targets

Notification Targets can be referenced while configuring notifications for Notifiable Objects using Spyctl. You can either specify a Notification Target or a Notification Template that map specific targets to templates like below.

**Example usage with Spyctl**:

Example:

```yaml
spyctl notifications configure saved-query QUERY_UID \
  --target TARGET_NAME_OR_UID \
  --target-map TARGET_NAME_OR_UID=TEMPLATE_NAME_OR_UID

```

Usage:

```yaml
spyctl notifications configure saved-query query:abc \
  --target OperationsTeam \
  --target-map SecurityTeam=email-template \
```

The `$spyctl notifications configure` command allows notifications to be sent using either Custom templates with Targets or directly via Targets (using Default Template).

## Types:

### Emails

Email Notification Targets contain a list of email address destinations for notifications.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTarget
metadata:
  name: Example
  type: email
spec:
  emails:
  - engineer1@example.com
  - engineer2@example.com
```

### Slack

Slack Notification Targets contain a single Slack Hook URL destination for notifications.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTarget
metadata:
  name: Example
  type: slack
spec:
  url: https://hooks.slack.com/services/xxxxxxxxxxx/xxxxxxxxxxx/xxxxxxxxxxxxxxxxxxxxxxxx
```

### Webhook

Webhook Notification Targets contain a single generic webhook URL destination for notifications.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTarget
metadata:
  name: Example
  type: webhook
spec:
  url: https://my.webhook.example/location/of/webhook
```

### PagerDuty

PagerDuty Notification Targets contain a single routing key used to send notifications to a specific PagerDuty service.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTarget
metadata:
  name: Example
  type: pagerduty
spec:
  routing_key: abcdef1234567890abcdef1234567890

```

## Manage Notification Targets Using Spyctl

To start creating Targets follow our tutorial using spyctl :: [Manage Notification Targets Using Spyctl](/reference/spyctl/notification-targets-management-spyctl)

### Quick Start Tutorial

If you already have Targets set, start configuring Spyderbat Notifications using spyctl.

[How to setup Spyderbat Notifications (Spyctl CLI)](/tutorials/notifications/how-to-set-up-spyderbat-notifications-spyctl)


# Notification Templates

## What are Notification Templates?

Notification Templates define the format and content of notifications sent to different destinations. They allow customization of messages based on the notification type.

You create Notification Templates for email, Slack, PagerDuty, and webhook notifications. These templates can be used when configuring notifications alongside Notification Targets.

Note: Notification Templates are Optional when configuring Notifications.

## Using Notification Templates

Notification Templates can be referenced while configuring notifications for Notifiable Objects using Spyctl. You can either specify a Notification Target or a Notification Template that map specific targets to templates like below.

**Example usage with Spyctl**:

Example:

```yaml
spyctl notifications configure saved-query QUERY_UID \
  --target-map TARGET_NAME_OR_UID=TEMPLATE_NAME_OR_UID
```

Usage:

```yaml
spyctl notifications configure saved-query query:abc \
  --target-map OperationsTeam=email-template \
  --cooldown 300
```

The `$spyctl notifications configure` command allows notifications to be sent using either Custom templates with Targets or directly via Targets (using Default Template).

## Types:

Note: Below examples shows YAML Templates, but they can also be generated in JSON format.

### Email

Email Notification Templates define the subject and body format for email notifications.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTemplate
metadata:
  name: email-template
  type: email
spec:
  subject: ''
  body_html: ''
  body_text: ''
```

**Note**: Users must populate subject, body\_html, and body\_text using placeholders to customize the email content.

```yaml
spec:
  subject: "Spyderbat Alert: {{ severity }} Severity Detected"
  body_html: "<p>Alert triggered at {{ __hr_time__ }}</p><p>Details: {{ description }}</p>"
  body_text: "Alert triggered at {{ __hr_time__ }}. Details: {{ description }}"
```

### Slack

Slack Notification Templates define the message structure for Slack notifications. Notification templates can be generated in YAML or JSON format as desired.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTemplate
metadata:
  name: slack
  type: slack
spec:
  text: ''
  blocks: []
```

After populating template:

```yaml
apiVersion: spyderbat/v1
kind: NotificationTemplate
metadata:
  name: slack
  type: slack
spec:
  text: "Alert: {{ severity }} - {{ description }}"
  blocks:
    - type: section
      text:
        type: mrkdwn
        text: "*Alert Triggered at:* {{ __hr_time__ }}\n*Details:* {{ description }}"
```

### Webhook

Webhook Notification Templates define the payload structure for webhook notifications.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTemplate
metadata:
  name: webhook-template
  type: webhook
spec:
  payload: {}
  entire_object: false
```

After populating template:

```yaml
spec:
  payload:
    severity: "{{ severity }}"
    details: "{{ description }}"
    timestamp: "{{ __hr_time__ }}"
```

### PagerDuty

PagerDuty Notification Templates define the format for alerts sent to PagerDuty.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTemplate
metadata:
  name: pg
  type: pagerduty
spec:
  class: null
  component: null
  source: ''
  summary: ''
  severity: ''
  dedup_key: null
  custom_details: {}
  group: null
```

After populating template:

```yaml
spec:
  summary: "Spyderbat Saved Query '{{ __query_name__ }}' Matched"
  source: "{{ __source__ }}"
  severity: "{{ __pd_severity__ }}"
  custom_details: 
    "description": "{{ description }}"
    "cluster": "{{ __cluster__ }}"
    "time": "{{ __hr_time__ }}"
    "linkback": "{{ __linkback__ }}"
```

Use these templates to ensure consistent and structured notifications across different channels.

## Placeholder Fields and Dynamic Variables

Some fields in Notification Templates are dynamically calculated and replaced at runtime using placeholders. These placeholders allow real-time data insertion into notification messages.

### Understanding Placeholder Fields

Placeholder fields allow dynamic values to be inserted into notification templates. These fields are replaced with actual data when a notification is sent.

They're represent with syntax: `__field__`

**Some Common Spyderbat Internal Placeholder Fields are:**

{{ \_\_source\_\_ }} - Source of the event

{{ \_\_cluster\_\_ }} - Cluster where the event occurred

{{ \_\_hr\_time\_\_ }} - Human-readable timestamp

{{ \_\_linkback\_\_ }} - Link to view the event in Spyderbat

{{ \_\_time\_int\_\_ }} - Timestamp in integer format

{{ \_\_origin\_\_ }} - Origin of the event

{{ \_\_hostname\_\_ }} - Hostname where the event occurred

{{ \_\_percent\_\_ }} - Percentage value related to the event

{{ \_\_pd\_severity\_\_ }} - Severity level formatted for PagerDuty

{{ \_\_query\_name\_\_ }} - Name of the saved query that triggered the event

Example Usage in Email Body:

```
<p>Spyderbat Custom Flag "{{ custom_flag_name }}" Emitted</p>
<ul>
    <li>Cluster: {{ __cluster__ }}</li>
    <li>Source: {{ __source__ }}</li>
    <li>Time: {{ __hr_time__ }}</li>
</ul>
<p>{{ description }}</p>
<p><a href="{{ __linkback__ }}">View in Spyderbat</a></p>
```

### Dereferencing Values from the Object:

Static fields or regular placeholders ({{ severity }}, {{ description }}) are fields that are passed directly from the model object. Static text remains unchanged and does not need placeholders.

* {{ severity }} - Severity level of the event
* {{ description }} - Description of the event

By customizing Notification Templates with placeholders, users can ensure notifications provide meaningful and actionable information tailored to their needs.

**To learn more about Placeholder fields and Constructing templates** [**Read this**](https://github.com/spyderbat/spyderbat-docs/tree/main/concepts/notifications/reference/notifications/README.md)

## Conclusion

By following this guide, you can create well-structured, dynamic Notification Templates for different destinations. Using placeholders correctly ensures your notifications contain relevant, real-time data.

## Manage Notification Templates Using Spyctl

To start creating Templates follow our tutorial using Spyctl : [Manage Notification Templates Using Spyctl](/reference/spyctl/notification-template-management-spyctl)

## Quick Start Tutorial

To quickly get started using using Spyderbat Notifications follow our tutorial using spyctl.

[How to setup Spyderbat Notifications (Spyctl CLI)](/tutorials/notifications/how-to-set-up-spyderbat-notifications-spyctl)


# Actions

Overview of manual response actions that can be executed in the UI, including killing a process or killing a pod.

The Spyderbat platform features include a powerful response capability: the option to kill processes on Linux machines or pods within Kubernetes clusters directly through the UI. This feature empowers security teams to take immediate action during an investigation, stopping active threats as soon as they are identified.

Whether dealing with malicious processes on a machine or a compromised pod within a Kubernetes cluster, users can mitigate the threat swiftly and efficiently.

This capability ensures real-time responsiveness by terminating the identified threat within seconds. The action is recorded in an audit log for accountability and compliance purposes.

### Who Can Kill a Process or Pod?

Killing a process or pod requires specific permissions for the logged in user. By default these permissions are preset for the following Spyderbat Roles:

* Admin
* Power User

Roles for users can be assigned or modiefied in the Admin section, Organization management. For more details, see [User and role management overview](/getting-started/spyderbat-user-and-role-management-overview)

### How to Kill a Process or Pod

#### 1. Initiating an Investigation

The process starts with an investigation, which can be the result of pivoting from a high scoring trace, or a security redflag, drilling down from a dashboard card, or just from the output of the search results to find specific proccess or kubernetes resources of interest.

Once you have identified the process of interest in the Investigation section of the UI, you can take action.

In this example, we identified a netcat server running on port 9000

![](/files/YAGNr45m1jO2byxZExnh)

#### 2. Taking the Kill Action

The kill action can be taken either at the bottom of the graph view, or within the process details view. You can opt to kill the process, or, if the process is running in a container from a Kubernetes cluster, to kill the entire pod.

#### 3. Providing a Reason, and Confirming the Kill Action

After selecting the "Kill Process" action, a confirmation dialog will appear, prompting you to provide a reason for the action. This reason will be recorded in the audit log for accountability and future reference. Input a clear and concise reason for the kill action, for example "Terminating malicious process" or "Shutting down compromised pod."

![](/files/tP7fBtiq4x22Cay8Tq9d)

To prevent accidental terminations, Spyderbat requires confirmation before executing the kill action. After entering your reason, click 'YES, KILL PROCESS' to proceed with the process termination.

The process for killing a pod is exactly the same, just select Kill Pod and follow the confirmation prompt.

![](/files/cn96EMOO8o1V9mUJjTFP)

#### 5. Action Execution

Once confirmed, Spyderbat will automatically terminate the process or pod within seconds. A popup at the bottom-page will provide confirmation.

#### 6. Review and Audit Logging

After a process is killed, its icon will be updated to reflect it is now defunct, and the process details will contain action audit log information

![](/files/flXK75HOMroO0Jxencqw)

Every kill action, including the reason for the termination and user details, is recorded in Spyderbat’s comprehensive audit log. You can review this log to track all interventions taken during an investigation.

You can find a full log of all actions taken in your account by navigating to the Reports, Action Log section

![](/files/pkgpv9j7ITPdsmsdx9jw)

In the actions log you will find what type of action was taken, the action status, when it was created, the action result code, who took the action, the reason provided and what process or pod was impacted.

You can filter for specific actions you are looking for by clicking on "Filters", and adjust the columns view by clicking on "Columns".

#### 7. Next Steps

After the kill action, you should continue to identify root-cause of the unexpected behavior you chose to terminate.

It is possible another process is still active that could respawn the same type of process you just killed, so reviewing processes again would be a great idea.

Similarly, if you chose to kill the pod, be mindful a new pod might be automatically created by the cluster, exhibiting the same threat you tried to eliminate. Confirm that the behavior you want to see addressed is not introduced by a higher-up kubernetes resource manager, such as a deployment or statefulset that was compromised.


# Integrations

Integrate Spyderbat with your existing security infrastructure — route events to your SIEM, connect AWS accounts, and forward data to external tools.

Spyderbat integrates with your security stack in two ways.

| Integration                                               | What it does                                                                                                                                         |
| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| [SIEM Forwarding](/concepts/integrations/siem-forwarding) | Route events matching saved queries to your SIEM — via the [Event Forwarder](/concepts/integrations/spyderbat-event-forwarder) or direct API polling |
| [AWS Integration](/concepts/integrations/aws)             | Connect Spyderbat to your AWS environment for cloud-native visibility                                                                                |


# SIEM Forwarding

How Spyderbat SIEM forwarding works — enabling forwarding on saved queries and deploying the Event Forwarder to deliver events to your SIEM.

{% hint style="success" %}
Just want to get started? See the [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart).
{% endhint %}

SIEM forwarding in Spyderbat is a two-part system. First, you enable forwarding on one or more saved searches (also called saved queries in the API and spyctl CLI), which tells Spyderbat which events to collect. Second, you deploy the Event Forwarder (or poll the API directly), which retrieves those collected events and delivers them to your SIEM. The Event Forwarder only receives events that a saved search has already selected for forwarding — without step one, step two delivers nothing. You can install the Event Forwarder before or after enabling forwarding on a saved search — the forwarder polls continuously and picks up new forwarding-enabled queries automatically.

## How it works

```mermaid
flowchart TD
    A["Saved query (SIEM Forwarding: enabled)"] -->|record matches| B["Buffered events"]
    B -->|Event Forwarder polls| C["Event Forwarder"]
    B -.->|or direct API polling| E["Your SIEM"]
    C --> D["File / Webhook / Stdout / Syslog"]
    D --> E
```

When an incoming record matches a saved search that has SIEM forwarding enabled, Spyderbat buffers that record for your organization. The Event Forwarder polls for new records, enriches each one with a `runtime_details` object containing host metadata, and writes to your configured destination. You can also skip the Event Forwarder and poll the API directly, but this requires custom scripting to manage the cursor.

## Step 1: Enable forwarding on a saved query

If you don't have a saved query yet, [create one first](/concepts/search/saved-search). Then enable the **SIEM Forwarding** toggle in **Additional Settings** when editing the query. Scroll to the bottom of the edit dialog to find **Additional Settings** — this section may not be visible without scrolling. Forwarding takes effect immediately for new matching records — it does not retroactively forward records that were created before forwarding was enabled. No forwarder restart is needed. You can enable it on as many queries as you want, across any Spyderbat schema. If a record matches multiple forwarding-enabled queries, it is forwarded once.

**Example queries:**

| Intent                   | Schema              | Filter              |
| ------------------------ | ------------------- | ------------------- |
| All Spydertraces         | `model_spydertrace` | `*`                 |
| High-scoring traces only | `model_spydertrace` | `score > 50`        |
| All security flags       | `event_redflag`     | `*`                 |
| High-severity flags only | `event_redflag`     | `severity = "high"` |
| All network connections  | `model_connection`  | `*`                 |

{% hint style="info" %}
The Schema values above (`model_spydertrace`, `event_redflag`, etc.) are the API and spyctl identifiers. The search UI uses friendlier display names — for example, the UI shows "Spydertrace" for `model_spydertrace`. Select the matching display name from the schema dropdown when building queries in the console.
{% endhint %}

<figure><img src="/files/NIzE9jVGXYMWokwoMtXE" alt="Saved search edit dialog with Additional Settings expanded showing the SIEM Forwarding toggle"><figcaption><p>Enable SIEM Forwarding in Additional Settings when editing a saved search.</p></figcaption></figure>

To confirm forwarding is enabled, open **Saved Searches**: expand the **Search** icon in the sidebar, then click **Saved Searches**. Scroll right in the table to the **SIEM Forwarding** column — it should show a checkmark (✓) for your query. Alternatively, click the **SIEM Forwarding** filter button at the top of the table to show only SIEM-enabled queries.

To disable forwarding, edit the saved search and uncheck **SIEM Forwarding** in **Additional Settings**. Changes take effect immediately without restarting the forwarder. The forwarder continues running and delivers events from other forwarding-enabled queries in your org.

For step-by-step instructions, see the [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart#step-1-enable-siem-forwarding-on-a-saved-query). Requires the `org:ManageSiemForwarding` permission.

## What gets forwarded

The API returns raw matching records, delivered as-is. If you poll the API directly (without the Event Forwarder), you receive raw records without `runtime_details` enrichment. If you use the Event Forwarder, it enriches each record with host metadata before writing to your destination. See [runtime\_details enrichment](/concepts/integrations/spyderbat-event-forwarder#runtime-details-enrichment) for the injected fields.

## Step 2: Deploy the Event Forwarder

The [Event Forwarder](https://github.com/spyderbat/event-forwarder) is the recommended way to deliver events to your SIEM. It polls the API, enriches records with host metadata, and writes to files, stdout, syslog, or HTTP webhooks. It persists its position to disk, so it resumes where it left off across restarts.

For architecture details, see [Spyderbat Event Forwarder](/concepts/integrations/spyderbat-event-forwarder). Install guides:

* [Kubernetes — Helm Chart](/installation/spyderbat-event-forwarder/helm-chart)
* [Linux — Traditional Installer](/installation/spyderbat-event-forwarder/traditional-installer)

## Choosing a delivery method

| Approach           | Best for            | Tradeoff                                                                                                              |
| ------------------ | ------------------- | --------------------------------------------------------------------------------------------------------------------- |
| Event Forwarder    | Most deployments    | Adds `runtime_details` enrichment, handles cursor management and retries. Requires deploying an additional component. |
| Direct API polling | Custom integrations | No extra component to deploy. Requires scripting to manage the cursor and handle errors. No host metadata enrichment. |

## Accessing the API directly

You can poll the API endpoint directly and skip the Event Forwarder. You won't get `runtime_details` enrichment, but you don't need to deploy an additional component. This approach requires custom scripting to manage the cursor — most SIEMs don't support cursor-based pagination natively.

Your first call omits the cursor (or passes `OLDEST` / `LATEST`):

```http
GET https://{api-host}/api/v1/org/{your-org-uid}/events
Authorization: Bearer <your-api-token>
```

Replace `{api-host}` with your Spyderbat API hostname (visible in your browser when using the console). For API key setup, see [How to Create and Use a Spyderbat API Key](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api).

The response is newline-delimited JSON (ndjson). Event records come first, followed by a final line containing the cursor for your next call:

```
{"schema":"model_process","id":"proc:abc123","muid":"mach:xyz789","name":"/usr/bin/curl","time":1719500000.0}
{"schema":"model_connection","id":"conn:def456","muid":"mach:xyz789","server_port":443,"time":1719500001.0}
{"iterator":"eyJvcmdfdWlkIjoib3JnOnNweWRlcmJhdCIsIm9mZnNldCI6MTcxOTUwMDAwMH0"}
```

Pass the returned cursor on subsequent calls:

```http
GET https://{api-host}/api/v1/org/{your-org-uid}/events/{cursor}
Authorization: Bearer <your-api-token>
```

**Cursor values:**

| Value           | Behavior                                                 |
| --------------- | -------------------------------------------------------- |
| *(omitted)*     | Same as `OLDEST`                                         |
| `OLDEST`        | Start from the earliest buffered record                  |
| `LATEST`        | Start from now (skip historical backlog)                 |
| Returned cursor | Continue from the position returned by the previous call |

**Query parameters:**

| Parameter | Description                                                      |
| --------- | ---------------------------------------------------------------- |
| `limit`   | Number of records per call. Range: 1,000–10,000. Default: 1,000. |

Pagination is forward-only. You cannot seek backward to a previous position.

## Related pages

* [Saved Searches](/concepts/search/saved-search)
* [Spyderbat Event Forwarder](/concepts/integrations/spyderbat-event-forwarder)
* [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart)
* [Panther Webhook Configuration](/tutorials/integrations/forwarder-panther-config)
* [Install: Helm Chart](/installation/spyderbat-event-forwarder/helm-chart)
* [Install: Traditional Installer](/installation/spyderbat-event-forwarder/traditional-installer)


# AWS Integration

Overview of the AWS Context Integration using the AWS Agent

## Role of the Spyderbat AWS Agent in the Spyderbat Platform

The Spyderbat platform is designed to provide a central, detailed contextual view of monitored assets. To achieve this, we start by gathering accurate data regarding the assets using our agent technology.

We then apply contextual and detection analytics to model this context, which forms the core of the platform. To communicate this context, along with security and operational insights, the platform provides a user interface for investigations (backed by an integration API), as well as Reports, Notifications and Actions.

To provide an integrated view, we collect various types of information:

* **Machine information**: This includes data such as processes, listening sockets, connections, and more, collected by the Spyderbat Nano Agent using eBPF technology.
* **Kubernetes orchestration information**: This covers details about active deployments, services, and pods in the cluster, gathered by the Spyderbat ClusterMonitor through the Kubernetes API.
* **Kubernetes IAM information**: This includes Service Accounts, Roles, ClusterRoles, and bindings within the cluster, also collected via the Kubernetes API.

![AWS Integration Diagram](/files/J8K0UodzQd9yIKschGJC)

The newest agent in this list is the AWS Agent, which uses various AWS APIs to gather cloud context from the AWS backend hosting your assets.

Currently, the AWS Agent collects the following information from configured AWS accounts:

1. **Cloud Compute Context from AWS EC2 and AWS EKS**: This includes all EC2 instances and EKS clusters within an AWS account, along with their detailed configurations and runtime statuses as reported by the AWS API.
2. **Cloud IAM Context from AWS**: This includes all AWS Roles and their associated Trust policies and Permission policies.

The AWS Agent is designed to be extendable to collect more information from additional AWS services. Future integrations are planned for AWS Config, AWS ECR Image Registry, AWS GuardDuty, AWS EKS Audit Logs, and AWS CloudTrail.

## How Is the AWS Context Leveraged in the Platform?

### Investigations UI

* **Kubernetes Workloads**: The platform highlights ServiceAccounts used by pods. If a ServiceAccount is linked to an AWS IAM Role (through a role annotation), that IAM Role is also displayed in the AWS accordion in the context of the investigation.

Kubernetes Service Account for a pod with associated IAM Role:

![Kubernetes Service Account for a pod with associated IAM Role](/files/bnHjlysTYlLlKRkI8k2t)

Details of the IAM Role integrated inline in the investigation UI:

![Details of the IAM Role integrated inline in investigation ui](/files/OorXRZnGmfskKEjrvJLj)

* **AWS EC2 Information**: Machine-associated AWS EC2 information is available in a new 'AWS' accordion in the Investigations UI, under the EC2 subtab.
* **AWS IAM Roles**: IAM Roles associated with an EC2 instance (via an instance profile) are displayed within the 'AWS' accordion, in the IAM subtab of the Investigations UI.

AWS EC2 and IAM integrated context in the investigation UI:

![AWS EC2 and IAM integrated context in the investigation UI](/files/4niGkmrByPu2PGDoz8Pb)

These updates allow investigators to quickly locate AWS resource context involved in incidents and assess the associated permissions to evaluate potential impact.

### Reports

Two new reports are available to leverage the AWS and Kubernetes IAM context collected:

#### 1. AWS Coverage Report

This report provides an overview of all EC2 instances and EKS clusters discovered within a specified AWS account. By comparing the complete list of compute assets with those that have a Spyderbat agent deployed, the report highlights detection coverage and helps identify assets that require monitoring and protection.

![AWS Asset Report](/files/wkhCb3Dt31IezoOyjekF)

#### 2. Cluster RBAC Report

This report provides an analysis of RBAC (Role-Based Access Control) and permissions for all Kubernetes workloads within a cluster. It summarizes the permissions associated with Service Accounts used by workloads — covering both Kubernetes roles and permissions (defining actions workloads can perform within Kubernetes) as well as AWS permissions (if the Service Account is associated with an AWS IAM Role).

![Cluster RBAC Report](/files/FQ6M7EnJofgkCsUfiZDe)

### Detection Analytics

The following detection analytics have been added to leverage the new context available:

* Creation of new Service Accounts in Kubernetes
* Deletion of Service Accounts in Kubernetes
* Creation of new Roles and ClusterRoles in Kubernetes
* Deletion of Roles and ClusterRoles in Kubernetes
* Creation of new AWS IAM Roles
* Deletion of AWS IAM Roles
* Permission drift in configured Kubernetes Roles or ClusterRoles
* Permission drift in AWS IAM Roles
* Compliance checks against Kubernetes RBAC best practices

## Getting started with the AWS Agent

To utilize this capability, an AWS Agent must be deployed to collect AWS API data. One agent is required for each AWS account you wish to monitor.

Spyderbat offers multiple deployment options for the AWS Agent, including self-hosted on an AWS VM, deployment on a Kubernetes cluster, or a hosted solution managed by Spyderbat.

For detailed installation and usage instructions, refer to the [AWS Agent Installation Documentation](/installation/spyderbat-aws-agent).


# Spyderbat Event Forwarder

The Spyderbat Event Forwarder polls the SIEM forwarding API and delivers events to your SIEM, log management platform, or any HTTP endpoint.

{% hint style="info" %}
The Event Forwarder only receives events when SIEM forwarding is enabled on at least one saved query. See [SIEM Forwarding](/concepts/integrations/siem-forwarding) for setup instructions.
{% endhint %}

The Event Forwarder is an open-source binary ([github.com/spyderbat/event-forwarder](https://github.com/spyderbat/event-forwarder)) that polls the Spyderbat API for events flagged by your saved queries and delivers them to a destination you configure. It supports file output, stdout, local syslog, and HTTP webhooks.

For installation guides, see [Install the Spyderbat Event Forwarder](/installation/spyderbat-event-forwarder).

The Event Forwarder does not filter events. To control which events are forwarded, enable SIEM forwarding on specific saved queries.

## How the Spyderbat Event Forwarder works

The Event Forwarder runs as a long-lived process that polls `GET /api/v1/org/{orgUID}/events/{iterator}` in a loop. The iterator is an opaque cursor that marks the forwarder's position in the event stream.

**Polling behavior**

When the forwarder fetches a full batch of 10,000 records, it knows more data is available and polls again after 1 second. When it receives fewer than 10,000 records, it's caught up and waits 30 seconds before the next poll. On API errors, it backs off to 30 seconds and counts consecutive failures — after five consecutive failures it logs an error but keeps running.

On first run, the forwarder starts from the oldest available buffered event. If your org has a large backlog of matching events, expect an initial burst of high-volume polling before the forwarder catches up to real time. To skip the backlog, you can reset the position using a direct API call with `LATEST` as the cursor — see [Accessing the API directly](/concepts/integrations/siem-forwarding#accessing-the-api-directly).

Forwarded events do not include metadata identifying which saved query matched them. If multiple queries in your org have SIEM forwarding enabled, their events are interleaved in the forwarder output. Use the `schema` field in each event to distinguish event types.

**Iterator persistence**

After each successful poll, the forwarder writes the next iterator to disk using a crash-safe write, which prevents corruption if the process is interrupted. On restart, it reads the saved iterator and resumes exactly where it left off. There is no data loss between restarts and no need to reconfigure a time window.

### Runtime details enrichment

Every five minutes, the forwarder fetches source metadata from the Spyderbat API and caches it in memory. When writing each event, it injects a `runtime_details` object:

```json
{
  "runtime_details": {
    "hostname": "prod-web-01",
    "ip_addresses": ["10.0.1.5", "172.16.0.3"],
    "mac_addresses": ["aa:bb:cc:dd:ee:ff"],
    "cloud_instance_id": "i-0abc123def456",
    "forwarder": "spyderbat-event-forwarder/v2.0"
  }
}
```

This enrichment lets your SIEM correlate Spyderbat events with host inventory without a separate lookup.

**Output**

The forwarder writes events as newline-delimited JSON (ndjson) to whichever output you configure:

| Output       | When to use                                                                                                                                    |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| File         | Default. Writes to `{log_path}/spyderbat_events.log`, auto-rotated at 10 MB. Use with Splunk Universal Forwarder or Filebeat.                  |
| Stdout       | Container environments. Use `stdout: true`. Logs appear in `kubectl logs` or Docker log drivers.                                               |
| Local syslog | Forward to `/dev/log`. Not recommended for remote or unencrypted syslog.                                                                       |
| Webhook      | Push directly to any HTTP endpoint (Panther, Splunk HEC, Elastic, custom). Supports gzip and zstd compression and four authentication methods. |

Failed webhook deliveries are retried automatically with exponential backoff.

## Event Forwarder validation

To confirm the forwarder is running and receiving events, check the logs for status messages. The forwarder logs as JSON — look for lines where the `"message"` field reports new record counts:

```
{"schema":"event_forwarder:meta:1.0.0","message":"5 new records (0 invalid, 5 logged)",...}
```

If using file output, you can also tail the output file directly for human-readable ndjson events:

```bash
tail -f /opt/spyderbat-events/var/log/spyderbat_events.log
```

If the forwarder runs but reports zero records after several minutes, verify that at least one saved query has SIEM forwarding enabled. See [SIEM Forwarding](/concepts/integrations/siem-forwarding).

For install-specific verification steps, see the [Helm Chart](/installation/spyderbat-event-forwarder/helm-chart) or [Traditional Installer](/installation/spyderbat-event-forwarder/traditional-installer) guides.

## Related pages

* [SIEM Forwarding](/concepts/integrations/siem-forwarding) — full architecture and control plane setup
* [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart) — end-to-end setup guide
* [Install: Helm Chart](/installation/spyderbat-event-forwarder/helm-chart)
* [Install: Traditional Installer](/installation/spyderbat-event-forwarder/traditional-installer)


# Suppression & Tuning

## Overview

Spyderbat is a powerful security tool that leverages Spydertraces to group security alerts (red flags) into scored traces of activity. This documentation page will guide you through the concepts of trace suppression to tune your Spyderbat environment.

## Spydertraces

Spydertraces are groups of security alerts that are scored based on the activity they represent. These traces provide a comprehensive view of potentially suspicious activity within your environment and are viewable on the Spyderbat dashboard. From the dashboard, you can investigate each Spydertrace to determine the nature and severity of the activity.

## Alert Suppression

Alert suppression in Spyderbat allows you to mark known Spydertrace activities as acceptable. Suppressing a Spydertrace reduces its score to 0 and prevents future traces that match the same activity from showing up in your Dashboards. This helps in reducing noise and focusing on genuinely suspicious activities.

Trace Suppression Policies are the current tool that enables Spydertrace Suppression. Suppression Policies can be generated automatically using the Spyctl CLI and a valid Spydertrace UID.

Example Suppression Policy:

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  name: Trace Suppression Policy for systemd/containerd-shim/sh/python/sh/nc
  type: trace
spec:
  traceSelector:
    matchFields:
      triggerAncestors: systemd/containerd-shim/sh/python/sh/nc
      triggerClass: redflag/proc/command/high_severity/suspicious/nc
  enabled: true
  mode: enforce
  allowedFlags:
  - class: redflag/proc/tmp_exec/high_severity/nc
  - class: redflag/proc/command/high_severity/suspicious/nc
  - class: redflag/proc/suspicious_crud_command/high_severity/cat
```

This example policy will suppress any Spydertraces triggered via a suspicious `nc` command with the specific process ancestors of `systemd/containerd-shim/sh/python/sh/nc`. Within that scope, the policy then specifies what other flags are allowed to be grouped within the trace.

Should additional flags appear outside of the allowed list, the trace would no longer be suppressed and have a new score based on the severity of any new flags.

### Matching Multiple Patterns with matchFieldsExpressions

The `traceSelector` in the example above uses `matchFields`, which supports a single value per field. When a workload runs across different container runtimes (e.g., CRI-O and containerd), the process ancestry paths differ. Instead of creating separate policies for each runtime, you can use `matchFieldsExpressions` with the `In` operator to match multiple patterns in a single policy.

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  name: Suppress health-check curl across runtimes
  type: trace
spec:
  containerSelector:
    matchFields:
      image: "*trinodb/trino*"
  traceSelector:
    matchFieldsExpressions:
      - key: triggerAncestors
        operator: In
        values:
          - "systemd/crio/conmon/runc/*/health-check/curl"
          - "systemd/containerd-shim/runc/*/health-check/curl"
      - key: triggerClass
        operator: In
        values:
          - "redflag/proc/command/high_severity/suspicious/curl"
  enabled: true
  mode: enforce
  allowedFlags:
    - class: redflag/proc/command/high_severity/suspicious/curl
```

Each value in the `values` array supports glob pattern matching and is evaluated independently. A trace matches if its field value matches **any** of the listed values (`In` operator provides OR semantics). Values containing wildcards must be quoted in YAML.

Supported patterns: `*` (any characters), `?` (single character), `[0-9]` or `[a-z]` (character ranges). Note that `*` matches across `/` separators, so `systemd/*/curl` matches ancestor chains of any depth. Use specific path segments to keep patterns precise.

This approach is more precise than using a single broad wildcard like `systemd/*/runc/*/health-check/curl`, which could match unexpected runtime configurations.

For the full pattern matching syntax, operators, and expression reference, see [Selectors](/reference/selectors).

{% hint style="info" %}
**Suppression is not retroactive.** After applying a suppression policy, existing traces that match the policy are not immediately re-scored to 0. The analytics engine evaluates traces periodically, so it may take up to 24 hours for all matching Spydertraces to disappear from your dashboard.

To see the effect of new policies sooner, adjust your dashboard or search time range to the **last 1 hour** — new traces generated after the policy was applied will be suppressed immediately. This is also recommended when analyzing noise levels to assess policy effectiveness.
{% endhint %}

## Methods of Suppressing Spydertraces.

There are two main methods to suppress Spydertraces in Spyderbat:

1. UI-based approach.
2. CLI-based approach.

With either approach, once you implement the suppression rule, any active Spydertraces that match the rule's scope and allowed flags will be immediately suppressed. New Spydertraces that fit these criteria will also be automatically suppressed going forward.

Don’t worry if you don’t know about Suppression Rules yet, we’ll get into it.

### 1. Using the UI

**The UI approach involves 4 simple steps**: finding the Spydertrace to suppress, clicking Suppress Trace, creating a suppression rule, and clicking Create.

You can find Spydertraces in three main ways: using Search, Dashboard, or Investigation.

The key part of this process is creating a suppression rule in the UI. Let's go over that first before explaining how to find Spydertraces.

#### Suppression Rule and Suppression Scope Customization:

Suppression rules allow you to reduce noise in your environment by marking known activity as acceptable, ensuring that your focus only remains on suspicious activities.

By default, suppression rules are applied globally across your environment (org).

To target specific areas and reduce the scope of the suppression, you can customize the rule by adding selectors, such as user, machine, cluster name, container, namespace, or any other available identifiers from the drop-down list available.

After you click "Suppress Trace" for a Spydertrace, this window pops up.

![](/files/o2RlUEhoBexU27YYlIJj)

Using selectors allows you to focus the suppression rule on particular components, ensuring that it only applies where necessary.

**You can limit the suppression Scope to**:

* Specific `users` to control which individuals the rule affects.
* Particular `machines` or `hosts` to contain the suppression to certain hardware.
* A particular `cluster` for Kubernetes-based environments.
* Specific `containers` or `pods` or `Namespace` to isolate suppression in a containerized setup.

You can also choose the allowed flags as part of Suppression.

Spyderbat allows you to edit the Suppression Rule context and the Suppression Rule Name.

To make the `Suppression Rule Context` generic add a wildcard (\*) to the Trigger Ancestor or the Trigger class as desired. This use of wildcards makes Suppression Rules flexible, allowing you to catch a wider range of patterns and reducing the need for multiple specific rules.

Note: You cannot edit the selectors in the console once the policy has been created, but using Spyctl CLI you can still edit the raw yaml. For simplicity we recommend deleting and recreating the suppression rule if you wish to edit the selectors.

Now that you understand what suppression rules are, let’s look at 3 different ways to find Spydertraces and apply suppression rules using the UI.

#### a. Searching on Spydertraces:

In the Search section of the Spyderbat UI, you can search for various Kubernetes objects, processes, connections, and Spyderbat-specific entities like Spydertraces.

Suppressed traces refer to Spydertraces that have been intentionally suppressed.

To begin, select `Spydertrace`, open the query builder, and select the fields you want to query. Use the appropriate operators and time filters to refine your search.

Below, we've used `score>40` query for our Search.

Additionally, you can apply filters to the result from the top-left corner to further narrow down and investigate the data that interests you.

Once you’ve found the trace of interest in the search results, select it and click Suppress Trace from the options when prompted. You can also go ahead and Investigate further and then Suppress the trace.

![](/files/6wwP6tXvakPYJCac7K21)

This will open the Create Trace Suppression Rule page, where you can customize the scope and add multiple selectors from the available list.

Refer to the 'Suppression Rule and Suppression Scope Customization' and finally, click Create to apply the rule.

#### b. Spydertraces Dashboard Card

Spydertraces represent potential security concerns, and by default, the ‘Security’ dashboard category in the Spyderbat UI includes a card dedicated to all security-related aspects, including Spydertraces.

You can view several key dashboard cards, such as:

* Recent Spydertraces with Score > 50: Displays high-priority traces for immediate attention.
* Suppressed Traces: Lists any traces that have been suppressed.

![](/files/0YHXjobrxqh2pgWjLm7p)

Spydertraces are automatically grouped by their trigger short names for easier review in Dashboard cards. If needed, you can ungroup them to focus on individual traces.

To suppress a specific trace, select it, click Suppress Trace, then customize the scope and settings as described earlier to create the suppression rule. This allows you to refine and manage trace suppression based on your security needs.

![](/files/tkDcTPMiK9rs4ywDzO5Q)

With this Suppression rules setting you have decided to reduce noise in your environment by marking known activity as acceptable.

Finally, click Create to finalize the rule. You have successfully suppressed a trace. You can also check it out in the “Suppression Trace” dashboard card for quick review.

You can also create your own Custom Dashboard Card dedicated to your Spydertrace query and suppress the trace from there as desired.

#### c. Investigation

Another method of suppressing a trace is through the Investigation feature in Spyderbat.

You can start your Kubernetes investigation via Kubernetes Section or Source Section in UI. If you observe a Spydertrace linked to the object that has generated flags, but after review, you determine it is not malicious, In this case, you can also suppress the trace directly from the investigation interface.

![](/files/ANTI1qvPVwQHhjVcDV3N)

**Alternatively, you can**:

a) Search for a Spydertrace and add it to the investigation.

b) Once added, you can suppress it from the investigation.

c) Customize the scope if needed, apply selectors, and create/apply the suppression rule.

### 2. CLI-Based Approach

The second method of creating a Suppression trace is using Spyctl CLI.

In the CLI-based approach, you can easily manage Spydertraces using the Spyctl CLI. Here’s a step-by-step guide to help you navigate:

#### (i) View Spydertraces:

To retrieve a list of Spydertraces, you can use the below command:

```
$ spyctl get spydertraces
```

This command will provide summarized information, including the trigger name, count of occurrences, etc for further investigation

If you're specifically interested in Spydertraces with a score above 50, you can add the --score\_above option:

```
$ spyctl get spydertraces --score_above 50 
```

There are a lot of other filter options that you can provide. Use '$spyctl get spydertraces –help' for more information.

#### (ii) Get the UID:

Identify the UID of the Spydertrace you want to suppress from the list. The UID uniquely identifies each trace and is necessary for suppression.

#### (iii) Suppress the Trace:

Once you have the UID, use the following command to suppress the specific Spydertrace.

```
  spyctl suppress trace TRACE_UID

```

Replace TRACE\_UID with the actual UID of the trace. You can also apply additional options for more control over the suppression policy:

```
-u, --include-users: Scope the suppression to specific users found in the trace.
-n, --name: Provide an optional name for the suppression policy. If you don’t provide a name, one will be generated automatically.
-y, --yes: Automatically answer "yes" to prompts, making the process non-interactive.
```

For more details on how to fine-tune your commands, you can always use the --help option: `spyctl suppress trace --help`

Editing a Suppression Rule/Policy:

You cannot edit the selectors for Suppression Rule in Console, but you can edit the raw YAML for a suppression rule or policy in Spyctl CLI using the following command:

```
$ spyctl edit trace-suppression-policy <policy_id>
```

Each suppression rule is linked to a unique policy ID. To find the policy ID, you can use the following command:

```
$ spyctl get policies --type trace
```

However, for simplicity, we recommend deleting and recreating the suppression rule if you need to modify the selectors.

## Managing Suppression Rules:

There may be cases where you want to delete or disable a suppressed Rule to further stop the traces from being generated.

For example, if the conditions that triggered the suppression are no longer valid or if the trace needs to be re-evaluated due to changes in your security policy. Spyderbat allows you to manage these traces easily.

### How to Delete a Suppressed Rule:

#### UI approach.

To permanently remove a suppression Rule:

* Go to the Suppression Rules section in the Spyderbat Console.
* Find the suppression rule associated with the trace you wish to delete.
* Click on the bin icon next to the suppression rule name.

This will delete the Suppression rule. Deleting a suppressed rule is useful when the trace is no longer relevant to suppress or if you need the activity information in your environment.

![](/files/PrsatSxVhkH5Vdo2j5wo)

#### CLI Approach:

You can delete the Suppression Rule/policy using Spyctl CLI with the following command:

```
$ spyctl delete policy <policy_id>
```

Each suppression rule is associated with a unique policy ID. To find the policy ID, use the command:

```
$ spyctl get policies
```

### How to Disable a Suppression Rule:

#### UI approach.

**To temporarily disable a suppression rule**:

* Go to the Suppression Rules section
* Click on View next to the suppression rule you want to manage.
* Navigate to Rule Settings.
* Locate the Rule Status option.
* Set the status from Enabled to Disabled to turn off the suppression rule.

Disabling a suppression rule is helpful when you want to pause the suppression without permanently deleting the rule, allowing you to enable it later if necessary.

![](/files/ibckINVmtjsW2UArQgMh)

#### CLI Approach:

```
$ spyctl edit trace-suppression-policy <policy_id>
```

Then set the `enabled` field to false.

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  ...
spec:
  allowedFlags:
    ...
  enabled: False
  ...
```


# Scout (Detections)


# Spydertraces

A Spydertrace is a causally connected record of system activity. This page explains how traces are triggered, scored, and reviewed by analysts.

A **Spydertrace** is a record of causally connected activity on a system. When Spyderbat detects a noteworthy behavior, it captures not just that event but everything causally linked to it: the process ancestry chain, subsequent child processes, network connections, and the container or host context. The result is a self-contained record of what happened and how, which analysts can investigate to determine whether the activity is a threat, benign, or a false positive.

## How Spydertraces are triggered

Spydertraces start with a **red flag** — a behavioral indicator detected by Spyderbat's built-in detections. Red flags are observations, not verdicts: they indicate that something worth examining occurred, not that it was necessarily malicious. The red flag that starts the trace is called the trigger. As activity continues within the same causal chain, additional red flags may fire and get absorbed into the same trace.

[Custom flags](/concepts/scout/custom-flags) behave the same way. A user-defined detection rule can trigger a new trace or contribute to the score of one already in progress, depending on whether the flagged activity is causally connected to existing trace activity.

## Scoring

Each trace carries a numeric score reflecting how many and how severe its flags are. It increases as additional red flags join the trace. Severity levels for individual red flags are: info, low, medium, high, and critical. Info-level flags don't contribute to traces.

The score is capped based on the highest severity flag present:

| Highest flag severity | Score cap |
| --------------------- | --------- |
| Critical              | None      |
| High                  | 65        |
| Medium or Low         | 45        |

> Info-level flags don't contribute to traces and are not included above.

The console uses score to indicate triage priority:

| Score | Priority | Console color |
| ----- | -------- | ------------- |
| ≥ 66  | Critical | Red           |
| ≥ 33  | High     | Orange        |
| < 33  | Low      | —             |

One factor can push a score above these caps: if the trace involved an interactive session running under a privileged account (`root`, `admin`, `SYSTEM`, or `administrator`), the trace gets an additional 25 points and the score cap is lifted entirely regardless of flag severity.

## Trace lifecycle

A trace is **active** while activity is ongoing. Once the associated activity ends, it moves to **closed**.

Each trace records:

* The full process ancestry chain leading to the triggering event
* The triggering flag and all contributing red flags
* Network connections made during the activity
* Container context (name, image, cluster, namespace) when applicable, or host context for non-containerized workloads

## Working with Spydertraces

Traces appear in the Spyderbat dashboard and can be queried from the **Search** page. From Search results, select one or more traces and open them in the **Investigations** view for causal analysis — see [Investigations](/concepts/flashback/investigations).

To get alerted when new traces match criteria you care about (e.g. score above a threshold, specific flag classes), save a search targeting Spydertraces and attach a notification to it. See [Notifications](/concepts/notifications).

Traces can be **suppressed** to reduce noise from known-safe activity. Suppression sets the score to zero and prevents future traces matching the same pattern from appearing. See [Suppression & Tuning](/concepts/suppression).


# Custom Flags

### What are Custom Flags?

Custom Flags in Spyderbat are a powerful feature that enable users to create tailored detection rules to monitor activities or behaviors specific to their environment.

Custom Flags are designed to address unique needs that Spyderbat's built-in detections may not cover and may be specific to your organization's requirements.

#### Leveraging the Spyderbat Query Language (SpyQL)

Spyderbat allows you to write Custom Flags using the `Spyderbat Query Language (SpyQL)`. SpyQL enables you to craft precise queries that define the conditions for your Custom Flags.

SpyQL supports complex queries that allow you to combine multiple conditions, use various logical operators (AND, OR, NOT), and apply patterns with `matches pattern` (\~=) and regular expressions with `Regex` using `~~=` operator, equality operator `=`, etc.

SpyQL is used for Historical Search in Console and for Custom Flags. The queries are composed of two parts, the schema or object type you are looking for and the query itself. In Historical Search you must also specify a time window, however Custom Flags operate in real-time so that section is not supported. The SpyQL query below is from Historical Search and is querying for any Container where the cluster-name field matches the value integrationc2 using the equality operator (=):

![Query Image](/files/nxfLNL0q94efIdYn10lm)

You can use Historical Search in the UI to test your Custom Flag queries.

#### Custom Flags Key Features:

1. **Real-Time Monitoring**: Once set up, Custom flags operate in real-time, triggering immediate flags when a record matches the SpyQL query.
2. **Flexibility**: You can define flags that range from broad conditions e.g., anytime a new StatefulSet is created to highly specific scenarios, e.g., a serviceaccount with `cluster-admin` role created in a particular namespace by a particular user.
3. **Red Flag vs Ops Flag**: You can choose between a Custom Red Flag (Security) or a Custom Ops Flag (Devops) based on your detection needs.

   * Redflag: Indicates a security issue or potential malicious activity.
   * Opsflag: Highlights operational or configuration issues that may need attention.

   *Custom flags also allow you to add your own description and select severity options such as low, high, critical, or info.*
4. **Integration with Spydertraces**: Custom Red Flags may trigger and/or contribute to the score of a Spydertrace just like the built-in Spyderbat detections do.
5. **Custom Flag Operations**: The ability to create, delete, edit, disable, and enable custom flags further enhances your control over managing the detection process, and gives the ability to evolve as required.

### Getting Started

Currently, Custom Flags are only manageable using the [Spyctl CLI](/installation/spyctl). Management via the Console UI is coming soon.

Follow the Spyctl CLI tutorial for setting up Custom Flags [here](/tutorials/scout/setup-custom-flags).


# Tutorials


# Flashback


# How to Use the Investigations Feature in Spyderbat

How to use the Causal Tree in a Process Investigation — adding and removing nodes, toolbar controls, right-click context menus, data layers, and sharing investigation links.

This guide covers how to work with the Causal Tree inside a Process Investigation. If you haven't opened an investigation yet, see [Investigations](/concepts/flashback/investigations) for how to get started.

## Adding Records to the Causal Tree

From the **Records** panel on the left side of the Investigation view, you can add records to the Causal Tree in several ways:

* **Star icon** — Click the star on any record row to add that individual record to the graph
* **Add All** — Click the **Add All** button above the records table to add every visible record at once
* **Show in Graph** — If a record is already displayed in the tree, this button pans the view to that node

Once records are on the graph, the Causal Tree renders their causal relationships — which process spawned which, what connections were made, and how events chain together.

<figure><img src="/files/TfPhR9e10WeRwD57mTwV" alt="Investigation view showing the Records panel on the left, Causal Tree graph on the right, and Details panel below"><figcaption><p>The Causal Tree (right) displays nodes added from the Records panel (left). The Details panel (bottom) shows metadata for the selected node.</p></figcaption></figure>

## Toolbar

The toolbar at the top of the Causal Tree provides controls for managing the graph view:

* **Clear All** (trash icon) — Removes all nodes from the graph. Use undo if you clear by mistake.
* **Undo / Redo** — Step backward or forward through actions performed on the tree (adding nodes, removing nodes, etc.)
* **Auto Focus** — Centers and zooms the view to fit all currently displayed nodes
* **Zoom In / Zoom Out** (magnifying glass icons) — Adjust zoom level. You can also scroll with your mouse wheel.
* **Highlight nodes** — A dropdown that color-codes nodes by a selected attribute (e.g., Container UID). Nodes sharing the same value get the same color, making it easy to spot container or host boundaries.
* **Options** — Opens a dropdown with display toggles (see [Options Dropdown](#options-dropdown) below)
* **Copy Investigation Link** (top right) — Generates a permalink that captures the current state of the investigation (which nodes are displayed, which are selected). Links are not public — the recipient must be a member of your organization with read access. Share with a colleague for collaborative analysis or save for future reference.
* **Summarize** (top right) — Produces an AI-generated summary of the investigation's findings.

## Navigating Nodes

At the bottom of the Causal Tree, two navigation buttons let you step through nodes chronologically:

* **Previous node** — Jump to the node that occurred before the current selection
* **Next node** — Jump to the node that occurred after the current selection

This is especially useful when reviewing a sequence of commands or events in the order they happened.

## Selecting and Inspecting Nodes

**Left-click** a node to select it. Selecting a node:

1. Shows detailed metadata in the **Details** panel below the tree
2. Highlights the corresponding record in the **Records** table

## Right-Click Context Menu

**Right-click** a node to open a context menu organized into sections.

<figure><img src="/files/1bcouuIKqCCKGvFIGDYj" alt="Right-click context menu on a process node showing Load, Remove, and Search sections"><figcaption><p>Right-click a node to access load, remove, and search options.</p></figcaption></figure>

**Top actions:**

* **Details View** — Open the full details view for this node
* **Set Time in Time Picker** — Set the time range to this node's timestamp
* **Open Subtree in New Tab** — Open this node's subtree in a separate browser tab

**LOAD** — Load related objects that are already within the enabled Data Layers:

* **Load Descendent Connections** — Bring in network connections causally related to this node

**REMOVE** — Prune the graph by removing nodes:

* **Remove Self** — Remove the selected node and its dependents. Useful for pruning irrelevant branches (e.g., removing a bash process removes all its child processes).
* **Remove Descendants** — Remove all nodes downstream from the selected node
* **Remove with name** — Remove all nodes sharing the same process name
* **Remove with cgroup** — Remove all nodes in the same cgroup (Kubernetes pod)
* **Remove Everything Else** — Keep only the selected node's subtree, removing all other nodes

**SEARCH** — Search across all time known to Spyderbat and load results as a new Data Layer. This is how you find activity outside the original query time range:

* **Search For Child Processes** — Find directly spawned child processes
* **Search For Descendent Processes** — Find all downstream processes
* **Search For Child Connections** — Find network connections from child processes
* **Search For Red Flags** — Find security-relevant flags associated with this node
* **Search For Ops Flags** — Find operational flags associated with this node
* **Search By euser / auser** — Find activity by the same effective or audit user

## Options Dropdown

Click **Options** in the toolbar to access display toggles:

<figure><img src="/files/YetxWofx8JWFZ28p0Ogg" alt="Options dropdown showing Hide Threads, Show Relative Time, Hide Future Nodes, Hide Container Box, and Hide Process Context Box toggles"><figcaption><p>The Options dropdown with display toggles. Show Relative Time is enabled, displaying timestamps on each node.</p></figcaption></figure>

* **Hide Threads** — Hide thread-level processes to reduce visual clutter
* **Show Relative Time** — Display a relative timestamp on each node (e.g., "45ms", "-1y") based on the currently selected node. This is useful when reviewing traces that span long periods — it makes the temporal distance between activities immediately visible.
* **Hide Future Nodes** — Hide nodes that occur after the selected node's timestamp
* **Hide Container Box** — Hide the container grouping outlines around nodes
* **Hide Process Context Box** — Hide the process context grouping outlines

## Data Layers

Every search or dashboard card that feeds into the investigation creates a **Data Layer**. Data Layers let you:

* **Toggle subsets on/off** — Disable a layer to hide its records from both the Causal Tree and Records table, letting you focus on a specific data set
* **Isolate activity** — Enable only one layer to view only that subset of records in both the tree and tabular format

The Data Layers section appears in the Records panel, showing the layer count (e.g., "Data Layers (2)") and checkboxes to toggle each layer.

## Related Pages

* [Investigations overview](/concepts/flashback/investigations) — Records Panel, Causal Tree, and Details Panel reference
* [Search](/concepts/search) — Query language for finding Spydertraces to investigate
* [Dashboards](/concepts/dashboards) — Launch investigations from dashboard cards
* [Spydertrace Summarize](/concepts/summarize/spydertrace-summarize) — AI-generated investigation summaries


# Guardian


# How to Lock Down Your Workloads With Guardian Policies Using Spyctl

This page will teach you about Guardian Workload Policies. It will explain what they are, how to create them, how to apply them, and how to manage them.

## Prerequisites

* [Install Spyctl](/installation/spyctl)
* [Configure Spyctl with a Context](/installation/spyctl/spyctl-initial-configuration)
* Have installed at least one [Spyderbat Nano Agent](/installation/spyderbat-nano-agent) installed on a machine of your choosing

## What is a Guardian Workload Policy

Spyderbat's Guardian Workload Policy feature empowers users to define a known-good whitelist of process and network behavior for containers and Linux services. These policies serve as a proactive measure, enabling users to receive notifications when Spyderbat detects deviant activity within their workloads. Not only does the Guardian Workload Policy provide alerts, but it also offers the capability to take decisive actions by terminating deviant processes and Kubernetes pods. This comprehensive approach ensures a robust security framework, allowing users to maintain a vigilant stance against potential threats and unauthorized activities within their environments.

Workload Policies also serve as a way to tune out certain Red Flags & Spydertraces. Red Flags and Spydertraces that would have been generated by your workloads will be marked as exceptions and reduce clutter in your security dashboards.

## Retrieving Fingerprints

Policies are created from Fingerprints. Fingerprints are auto-generated documents with the process and network activity for a single container instance or instance of a Linux Service.

To view all Fingerprints generated across your organization, issue the following command:

```
spyctl get fingerprints --type FINGERPRINT_TYPE [NAME_OR_UID]
```

For example:

```
$ spyctl get fingerprints --type container docker
Getting fingerprints from 2024-01-16T13:52:51Z to 2024-01-16T15:22:51Z
IMAGE_NAME:TAG        IMAGEID       REPO                        COVERED_BY_POLICY    LATEST_TIMESTAMP                                                                                                                                                                                                                                                                 
mongo:latest          8248f2793e07  docker.io/library           0/2                  2024-01-16T15:00:43Z
nginx:latest          10d1f5b58f74  docker.io/library           0/14                 2024-01-16T15:01:08Z
node:v3.23.5          b7f4f7a0ce46  docker.io/calico            0/1                  2024-01-16T15:01:08Z
```

By default, Spyderbat queries for all Fingerprints in your organization for the last 1.5 hours. This means that it will retrieve Fingerprints for any container or Linux service instance running during that time window (Even if the instances started well before the time window). You can increase the time range with the `-t` option.

In this example organization we have 3 workloads running across multiple instances. Spyderbat has Fingerprints for two instances of mongo, 14 instances of nginx, and one instance of node that were online during the query's time window. None of these Fingerprints are covered by a policy as seen in the COVERED\_BY\_POLICY column.

### Download Fingerprints to a File

To create a policy we must first download the fingerprints we wish to use to build the policy.

```
spyctl get fingerprints --type TYPE --output OUTPUT NAME_OR_UID > FILENAME
```

For example:

```
spyctl get fingerprints --type container -o yaml mongo:latest > fprints.yaml
```

#### \[Optional] Downloading Fingerprints from a K8s Namespace

In certain instances you may have the same container image running in different Kubernetes namespaces with different allowed network activity. To separate allowed network activity by namespace you can use multiple policies for the same image. Using only Fingerprints from the same namespace will automatically scope the policy to that namespace.

```
spyctl get fingerprints --type container --output OUTPUT [--cluster CLUSTER_NAME_OR_UID] --namespace NAMESPACE NAME_OR_UID > FILENAME
```

for example:

```
spyctl get fingerprints --type container -o yaml --namespace dev mongo:latest > fprints-dev.yaml
spyctl get fingerprints --type container -o yaml --namespace prod mongo:latest > fprints-prod.yaml
```

This will only download Fingerprints tied to a the namespace specified in the command.

## Create the Policy

Once you have the Fingerprints to create the policy from, issue the following command:

```
$ spyctl create policy --from-file FILENAME --name NAME_FOR_POLICY --mode MODE > policy.yaml
```

For example:

```
$ spyctl create policy --from-file fprints.yaml --name mongo-policy --mode audit > policy.yaml
```

{% hint style="info" %}
Running this command does not make any changes to your Spyderbat Environment. It is not until you have applied a Policy, that enforcement takes effect.
{% endhint %}

The Policy file we just created `policy.yaml` now has a new resource, the `kind` field is now "SpyderbatPolicy". This document is a merged version of the Fingerprints that went into it:

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  latestTimestamp: 1705435213.327981
  name: mongo-policy
  type: container
spec:
  containerSelector:
    image: docker.io/library/mongo:latest
    imageID: sha256:68248f2793e077e818710fc5d6f6f93f1ae5739b694d541b7e0cd114e064fa11
  mode: audit
  processPolicy:
  - name: mongod
    exe:
    - /usr/bin/mongod
    id: mongod_0
    euser:
    - mongo
  networkPolicy:
    egress: []
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.229/32
      - ipBlock:
          cidr: 192.168.1.146/32
      - ipBlock:
          cidr: 192.168.2.221/32
      - ipBlock:
          cidr: 192.168.4.31/32
      processes:
      - mongod_0
      ports:
      - protocol: TCP
        port: 27017
  response:
    default:
    - makeRedFlag:
        severity: high
    actions: []
```

{% hint style="info" %}
Policies are created in `audit` mode by default. If you apply a Policy in `audit` mode it will not take response actions, but will log the activity it would have taken. You can use the command `spyctl logs policy POLICY_UID` to monitor those log.
{% endhint %}

## Generalize the Policy

In its current form, this policy will only apply to mongo containers with the `latest` tag and only with the image ID `sha256:68248f2793e077e818710fc5d6f6f93f1ae5739b694d541b7e0cd114e064fa11`

We can remove selector fields and wildcard values to broaden the Policy's scope.

Using the `edit` command will open your favorite text-editor and perform syntax checking when you save.

```
spyctl edit policy.yaml
```

Then we can generalize the containerSelector and the IP blocks

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  latestTimestamp: 1705435213.327981
  name: mongo-policy
  type: container
spec:
  containerSelector:
    image: docker.io/library/mongo:*
  mode: audit
  processPolicy:
  - name: mongod
    exe:
    - /usr/bin/mongod
    id: mongod_0
    euser:
    - mongo
  networkPolicy:
    egress: []
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.0/16
      processes:
      - mongod_0
      ports:
      - protocol: TCP
        port: 27017
  response:
    default:
    - makeRedFlag:
        severity: high
    actions: []
```

In the above policy we removed the `imageID` field in the `containerSelector` we also increase the scope of the ipBlock in `ingest` from multiple /32 CIDRs to a single /16 CIDR.

## Applying the Policy

To apply a Policy you must use the `apply` command:

```
$ spyctl apply -f FILENAME
```

The apply command will recognize the `kind` of the file, perform validation, and attempt to apply the resource to the policy database for the organization in your current Context. It accomplishes this via the Spyderbat API.

For example, to apply the Policy we created above:

```
$ spyctl apply -f policy.yaml
```

This will apply the Policy to the organization in your current Context.

To view the applied Policies in your current Context you can use the `get` command:

```
$ spyctl get RESOURCE [OPTIONS] [NAME_OR_ID]
```

For example:

```
$ spyctl get policies
UID                       NAME              STATUS     TYPE       CREATE_TIME
pol:CB1fSLq4wpkFG5kWsQ2r  mongo-policy      Auditing   container  2024-01-16T15:00:43Z
```

To view the yaml of the Policy you just applied, issue the command:

```
$ spyctl get policies -o yaml CB1fSLq4wpkFG5kWsQ2r
```

The Policy will look something like this:

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  name: mongo-policy
  type: container
  uid: pol:CB1fSLq4wpkFG5kWsQ2r
  creationTimestamp: 1673477668
  latestTimestamp: 1670001133
spec:
  containerSelector:
    image: docker.io/library/mongo:*
  mode: audit
  processPolicy:
  - name: mongod
    exe:
    - /usr/bin/mongod
    id: mongod_0
    euser:
    - mongo
  networkPolicy:
    egress: []
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.0/16
      processes:
      - mongod_0
      ports:
      - protocol: TCP
        port: 27017
  response:
    default:
    - makeRedFlag:
        severity: high
    actions: []
```

### \[Optional] Adding "Interceptor" Response Actions

When a new Policy is created it will have a `default` Actions list, and an empty list of `actions`. The `default` Actions are taken when a policy is violated and no Actions of the same type in the `actions` list are taken.

```yaml
response:
  default:
  - makeRedFlag:
      severity: high
  actions: []
```

By default, `spyctl` includes a `makeRedFlag` Action in the `default` section of the policy’s `response` field. This tells the Spyderbat backend to generate a redflag of high `severity` which will show up in the Spyderbat Console. The full list of redflag severities, in increasing severity, is as follows:

* info
* low
* medium
* high
* critical

The Actions in the `actions` field are taken when certain criteria are met. Every Action in the `actions` field must include a Selector. Selectors are a way of limiting the scope of an Action. For example, you can tell Spyderbat to kill a bash process that deviates from the Policy by using the `processSelector`:

```yaml
actions:
- agentKillProcess:
    processSelector:
      name:
      - bash
```

If you are in a Kubernetes environment you can also set up an Action to kill a pod when a Policy violation occurs. Let's say you want to kill a pod in your staging environment, the action would look like so:

```yaml
actions:
- agentKillPod:
    podSelector:
      matchLabels:
        env: staging
```

To add a kill process action, edit your policy file. For example:

```
$ spyctl edit policy pol:CB1fSLq4wpkFG5kWsQ2r
```

And add a kill process Action to the `actions` list.

```yaml
response:
  default:
  - makeRedFlag:
      severity: high
  actions:
  - agentKillProcess:
      processSelector:
        name:
        - bash
```

Our Policy now looks like this:

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  name: mongo-policy
  type: container
  uid: pol:CB1fSLq4wpkFG5kWsQ2r
  creationTimestamp: 1673477668
  latestTimestamp: 1670001133
spec:
  containerSelector:
    image: docker.io/library/mongo:*
  mode: audit
  processPolicy:
  - name: mongod
    exe:
    - /usr/bin/mongod
    id: mongod_0
    euser:
    - mongo
  networkPolicy:
    egress: []
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.0/16
      processes:
      - mongod_0
      ports:
      - protocol: TCP
        port: 27017
  response:
    default:
    - makeRedFlag:
        severity: high
    actions:
    - agentKillProcess:
        processSelector:
          name:
          - bash
```

## Summary and Next Steps

At this point you should have an applied policy in `audit` mode. You'll find that if you run `spyctl get fingerprints --type container` or `spyctl get fingerprints --type linux-service` the fingerprints you included in the policy will now be covered.

```
$ spyctl get fingerprints --type container docker
Getting fingerprints from 2024-01-16T13:52:51Z to 2024-01-16T15:22:51Z
IMAGE_NAME:TAG        IMAGEID       REPO                        COVERED_BY_POLICY    LATEST_TIMESTAMP                                                                                                                                                                                                                                                                 
mongo:latest          8248f2793e07  docker.io/library           2/2                  2024-01-16T15:00:43Z
nginx:latest          10d1f5b58f74  docker.io/library           0/14                 2024-01-16T15:01:08Z
node:v3.23.5          b7f4f7a0ce46  docker.io/calico            0/1                  2024-01-16T15:01:08Z
```

While in audit mode your policy will generate logs of the actions it would have taken, in addition to any deviations it detects. To learn how to manage and update your policies refer to [Guardian Policy Management in Spyctl](/reference/spyctl/guardian-policy-management-spyctl). It explains how to edit, update, and diff your policies. It will also explain when to graduate your policies from `audit` to `enforce` mode.

When all of your policies are in `enforce` mode and every fingerprint is covered by a policy you will have locked down your environment using Guardian. You will have established a whitelist of activity for your all of your critical workloads which simultaneously allows you to quickly be notified when real threats occur, and reduce red flag and spydertrace noise in your Spyderbat Dashboards.


# How to Put Guardrails Around Your K8s Clusters Using Spyctl

This tutorial will walk you through the creation, tuning, and management of Cluster Ruleset Policies.

## Prerequisites

* [Install Spyctl](/installation/spyctl)
* [Configure Spyctl with a Context](/installation/spyctl/spyctl-initial-configuration)
* Install the [Spyderbat Nano Agent](/installation/spyderbat-nano-agent) on a cluster via helm install

## What is a Cluster Ruleset Policy?

A Cluster Ruleset Policy is a special type of [Ruleset Policy](/concepts/guardian/ruleset_policies) focused on establishing allowed or disallowed resources or activity within a Kubernetes Cluster. Through Cluster Ruleset Policies users can receive customized notifications when deviant activity occurs within your clusters. For example, users can specify the container images that are allowed to run within a namespace. Should a new image appear, a deviation is created, with a link to investigate the problem. Users can then take manual or automated actions to address the deviation.

## Creating a Cluster Policy

Cluster Policies and their accompanying Cluster Rulesets are generated using the `spyctl create` command. First, identify which cluster you wish to create a cluster policy for.

```sh
spyctl get clusters
```

For example:

```
$ spyctl get clusters
Getting clusters
NAME            UID               CLUSTER_ID                            FIRST_SEEN            LAST_DATA
demo-cluster    clus:VyTE0-BPVmo  xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx  2024-03-14T17:14:19Z  2024-05-06T18:07:24Z
```

{% hint style="info" %}
If the previous command does not return any results, follow the [helm installation guide](/installation/spyderbat-nano-agent) to install the Spyderbat Nano Agent in your K8s cluster.
{% endhint %}

Next, consider how you would like the auto-generated rules to be scoped. Certain rule types may be scoped specifically to namespaces.

|                             |                                                                   |
| --------------------------- | ----------------------------------------------------------------- |
| --namespace                 | Generate rules for all namespaces including namespace scope       |
| --namespace NAMESPACE\_NAME | Generate rules for a specific namespace including namespace scope |
| OMITTED                     | Generate rules for all namespaces scoped globally                 |

Use the following command to generate a cluster policy and its ruleset(s).

```sh
spyctl create cluster-policy -C CLUSTER [--namespace [NAMESPACE_NAME]] -n POLICY_NAME > cluster-policy.yaml
```

For example:

```
$ spyctl create cluster-policy -C demo-cluster --namespace -n demo-cluster-policy > cluster-policy.yaml
Validating cluster(s) exist within the system.
Creating ruleset for cluster demo-cluster
Generating container rules...
Cluster(s) validated... creating policy.
```

{% hint style="info" %}
By default, rules are generated using data from the last 1.5 hrs. You can use the `-t` option to override that.
{% endhint %}

The file you just generated `cluster-policy.yaml` now contains the Cluster Policy itself and any automatically generated rulesets the policy requires.

```yaml
apiVersion: spyderbat/v1
items:
- apiVersion: spyderbat/v1
  kind: SpyderbatRuleset
  metadata:
    name: demo-cluster_ruleset
    type: cluster
  spec:
    rules:
    - namespaceSelector:
        matchExpressions:
        - {key: kubernetes.io/metadata.name, operator: In, values: [rsvp-svc-dev, rsvp-svc-prod]}
      verb: allow
      target: container::image
      values:
      - docker.io/guyduchatelet/spyderbat-demo:1
      - docker.io/library/mongo:latest
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      verb: allow
      target: container::image
      values:
      - 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon-k8s-cni-init:v1.10.1-eksbuild.1
      - 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon-k8s-cni:v1.10.1-eksbuild.1
      - 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/coredns:v1.8.7-eksbuild.1
      - 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/kube-proxy:v1.22.6-eksbuild.1
      - public.ecr.aws/aws-secrets-manager/secrets-store-csi-driver-provider-aws:1.0.r2-58-g4ddce6a-2024.01.31.21.42
      - registry.k8s.io/csi-secrets-store/driver:v1.4.2
      - registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.10.0
      - registry.k8s.io/sig-storage/livenessprobe:v2.12.0
- apiVersion: spyderbat/v1
  kind: SpyderbatPolicy
  metadata:
    name: demo-cluster-policy
    type: cluster
  spec:
    enabled: true
    mode: audit
    clusterSelector:
      matchFields:
        name: demo-cluster
    rulesets:
    - demo-cluster_ruleset
    response:
      default:
      - makeRedFlag:
          severity: high
      actions: []
```

You can edit or add rules if you wish, or you can `apply` the policy at this point. To apply this policy, run the following command:

```sh
spyctl apply -f FILENAME
```

For example:

```
$ spyctl apply -f cluster-policy.yaml
Successfully applied new cluster ruleset with uid: rs:xxxxxxxxxxxxxxxxxxxx
Successfully applied new cluster guardian policy with uid: pol:xxxxxxxxxxxxxxxxxxxx
```

To confirm that your policy applied successfully you can run the following command:

```sh
spyctl get policies --type cluster
```

And to view your cluster-rulesets you can run the command:

```sh
spyctl get rulesets --type cluster
```

For example:

```
$ spyctl get policies --type cluster
UID                       NAME                 STATUS    TYPE       VERSION  CREATE_TIME
pol:xxxxxxxxxxxxxxxxxxxx  demo-cluster-policy  Auditing  cluster          1  2024-05-06T19:22:43Z
$
$ spyctl get rulesets --type cluster
UID                      NAME                   TYPE       VERSION  CREATE_TIME           LAST_UPDATED
rs:xxxxxxxxxxxxxxxxxxxx  demo-cluster_ruleset   cluster          1  2024-05-06T19:22:42Z  2024-05-06T19:22:42Z
```

## \[Optional] Adding "Interceptor" Response Actions

By default, Cluster Policies have a single response action `makeRedFlag` this action generates a redflag that references a deviant object. For example, if a container violates one of the ruleset rules, a redflag will generate for that container object. Redflags are used to populate security dashboards within the Spyderbat Console, but may also be forwarded to a SIEM and/or used to trigger notifications.

Containers that violate a cluster policy rule can also be used to trigger the `agentKillPod` response action. You can add a default action to kill the pod of any violating container by editing the policy yaml:

```sh
spyctl edit policy demo-cluster-policy
```

Then, under the `response` section of the `spec` you can add a new default action:

```yaml
response:
  default:
  - makeRedFlag:
      severity: high
  - agentKillPod:
  actions: []
```

Alternatively, you can scope the kill pod action to a sensitive namespace:

```yaml
response:
  default:
  - makeRedFlag:
      severity: high
  actions:
  - agentKillPod:
      namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: MY_CRITICAL_NAMESPACE
```

## Reviewing Policy Activity

Using the `spyctl` logs command, you can see what sorts of activity are going on within the scope of your policy.

```sh
spyctl logs policy NAME_OR_UID
```

for example:

```
$ spyctl logs policy demo-cluster-policy
(audit mode): Container image "docker.io/guyduchatelet/spyderbat-demo:2" ns:"rsvp-svc-dev" cluster:"demo-cluster" deviated from policy "integrationc3_policy".
(audit mode): Would have initiated "makeRedFlag" action for "cont:8vuJRMgyTEs:AAYXziCHi5g:31961a985651". Not initiated due to "audit" mode.
```

## Summary and Next Steps

At this point you should have an applied Cluster Policy in `audit` mode. This means that your policy is in a learning phase, it will generate logs and deviations, but will not take any response actions. After you feel the policy has stabilized (not generating deviations or generating them rarely) you can set the policy to `enforce` mode.

You can create Cluster Policies for any other Kubernetes Clusters you manage.

For additional details on ruleset rules view the [Ruleset Reference Guide](/reference/rulesets). There you can find out additional scoping options and rule targets.

For additional details on managing policies (updating, disabling, deleting) see the [Guardian Policy Management Reference Guide](/reference/spyctl/guardian-policy-management-spyctl)


# Integrations

Step-by-step guides for integrating Spyderbat with external tools and SIEMs.

| Tutorial                                                                                                | What it covers                                                                            |
| ------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart)                        | End-to-end setup: enable a saved query, install the Event Forwarder, verify events arrive |
| [Panther Webhook Configuration](/tutorials/integrations/forwarder-panther-config)                       | Configure the Event Forwarder to send events to Panther via HTTP webhook                  |
| [Falco Connector](/tutorials/integrations/falco-connector-deployment)                                   | Ingest Falco alerts into Spyderbat                                                        |
| [API Key Setup](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api) | Create and use a Spyderbat API key                                                        |


# SIEM Forwarding Quickstart

Set up SIEM forwarding end to end — enable a saved search, install the Event Forwarder, and verify events reach your SIEM.

This guide walks you through the full SIEM forwarding setup: enabling forwarding on a saved search, installing the Event Forwarder, and confirming that events are flowing to your destination.

## Prerequisites

* Spyderbat Nano Agents deployed and healthy on the hosts you want to monitor
* `org:ManageSiemForwarding` permission (contact your org admin if you don't have it)
* A destination ready to receive events: a SIEM, Splunk instance, webhook endpoint, or a Linux host with disk space for log files

## Step 1: Enable SIEM forwarding on a saved query

The Event Forwarder only delivers records that a saved search has already selected for forwarding. Complete this step first — skipping it means the forwarder has nothing to deliver.

If you don't have a saved search yet, [create one first](/concepts/search/saved-search). Two common starting points:

* **Forward targeted events** (recommended): create a query scoped to what you actually care about — for example, schema `model_spydertrace` with filter `score > 50` for high-scoring traces, or schema `event_redflag` with filter `severity = "high"` for high-severity flags only.
* **Forward everything** (use with caution — high volume): use filter `*` to match all records in a schema. Start with `model_spydertrace` to get behavioral traces, or `event_redflag` to get all security flags.

Then enable the **SIEM Forwarding** toggle on at least one saved search. See [SIEM Forwarding — Enable forwarding on a saved query](/concepts/integrations/siem-forwarding#step-1-enable-forwarding-on-a-saved-query) for detailed instructions.

To confirm forwarding is enabled, open **Saved Searches**: expand the **Search** icon in the sidebar, then click **Saved Searches**. Scroll right in the table to the **SIEM Forwarding** column — it should show a checkmark (✓) for your query. (The column may be off-screen at standard viewport widths.) Alternatively, click the **SIEM Forwarding** filter button at the top of the table to show only SIEM-enabled queries.

## Step 2: Install the Event Forwarder

Choose your deployment method:

**Kubernetes:** Follow the [Helm Chart guide](/installation/spyderbat-event-forwarder/helm-chart).

**Linux:** Follow the [Traditional Installer guide](/installation/spyderbat-event-forwarder/traditional-installer).

Both guides require these two values from your Spyderbat console:

```yaml
spyderbat_org_uid: YOUR-ORG-UID
spyderbat_secret_api_key: YOUR-API-KEY
```

Your org UID appears in the URL of any org-level page (for example, the Dashboard). Navigate to such a page first — the API Keys page (`/app/user/apikey`) does not include the org UID in its URL. To generate an API key, see [How to Create and Use a Spyderbat API Key](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api).

{% hint style="warning" %}
A wrong org UID produces zero events with no error message. If your API key has access to multiple orgs, double-check that you're using the UID for the org where you enabled SIEM forwarding.
{% endhint %}

{% hint style="info" %}
Create a dedicated service account for the API key rather than using a personal account. API keys can expire — check your key's expiration date and rotate before it lapses to avoid a forwarding outage.
{% endhint %}

## Step 3: Verify events are flowing

**Linux:**

```bash
sudo tail -f /opt/spyderbat-events/var/log/spyderbat_events.log
```

You should see ndjson records appearing within a minute or two of activity that matches your saved query. Each record is one JSON object per line.

**Kubernetes:**

```bash
kubectl logs -f statefulset.apps/sb-forwarder-event-forwarder -n spyderbat
```

The forwarder logs status as JSON. Look for lines where the `"message"` field reports new record counts:

```
{"schema":"event_forwarder:meta:1.0.0","message":"5 new records (0 invalid, 5 logged)",...}
```

A log that shows the forwarder running but reports zero new records usually means no activity is currently matching the saved search — not a forwarder problem. Run the saved search manually from the Search page to confirm it returns results.

## Troubleshooting

**No events after several minutes:**

1. Check that SIEM forwarding is enabled on at least one saved search. Open **Saved Searches** (expand the **Search** icon in the sidebar → **Saved Searches**) and scroll right to the **SIEM Forwarding** column — it should show a checkmark (✓). Alternatively, use the **SIEM Forwarding** filter button at the top of the table to show only SIEM-enabled queries.
2. Run the saved search manually from the Search page. If it returns no results, no activity matches — adjust the query.
3. Confirm the forwarder is running. On Linux: `journalctl -fu spyderbat-event-forwarder.service`. On Kubernetes: check the pod logs above.
4. Verify credentials: a wrong org UID silently produces zero events (no error), while an invalid API key causes authentication errors in the forwarder logs.

**Service fails to start on Linux (crash loop):**

If the forwarder crashes repeatedly on startup, check the logs for a DNS or URL error:

```bash
sudo journalctl -u spyderbat-event-forwarder.service --no-pager -n 20
```

A common cause is an incorrectly formatted `api_host` — the value must be a hostname only, without a scheme (correct: `api.example.com`, not `https://api.example.com`). After fixing the config, clear the failed state before restarting:

```bash
sudo systemctl reset-failed spyderbat-event-forwarder.service
sudo systemctl start spyderbat-event-forwarder.service
```

**Duplicate events:**

Run only one Event Forwarder instance per organization. Multiple instances poll the same API endpoint independently — each one receives and delivers the full event stream, causing duplicates in your SIEM.

**Events stopped flowing after working previously:**

API keys can expire. Check the forwarder logs for authentication errors, then generate a new key and update the config.

## Related pages

* [SIEM Forwarding](/concepts/integrations/siem-forwarding) — full architecture and reference
* [Spyderbat Event Forwarder](/concepts/integrations/spyderbat-event-forwarder) — detailed forwarder reference
* [Panther Webhook Configuration](/tutorials/integrations/forwarder-panther-config)
* [Saved Searches](/concepts/search/saved-search)


# Panther Webhook Configuration

Configure the Spyderbat Event Forwarder to send events to Panther via webhook.

This guide covers the Panther-specific steps for configuring the Event Forwarder webhook. Before continuing, complete the [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart) to install and configure the Event Forwarder.

## Step 1: Create the Panther schema

Panther requires an ingestion schema to parse incoming log data.

1. Download the [example Panther schema](https://raw.githubusercontent.com/spyderbat/event-forwarder/main/panther/Custom.SpyderbatR0.schema.yaml) from the event-forwarder repository.
2. In the Panther console, go to **Configure > Schemas** and click **Create New**.
3. Give the schema a name, such as `SpyderbatR0`.
4. Paste the schema contents into the text box.
5. Click **Validate**, then **Save**.

## Step 2: Create the Panther log source

1. In the Panther console, go to **Configure > Log Sources** and click **Create New**.
2. Select **Custom log formats**, then click **Start** under **HTTP logs**.
3. Enter a name for the source — for example, `Spyderbat Forwarder on <hostname>` (32-character limit).
4. Select the `Custom.SpyderbatR0` schema you created.
5. Set the auth method to **Bearer** and click the refresh button to generate a bearer secret. Copy the secret immediately — it cannot be retrieved after you leave this screen.
6. Click **Setup**.

## Step 3: Convert the bearer secret to base64

The event forwarder expects the bearer secret in base64 format. Convert it with:

```bash
echo -n YOUR_SECRET | base64
```

{% hint style="warning" %}
The `-n` flag is required. Without it, `echo` appends a trailing newline to the secret, producing invalid base64 that causes silent authentication failures in Panther.
{% endhint %}

Keep the base64 output handy for the next step.

## Step 4: Configure the Event Forwarder webhook

Edit `/opt/spyderbat-events/etc/config.yaml` and add the webhook block. Replace the placeholders with your Panther HTTP ingest URL and the base64 bearer secret from the previous step.

```yaml
spyderbat_org_uid: YOUR-ORG-UID
spyderbat_secret_api_key: YOUR-API-KEY

webhook:
  endpoint_url: PANTHER-INGEST-URL
  compression_algo: zstd
  max_payload_bytes: 500000
  authentication:
    method: bearer
    parameters:
      secret_key: YOUR-BASE64-SECRET
```

Restart the service to apply the config:

```bash
sudo systemctl restart spyderbat-event-forwarder.service
```

Tail the logs to confirm events are reaching Panther without errors:

```bash
sudo journalctl -fu spyderbat-event-forwarder.service
```

## Step 5: Verify events in Panther

In the Panther console, go to **Configure > Log Sources** and confirm a recent ingest timestamp appears for your Spyderbat log source. If the timestamp doesn't update within a few minutes, check the forwarder logs for authentication errors — the most common cause is an incorrectly encoded bearer secret (see Step 3).

## Related pages

* [SIEM Forwarding](/concepts/integrations/siem-forwarding) — full architecture and setup
* [SIEM Forwarding Quickstart](/tutorials/integrations/siem-forwarding-quickstart) — end-to-end setup guide
* [Spyderbat Event Forwarder](/concepts/integrations/spyderbat-event-forwarder) — architecture overview


# How to Set Up Spyderbat to Ingest Falco Alerts

Let's talk about the most optimal way to integrate Falco security detections with Spyderbat platform to further enhance the cloud-native runtime security monitoring value of Spyderbat.

<mark style="color:blue;">Last Updated: August 16, 2024</mark>

You can enhance Spyderbat detections by integrating with the Falco detection rule sets to add more security context to Spyderbat traces and living causal maps, including process details, user sessions, and network connections.

By integrating with Falco Sidekick, you will be able to identify, collect, and send Falco events to the Spyderbat platform and view them as well as take action within Spyderbat UI.

Spyderbat offers a simple deployment approach, and all the needed deployment instructions can be viewed here as well as retrieved via the public GitHub repository.

### Infrastructure Prerequisites

As a minimum, the user should have an organization set up in the Spyderbat Community Edition. You can go to <https://www.spyderbat.com/start-free> and request a free trial to install up to 5 Spyderbat nano agents.

The Spyderbat Nano Agent must be installed on the machines that you wish to monitor using Falco rule sets. The Spyderbat Nano Agent leverages eBPF technology on Linux systems to gather data and forward it to the Spyderbat backend. A full list of supported Linux OS can be found on our website [here](/installation/spyderbat-nano-agent) (paragraph 4).

Please refer to [the following guide](/installation/spyderbat-nano-agent/kubernetes) on how to install Spyderbat Nano Agent into a Kubernetes cluster.

Falco does not have to be installed in your environment prior to Spyderbat integration, as it will be taken care of as part of the integration process. We will provide instructions below on how to handle the integration without Falco running yet, as well as if Falco is already in place. For reference, here is the [official installation guide](https://github.com/falcosecurity/charts/tree/master/charts/falco/) available on the Falco Helm chart repository.

### Installing Falco Sidekick Using Helm Chart

You can configure the Falco Sidekick daemon to connect Falco to your existing ecosystem, which will allow you to take Falco-generated events and forward them to your Spyderbat platform to be seamlessly integrated with the Spyderbat security content and displayed in the causal activity graphs in Spyderbat Investigation UI to supplement and further enrich Spyderbat output.

If you do not already have Falco installed, you can install it and configure it to use the Spyderbat integration at the same time. First, add the Falco security helm chart repository:

```sh
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
```

Then, install Falco and the Spyderbat integration with:

```sh
helm install falco falcosecurity/falco \
    --create-namespace \
    --namespace falco \
    --set falcosidekick.enabled=true \
    --set falcosidekick.config.spyderbat.orguid="YOUR_ORG_ID" \
    --set falcosidekick.config.spyderbat.apiurl="https://api.spyderbat.com" \
    --set falcosidekick.config.spyderbat.apikey="YOUR_API_KEY" \
    --set extra.args=\{"-p","%proc.pid"\} \
    --set driver.kind=modern_ebpf
```

If you already have Falco installed through the Helm chart, changing `helm install` to `helm upgrade` should update it properly. Make sure to include any existing custom configuration that you are using for Falco or the Sidekick Pod.

The "orguid", which stands for Unique Organization ID, is specific to your organization and can be retrieved from the Spyderbat UI URL once you log into your console. Look for the organization ID in the URL path after `/app/org/`:

```
https://app.spyderbat.com/app/org/<YOUR_ORG_ID>/dashboard
```

Aside from enabling and configuring the Spyderbat integration, these configuration options enable additional ID information in the Falco event messages that Spyderbat uses to tie them into our existing context. It also sets the driver type to `modern_ebpf` instead of the default kernel driver. If your machine does not support the new driver, you may need to remove that argument.

Please refer to the KBA [“How to Set Up Your Spyderbat API Key and Use the Spyderbat API”](/tutorials/integrations/how-to-set-up-your-spyderbat-api-key-and-use-the-spyderbat-api) for more information on Spyderbat API use. For your convenience, the main steps for API Key generation are listed below:

1. Login into the Spyderbat console
2. Click on your User icon in the upper right corner and go to the “API Keys” section
3. If you do not have any active API keys, click “+ Create API Key” and save it in your user profile
4. Once generated, copy the API key into the clipboard

### Validation

If the installation proceeded correctly, you should receive no error messages and can run the following command to validate that all pods deployed successfully:

```sh
kubectl get pods --all-namespaces
```

Verify that the `falco` and `falco-falcosidekick` pods are running in the `falco` namespace:

```
NAMESPACE   NAME                                  READY   STATUS    RESTARTS   AGE
falco       falco-xxxxx                           2/2     Running   0          2m
falco       falco-falcosidekick-xxxxx-xxxxx       1/1     Running   0          2m
```

Once Falco starts detecting suspicious activity, respective “FALCO” labeled flags will be generated in the Spyderbat data stream and made visible in the Spydergraph Investigation section. These Flags can be located by running a search query. You will select the “Search” option in the left-hand navigation menu, run your search query, and then select a Flag you wish to investigate on a visual causal graph by checking the box and clicking “Start Investigation”:

![An example of searching for Falco objects](/files/8010u0oDaCjJ36q7Ybbj)

Once you click the "Start Investigation" button, you will be redirected to the [Investigation](/concepts/flashback/investigations) page where you will be able to see the selected flags and all associated processes as well as other security content.

You can also locate these flags by applying filtering options to our default Flags Dashboard and selecting the flags to start an Investigation this way:

![An example of a dashboard filtering rule for Falco.](/files/SlzNjryZbN3ByQPrusPo)

Please refer to our [Spyderbat Overview Video](https://www.youtube.com/watch?v=h2m-Jn3AgDc) for a more detailed walkthrough of the UI and its key functionality.

To stay on top of incoming Falco findings, you can create a custom dashboard card to pull in all Falco flags with desired severity by building the following search query for Redflag objects:

```
short_name = "falco_flag"
```

Once you have run your search, you can save the output as a custom dashboard card to be easily accessible through the UI:

![An example search, highlighting the "Save Dashboard Card" button](/files/YWMjjSe0mH39ha3WcENS)

Falco flags will differ in severity values that are mapped to Spyderbat severity values as follows:

| Falco Severity Value | Spyderbat Severity Value |
| -------------------- | ------------------------ |
| Emergency            | Critical                 |
| Critical             | Critical                 |
| Alert                | High                     |
| Error                | High                     |
| Warning              | Medium                   |
| Notice               | Low                      |
| Informational        | Info                     |
| Debug                | Info                     |

Note that the free Spyderbat Community account allows you to monitor up to 5 nodes, i.e. register up to 5 sources in the Spyderbat UI. If you have a cluster that contains more than 5 nodes or anticipate scaling up in the near future, please visit <https://www.spyderbat.com/pricing/> to sign up for our Professional tier.


# How to Create and Use a Spyderbat API Key

Spyderbat leverages API very heavily. To configure a variety of inbound and outbound API integrations, you will need to generate your API key. Learn how to create, maintain and manage your API keys.

Setting up your API key for a user account is necessary to be able to leverage the Spyderbat API. This document outlines how to set up your first API key and perform a basic operation against the Spyderbat API to test it.

### A Note on Spyderbat RBAC

Spyderbat uses Role Based Access Control for user accounts, and an API key is bound to a particular user account. A user account may belong to one or more organizations and maps to a role within each organization. The role determines which API operations are permitted — see the [Spyderbat API documentation](https://api.spyderbat.com/openapi.pdf) for a full list of API operations and their required roles. Some features (such as SIEM forwarding) require specific permissions like `org:ManageSiemForwarding` — contact your org admin if you need access to a specific capability.

### How to Create Your API Key

For the account you are using (Admin or Read Only) – you can click on the account icon in the top right corner of the UI and you will see an “API Keys” Link

<figure><img src="/files/hE3RAbDmPwOM4v8BFItF" alt="API key menu in account dropdown"><figcaption></figcaption></figure>

Click on the “API Keys” link and you will be taken to a page where you can create your API Key(s)

<figure><img src="/files/lYwjRSdUtXKHJLtpCN7N" alt="API Keys management page"><figcaption></figcaption></figure>

This will bring up a modal box where you can give the key you want to create a name and click on save.

<figure><img src="/files/ZZCZgkbM4TnG44weikjT" alt="Create API key modal dialog"><figcaption></figcaption></figure>

Once the API key has been created, copy the full API key value for use with the Spyderbat API. The key is masked in the list (`****xxxx`) — to retrieve the full value, click the **Example Usage** button (the middle icon of the three icons per row) next to the key. The full API key appears as a Bearer token in the `Authorization` header of the curl example. Copy the token value (starting with `eyJ...`).

{% hint style="info" %}
API keys can have an expiration date. Check your key's expiration date and rotate before it lapses to avoid an unexpected loss of access.
{% endhint %}

{% hint style="info" %}
The API key is not destroyed after navigating away — you can always retrieve the full value again from the **Example Usage** dialog.
{% endhint %}

### Retrieving Your Organization ID

In addition to the API key, you will also need your "Organization ID" (or Org ID) to leverage the Spyderbat API. This ID is a unique identifier for your organization. To find your Org ID, navigate to an org-level page in the Spyderbat Console — for example, the **Dashboard** — and examine the URL in your browser. Your Org ID is the string between `org/` and the next `/` in the path:

```
https://app.spyderbat.com/app/org/P6V31v0uIG5dtqXTHLsd/dashboard
```

So the Org ID in this example is **P6V31v0uIG5dtqXTHLsd**.

{% hint style="info" %}
The **Example Usage** dialog (opened from the middle icon next to any API key) includes an org dropdown and a **Copy org ID** button. If your account belongs to multiple orgs, this is often the easiest way to grab the correct org UID — and it lets you verify you're copying from the right org before you proceed.
{% endhint %}

{% hint style="warning" %}
The API Keys page (`/app/user/apikey`) does not include your org UID in its URL. Navigate to a different page — such as the Dashboard — to find it.
{% endhint %}

If your account has access to multiple organizations, make sure you copy the Org ID for the correct org. Using the wrong org UID produces zero results with no error message.

Copy your specific Org ID for reference.

### Testing the Spyderbat API with a Basic Example

Now that you have created your API key and have your Org ID, you can query the Spyderbat API with the simple examples below (for more details on the API see the [Spyderbat API documentation](https://api.spyderbat.com/openapi.pdf))

To list all the organizations that my user is part of where **API\_key** is the API key you created above:

<pre><code><strong>curl https://api.prod.spyderbat.com/api/v1/org/ -H "Authorization: Bearer API_key"
</strong></code></pre>

To list all the sources/agents in an organization, where **API\_key** is the API key you created above and **Org\_id** is the Org ID you retrieved for your organization:

<pre><code><strong>curl https://api.prod.spyderbat.com/api/v1/org/Org_id/source/ -H "Authorization: Bearer API_key"
</strong></code></pre>


# Notifications


# How to Set Up Notifications Using Spyctl

Configure Notifications using Spyctl to receive alerts for significant Security or Operations events.

### Prerequisites

* [Install Spyctl](/installation/spyctl)
* [Configure Spyctl with a Context](/installation/spyctl/spyctl-initial-configuration)
* [Spyderbat Notification Concept](/concepts/notifications)

### Overview

Spyderbat's notification system has 3 main components:

* **Notification Targets**: Named destinations to where notifications can be sent.
* **Notification Templates**: Pre-built templates containing most of the information required to create a Notification Config. These templates simplify the configuration process.
* **Notifications**: Allows to you configure notifications to the targets for the Notifiable Objects.

For the the full documentation of the Spyderbat Notifications System refer to [this section](/reference/notifications).

### How to Set Up Notifications

Spyderbat allows you to configure notifications for a variety of resources and targets, enabling you to streamline your workflow and stay informed about important events. Here’s how you can set up notifications for different targets:

***

### Step 1: Identify Notification Targets/Target Template.

Before you can configure notifications, you need to decide where to send them. Spyderbat supports several Notification Targets, such as:

* **Email**
* **Slack Channel**
* **Webhook**
* **PagerDuty**

Ensure that you have already configured the Notification Targets for these destinations before proceeding.

All of the commands to manage `Notification Targets` using Spyctl can be found [here](/reference/spyctl/notification-targets-management-spyctl).

Each Notification Target can be mapped to an optional Custom `Notification Template`, which defines the structure of the notification. Pre-configured templates help streamline the setup process.

All of the commands to manage Notification Templates using Spyctl can be found [here](/reference/spyctl/notification-template-management-spyctl).

If no template is specified, Spyderbat applies its **default template**.

***

### Step 2: Access Notification Command Help

To learn more about the spyctl notifications commands and their usage, run the following command:

```sh
  spyctl notifications -h
```

This will display the following help message:

```
Usage: spyctl notifications [OPTIONS] COMMAND [ARGS]...

Configure notifications for a Spyderbat resource.

Commands:
  configure  Configure notifications for a Spyderbat resource.
  disable    Disable notifications for a Spyderbat resource.
  enable     Enable notifications for a Spyderbat resource.
  list       List notifications on a Spyderbat resource.
```

Notification command allows you to Configure, disable, enable, list Notifications. Let's learn them one by one below.

Use `spyctl notifications <command> --help` for more information about a given command.

***

### Step 3: Configure Notifications for a Resource

You can configure notifications for 3 resources within Spyderbat's Spyctl CLI, mainly **Saved Queries, Custom flags, and Agent Health.**

In this section we'll learn about Configuring Notifications for **Saved Queries** and **Custom flags**.

**To Configure Notifications for Agent Health** [Refer here](https://github.com/spyderbat/spyderbat-docs/tree/main/tutorials/notifications/how-to-set-up-agent-health-notifications-in-the-console.md)

To begin, use the spyctl notifications configure command. Below are the available commands and options:

**View Help for Notifications Command:**

To view the available options for configuring notifications, use the following command:

```sh
 spyctl notifications -h
```

This will display the following help message:

```sh
Configure notifications for a Spyderbat resource.

Commands:
  custom-flag  Configure notifications for a custom flag.
  saved-query  Configure notifications for a saved query.
  agent-health Configure notificatons for agent heatlh.

Usage:
  spyctl notifications configure [OPTIONS] COMMAND [ARGS]...

```

**For a Saved Query:**

To configure notifications for a saved query, use the following command:

```sh
 spyctl notifications configure saved-query -h
```

This will show the usage and options for configuring notifications for a saved query:

```sh
Usage: spyctl notifications configure saved-query [OPTIONS] NAME_OR_UID

  Configure notifications for a saved query.

Options:
  --target-map    Map target names to template names. Can be used multiple times. 
                  Usage: --target-map TGT_NAME=TEMPLATE_NAME
  --targets       The Name or UID of targets to send notifications to.
  --cooldown-by   The cooldown by field(s).
  --cooldown      The cooldown period in seconds.
  --is-disabled   Disable notifications.
```

To configure a saved query with a target, use the following command:

```sh
spyctl notifications configure saved-query query:uOabbGEeJ \
--targets "email-target"
```

In this example:

```
--targets specifies the target (e.g., email) for sending notifications.
You can also customize settings like --cooldown or --is-disabled.
```

Note: You can configure notifications for multiple targets separated by comma.

**For a Custom Flag:**

To configure notifications for a custom flag, use the following command:

```sh
spyctl notifications configure custom-flag -h
```

This will display the usage and options for configuring notifications for a custom flag:

```sh
Usage: spyctl notifications configure custom-flag [OPTIONS] NAME_OR_UID

  Configure notifications for a custom flag.

Options:
  --target-map    Map target names to template names. Can be used multiple times. 
                  Usage: --target-map TGT_NAME=TEMPLATE_NAME
  --targets       The Name or UID of targets to send notifications to.
  --cooldown-by   The cooldown by field(s).
  --cooldown      The cooldown period in seconds.
  --is-disabled   Disable notifications.
```

To configure a custom flag with a target, run:

```sh
spyctl notifications configure custom-flag flag:teauh \
--targets "pagerduty-target"
```

This command configures notifications for a custom flag, sending them to the specified target (e.g., PagerDuty).

With these commands, you can easily configure notifications for saved queries and custom flags, specifying targets, templates, and additional options like cooldown periods

***

### Step 4: Test Notifications

After configuring notifications, you can test whether they are properly set up using the below command. This can be done before enabling the notifications.

```sh
spyctl test-notification --target "email-alerts" \
--template "default-template"
```

This command sends a test notification to the specified target using the provided template. You can have to provide a JSON record file for the template values:

Example: For a Custom Flag (Redflag) - Provide the Redflag record For a Custom Flag (Opsflag) - Provide the Opsflag record For a Saved QUery - Provide the object record for which query is saved.

```sh
spyctl test-notification --target "slack-channel" \
--template "custom-template" \
--record-file test_record.json
```

| Option              | Description                                                                  |
| ------------------- | ---------------------------------------------------------------------------- |
| `-T, --target`      | Target name or UID to send a test notification to. **(Required)**            |
| `-P, --template`    | Template name or UID of the same type as the target. **(Required)**          |
| `-f, --record-file` | File containing a JSON record used to build the notification. **(Required)** |

You should be able to get a test notification in some-time after setting it up.

***

### Step 5: Enable or Disable Notifications

After configuring and testing the notifications, you can toggle their status with the following commands to receive actual real-time Notfications:

**Enable notifications:**

```sh
spyctl notifications enable [OPTIONS] COMMAND [ARGS]...
```

Example:

```sh
spyctl notifications enable saved-query query:PpEjGdOSUJ
```

**Disable notifications:**

```sh
spyctl notifications disable [OPTIONS] COMMAND [ARGS]...
```

Example:

```sh
spyctl notifications disable saved-query query:PpEjGdOSUJ
```


# How to Set up Agent-Health Notifications Using Spyctl

## Overview

The **`spyctl create agent-health-notification-settings`** command in Spyctl allows you to configure notifications for agent health events. This helps you stay informed about the status (Unhealthy, Offline, Online, Healthy) of agents in your environment.

## Prerequisites

Before configuring agent health notifications, ensure you have:

* Install Spyctl ([Installation Guide](/installation/spyctl))
* Configure Spyctl with a context ([Spyctl Initial Configuration](/installation/spyctl/spyctl-initial-configuration))
* Spyderbat Notifications [Spyderbat Notifications Overview](/concepts/notifications)
* What are Notification Targets? ([Notification Targets Management](/reference/spyctl/notification-targets-management-spyctl))

***

## Step-by-Step Guide

### Step 1: Identify Notification Targets

Before setting up agent health notifications, ensure you have configured notification targets. These can include:

* Email
* Slack Channel
* Webhook
* PagerDuty

***

### Step 2: Create an Agent Health Notification Setting

Use the below command to create a new notification setting. Once configured, agent health alerts are received in real-time on the chosen targets.

#### Available Options

```
spyctl create  agent-health-notification-settings -h
```

| Option                   | Description                                                                           |
| ------------------------ | ------------------------------------------------------------------------------------- |
| `-a, --apply`            | Apply the agent health notification settings during creation.                         |
| `-n, --name`             | Custom name for the agent health notification settings. **(Required)**                |
| `-d, --description`      | Description of the agent health notification settings.                                |
| `-q, --scope-query TEXT` | SpyQL query on `model_agents` table to determine which agents the setting applies to. |
| `-T, --targets`          | Comma-separated list of notification targets.                                         |
| `--is-disabled`          | Disable the agent health notification settings on creation.                           |
| \`-o, --output \[yaml    | json                                                                                  |
| `-y, --yes`              | Automatically answer yes to all prompts.                                              |

**Creating a agent-health-notification-settings**

```sh
spyctl create agent-health-notification-settings \
  --name "Agent Health Alerts" \
  --description "Alerts for agent health issues" \
  --targets "work-email"
```

This command creates an agent health notification setting named **Agent Health Alerts**, which triggers notifications for unhealthy agents and sends them to the specified targets.

***

### Step 3: Edit an Existing Notification Setting

To modify an existing agent health notification setting, use:

```sh
spyctl edit agent-health-notification-settings <NAME_OR_UID>
```

For example:

```sh
spyctl edit agent-health-notification-settings "Agent-Health Alerts"
```

***

### Step 4: List All Existing Notification Settings

To view all configured agent health notification settings, run:

```sh
spyctl get agent-health-notification-settings
```

To get a particular `agent-health-notification-settings` use \<NAME\_OR\_UID>

```sh
spyctl get agent-health-notification-settings -o json
```

***

### Step 5: Delete an Agent Health Notification Setting

To stop receiving agent health notification, use:

```sh
spyctl delete agent-health-notification-settings <NAME_OR_UID>
```

For example:

```sh
spyctl delete agent-health-notification-settings "Agent Health-Alerts"
```


# Dashboards

Learn about how to create custom Spyderbat dashboards and dashboard cards using Athena Search or existing dashboard templates, as well as how to share and manage access to the custom dashboards.

<mark style="color:blue;">Published: July 20, 2023</mark>

{% hint style="info" %}
If you are looking for information on out-of-the-box Spyderbat Dashboards, please check out [All About Spyderbat Dashboards](/concepts/dashboards) and [Spyderbat Dashboard Categories](/concepts/dashboards/spyderbat-dashboard-categories) articles.
{% endhint %}

In addition to a number of dashboard cards in 7 different default dashboard categories, Spyderbat users with [adequate permissions](/getting-started/spyderbat-user-and-role-management-overview) are able to create custom dashboard cards and categories for their organization in the Spyderbat UI.

In this article we will discuss:

* Creating a brand new dashboard card from scratch
* Creating a new dashboard card off an existing dashboard card
* Managing custom dashboards and dashboard cards

### **How To Create a Dashboard Card from Scratch**

Spyderbat allows you to create custom dashboard cards based on your specific search queries. Follow the steps below to create a custom dashboard card:

1. **Access the Search Section**:

* In the top-left corner of the Spyderbat interface, navigate to the Search section.
* The search section includes a list of popular queries to get started with.

  ![](/files/Y1DVRQkGxrM4W6DsHkyK)

2. **Build Your Search Query**:

* The search section provides predefined categories such as System, Operation, Security, and Kubernetes (K8s). Each category offers a variety of search objects/Schemas with relevant fields.

  ![](/files/I0B7hbJxng0EQy50VXvi)
* For example, if you're searching for Kubernetes Nodes, choose `Node` from the Kubernetes category. Click on Open Query Builder to start building your query.
* Select the fields that you need (e.g., Cluster Name, Node Name, etc.) available from the list. You can add additional rows and conditions to refine the query.

  ![](/files/MLMpjqkFhrOutsApIUf6)
* You can refer the Search doc [here](/concepts/search) on how to write Search queries.

3. **Set Filters and Execute the Query**:

* Once your query is built, click on Send to Search.
* Apply the appropriate time filter (e.g., last 24 hours, custom date range).
* Click Search to run the query and display the results.

4. **Save the Query as a Dashboard Card**:

* After the results are displayed, click on Save Dashboard Card.

  ![](/files/K0XlyNslLA4mYu8oH4Kr)
* You can either add the results to an existing dashboard card or create a New Dashboard Card. Both these options are available in the drop down list.
* Click Create to finalize and add your custom card to the dashboard.
* Your custom dashboard card will now be available in the Dashboard section, reflecting real-time data based on the query results.

### **How to Build a New Dashboard Card Off an Existing Card**

Perhaps, an easier way to create custom dashboard cards is by tweaking some of the existing out-of-the-box cards that are available in the Spyderbat UI.

For example, let’s take a look at one of the Security cards named “Recent Spydertraces with Score >50”. Assume you would like to prioritize your focus on Spydertraces with much higher severity scores of 100+. The quickest way to build out a dashboard card like that would be to take the existing card and click “Run In Search”:

![](/files/GhbJuWin2w1rC3kUdR4m)

You can see the full query and can easily find the parameter to modify, which would be the score:

Once you update the score value to “>100”, you can save this as a new Dashboard Card and place it into your custom Dashboard category of choice to be easily accessible. You can also set notifications to be alerted if there is data pulling into that custom card.

![](/files/g98Md8y9m3pdyJTGnQKK)

Use this method for minor query changes, else start building your query from Search section.

{% hint style="info" %}
One thing to keep in mind: you cannot edit a query in the custom dashboard card. If you saved a card and then decided to further tweak it, you will need to follow the steps outlined above: select a card you wish to modify, click “Run in Search”, update the query as desired, and save as a new dashboard card.
{% endhint %}

### **How to Manage Custom Dashboards**

Once you have created a number of custom dashboards and dashboard cards, they will be visible to all the users in your organization in the Spyderbat UI. Users with [adequate permissions](/getting-started/spyderbat-user-and-role-management-overview) will be able to rename dashboards and cards, add new dashboard cards to customer dashboards created by other users and delete dashboard cards and entire dashboards.

All dashboard management options can be accessed by clicking on the “pencil” icon in the upper right corner of the custom Dashboard, you wish to modify:

![](/files/9yAYis5J5wt10sNkD7j1)

![](/files/XLBq6zdZXdiiS533UPht)

Here you can do a number of things:

* See if you have configured notifications for any of the cards in your custom dashboard.
* Hide a dashboard card from view by using the on/off slider on the left side of the dashboard card name
* Delete a dashboard card by clicking the “x” icon (Note: you will not be prompted to confirm your deletion, but it won’t be applied until you click “Save” in the lower right corner of the Edit window)
* Rename a dashboard card by clicking a “pencil” icon and then a “save” icon that looks like a floppy disk
* Change the order of appearance for the dashboard cards within the dashboard by dragging and dropping the “=” on the right hand side of the respective dashboard card names
* Rename the Dashboard category by clicking the ellipsis (three vertical dots) and selecting the “Edit Dashboard Name” option

![](/files/d37THCaPghJsI82r4utR)

All these changes will only apply after you click “Save” and will be in effect for all users in your organization, as stated on the next screen:

![](/files/0MUFrtucS6ExD2yk6N3R)

You can also delete an entire dashboard with all the cards in it by selecting “Delete Dashboard + Cards”, in which case you will need to confirm your decision:

![](/files/oqisWUHNUryYanDmCfgz)

Custom user-created dashboards will appear in the front positions of the category menu, pushing all default out-of-the-box categories to the right. The order of dashboard categories cannot be modified at this time, and categories cannot be hidden from view.


# Miscellaneous


# Scout (Detections)


# How to Set Up Custom Flags Using Spyctl CLI

### How to Set Up Custom Detections Using Spyctl CLI

Setting up custom detections using the Spyctl CLI is straightforward. Before you start, ensure you have the Spyctl CLI installed and your organization set as a Spyctl context. You can follow the guide [here](https://docs.spyderbat.com/installation/spyctl/spyctl-initial-configuration) for more details.

The Spyctl CLI supports various operations for custom flags (also known as custom detections), including creating, editing, deleting, disabling, and enabling. In this section, we'll go through these operations one by one.

#### 1. Create a Custom Detection

The `create` command for custom flags allow you to create a custom detection using Spyderbat Query Language (SpyQL) in Spyctl CLI. Spyctl provides help options (`--help`) to guide you for every command. To view the help for creating a custom flag, run:

```
$ spyctl create custom-flag --help
Create a custom flag from a saved query.

This command allows you to write custom detections using the Spyderbat Query
Language (SpyQL).

At a minimum you must provide the following:
- schema
- query
- description
- severity
- name

To view available schema options run:
  'spyctl search --describe'
To view available query fields for your schema run:
  'spyctl search --describe <schema>'
Query operators are described here:
  https://docs.spyderbat.com/reference/search/search-operators

Example:
spyctl create custom-flag --schema Process --query "interactive = true and container_uid ~= '*'" --description "Detects interactive processes in containers" --severity high interactive-container-process

Options:
  -o, --output [yaml|json|ndjson|default]
  -a, --apply                     Apply the custom flag during creation.
  -d, --description               A description explaining what the flag
                                  detects.  [required]
  -q, --query                     Objects matching this query + schema
                                  combination will be flagged. If used, this
                                  will create a saved query.
  -s, --schema                    The schema for the SpyQL query used by the
                                  custom flag. If used, this will create a
                                  saved query.
  -Q, --saved-query               The UID of a previously saved query. If
                                  used, this will override the query and
                                  schema options.
  -t, --type                      The type of the custom flag. One of
                                  ['redflag', 'opsflag'].
  -S, --severity                  The severity of the custom flag. One of
                                  ['critical', 'high', 'medium', 'low',
                                  'info'].  [required]
  -D, --disable                   Disable the custom flag on creation.
  -T, --tags                      The tags associated with the custom flag.
                                  Comma delimited.
  -i, --impact                    The impact of the custom flag on the
                                  organization.
  -c, --content                   Markdown content describing extra details
                                  about the custom flag.
  -N, --saved_query_name          If a new saved query needs to be created,
                                  this overrides the auto-generated name.
  -y, --yes                       Automatically answer yes to all prompts.

Usage:
  spyctl create custom-flag [OPTIONS] NAME

```

To start, you must select the object you want to generate a flag for. This is done via the `--schema` option. You can view the list of available search schemas with the `spyctl search --describe` command.

Next you will want to craft a query for the schema you just selected. Each schema has a number of searchable fields, you can view them with Spyctl using the `spyctl search --describe SCHEMA` command. For example: `spyctl search --describe Process` or `spyctl search --describe model_process` both will retrieve the same results.

Using the above information let's create a simple custom flag for a K8s ReplicaSet having more than 6 replica instances:

```
$ spyctl create custom-flag replica-flag --schema "Replicaset" --query "spec.replicas > 6" -t "redflag" --severity "high" --description "A ReplicaSet running more than 6 replicas found"
```

**Explanation:**

* `replica-flag` - The name of the custom flag.
* `--schema "Replicaset"` - The schema used for the custom flag. To view available schemas/objects for creating custom flags, run $ spyctl search. The list includes processes, connections, all Kubernetes resource schemas, and more. You can also use `model_k8s_replicaset` for this option.
  * Note: Custom flags cannot be created for event\_deviation, event\_opsflag, event\_redflag, or model\_spydertrace Schemas.
* `--query "spec.replicas > 6"` - The SpyQL query used for the custom flag. The suggested method is to utilize the search functionality in the UI under the Search Section to identify and test the queries you want to flag. Once identified, you can copy and paste the query as a value for the -q option.
* `--type "redflag"` - The type of the custom flag. By default, the flag type is set to redflag
* `--severity "high"`- Specifies the perceived severity level of the flag.
* `--description "A ReplicaSet running more than 6 replicas found"` - A description of the custom flag.

You can also include other options like `--content` and `--impact` for the custom flag. These will show up in the console during an investigation. The YAML configuration generated by the create command will look like the example below. Verify the yaml before applying.

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatCustomFlag
metadata:
  name: replica-flag
  schema: model_k8s_replicaset
spec:
  enabled: true
  query: spec.replicas > 6
  flagSettings:
    type: redflag
    description: A ReplicaSet running more than 5 replicas found
    severity: high
```

This step only generates the YAML. The next step is to apply this flag.

**To apply the custom flag, you have two options:**

a. `Apply Immediately`: Run the same command as above and include the `--apply` flag to apply the flag immediately.

b. `Apply from a File`: Save the YAML configuration to a file and then apply it using following command: `spyctl apply -f FILENAME`

You should get *"Successfully applied new custom flag with uid: flag:*"\* after applying the flag. Once set up, custom flags operate in real-time, triggering immediate flag as the query is met.

#### 2. Get All Custom Flags

To retrieve all custom flags that were created, use the following command:

```bash
$ spyctl get custom-flags
```

You'll see a list of custom flags like this:

```
Getting custom-flags
Page 1/1
NAME                 UID       DESCRIPTION                                         SEVERITY      SCHEMA                     STATUS    AGE
replica-flag         flag:*    A ReplicaSet running more than 8 replicas found     high          model_k8s_replicaset       ENABLED   20m
```

#### 3. Edit a Custom Flag

You can edit a custom flag if required using the below command, by passing the flag ID or name.

```bash
$ spyctl edit custom-flag <NAME_OR_ID>
```

After editing the yaml and saving it, you should see:

> Successfully edited custom flag with uid: flag:\*

#### 4. Delete a Custom Flag

To remove custom flags that are no longer needed, use the below command:

```bash
$ spyctl delete custom-flag <NAME_OR_ID>
```

#### 5. Disable a Custom Flag

To temporarily turn off a custom flag without deleting it, use:

```bash
$ spyctl disable custom-flag <NAME_OR_ID>
```

#### 6. Enable a Custom Flag

If you need to re-enable a custom flag that has been disabled, use:

```bash
$ spyctl enable custom-flag NAME_OR_ID.
```


# Reference


# Policies

Policies are the main way for users to configure their Spyderbat environment. Policies provide users with a way to generate tailored alerts, and tune out noise. Currently, policies fall under 1 of 2 categories, guardian and suppression.

* Guardian Policies are designed to establish expected behavior of the resources within their scope be it Linux Services, Containers, or Kubernetes Clusters.
* Suppression Policies are a way of tuning out the noise of Spyderbat's built-in detections. While Spydertraces aim at reducing the number of alerts a user should investigate, varying factors can lead to situations where suppression policies are necessary.

| Policy Type   | Category          | Supported Selectors                                        | Supported Response Actions                                                                     | Supports Rulesets |
| ------------- | ----------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | ----------------- |
| linux-service | Guardian Workload | <p>Cluster<br>Machine<br>Service</p>                       | <p>makeRedFlag<br>makeOpsFlag<br>agentKillProcess<br>agentKillProcessGroup</p>                 | No                |
| container     | Guardian Workload | <p>Cluster<br>Machine<br>Namespace<br>Pod<br>Container</p> | <p>makeRedFlag<br>makeOpsFlag<br>agentKillProcess<br>agentKillProcessGroup<br>agentKillPod</p> | No                |
| cluster       | Guardian Ruleset  | Cluster                                                    | <p>makeRedFlag<br>makeOpsFlag<br>agentKillPod</p>                                              | Yes               |
| trace         | Suppression       | <p>Cluster<br>Machine<br>Trace<br>User</p>                 | N/A                                                                                            | No                |

## Related Pages

* [Response Actions](/reference/policies/response-actions) - Actions that can be taken by Guardian Policies.
* [Selectors](/reference/selectors) - Reference documentation on the various selector types.
* [Rulesets](/reference/rulesets) - Rulesets supported by some policies.


# Response Actions

Spyderbat’s Policy Response Actions provide a powerful mechanism for responding to security events and deviations within your environment. These actions allow you to automate responses, enforce security policies, and maintain operational integrity. Response actions fall under two main categories, Agent and Standard.

## Categories of Response Actions

1. Agent Response Actions:
   * Purpose: These actions are executed directly on machines where the Spyderbat Nano Agent is installed.
   * Targeted Scope: They allow for machine-specific responses.
   * Examples:
     * Kill a process.
     * Kill a pod.
     * Kill a process tree.
     * Renice a process.
2. Standard Response Actions:
   * Purpose: These actions generate security and operations flags.
   * Insights: They provide visibility into policy violations or anomalies. They serve as alerts that can be further processed by other systems or personnel.
   * Examples:
     * Creating red flags. (Security focused, can trigger Spydertraces)
     * Creating operations flags. (Operations focused, highlight potential problems with infrastructure)

## Actions

Response actions are defined within the spec field of policies.

For Example:

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  name: demo-cluster-policy
  type: cluster
spec:
  enabled: true
  mode: audit
  clusterSelector:
    matchFields:
      name: demo-cluster
  rulesets:
  - demo-cluster_ruleset
  response:
    default:
    - makeRedFlag:
        severity: high
    actions:
    - agentKillProcess:
        processSelector:
          matchFields:
            exe: /bin/bash
```

* The default section contains global actions that apply to the entire policy. Whenever a deviation occurs, default actions are taken if applicable.
* The actions section allows you to define more specific actions with selectors that narrow the scope of when the action should be executed.

### makeRedFlag Action

This action makes a security flag. The ultimate consumer of these types of flags are security personnel investigating an anomaly. They can trigger spydertraces and/or be used to trigger notifications.

| Supported Selectors |
| ------------------- |
| Cluster             |
| Machine             |
| Namespace           |
| Pod                 |
| Container           |
| Service             |
| Process             |

#### Fields

* severity: The priority level of the red flag. Can be `critical`, `high`, `medium`, `low`, or `info`.
* impact: \[Optional] A string describing the security impact should the flag be generated.
* content: \[Optional] A string containing markdown that can detail next steps or who to contact.

Example:

```yaml
response:
    default:
    - makeRedFlag:
        severity: high
    actions:
    - makeRedFlag:
      namespaceSelector:
        kubernetes.io/metadata.name: production
      severity: critical
      impact: Unexpected activity on this critical workload could be malicious and should be investigated immediately.
      content: '### Remediation
      1. Contact developer
      2. Confirm if activity is expected or not
      3. If not, conduct investigation
      '
```

### makeOpsFlag Action

This action makes an operations flag. The ultimate consumer of these types of flags are operations personnel responsible for maintaining infrastructure.

| Supported Selectors |
| ------------------- |
| Cluster             |
| Machine             |
| Namespace           |
| Pod                 |
| Container           |
| Service             |
| Process             |

#### Fields

* severity: The priority level of the operations flag. Can be `critical`, `high`, `medium`, `low`, or `info`.
* impact: \[Optional] A string describing the operations impact should the flag be generated.
* content: \[Optional] A string containing markdown that can detail next steps or who to contact.

Example:

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    actions:
    - makeOpsFlag:
      namespaceSelector:
        kubernetes.io/metadata.name: production
      severity: critical
      impact: This workload appears to be behaving abnormally, operations should investigate.
      content: '### Remediation
      1. Confirm configuration
      2. Deploy fix
      '
```

### agentKillPod

This action tells the Spyderbat Nano Agent to kill a deviant process.

| Supported Selectors |
| ------------------- |
| Cluster             |
| Machine             |
| Namespace           |
| Pod                 |
| Container           |
| Process             |

Examples:

Kill pods running deviant netcat processes.

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    actions:
    - agentKillPod:
        processSelector:
          matchFields:
            name: nc
```

Kill all pods with deviations

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    - agentKillPod:
    actions: []
```

### agentKillProcess

This action tells the Spyderbat Nano Agent to kill a deviant process.

| Supported Selectors |
| ------------------- |
| Cluster             |
| Machine             |
| Namespace           |
| Pod                 |
| Container           |
| Service             |
| Process             |

Examples:

Kill deviant processes running the /bin/bash executable.

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    actions:
    - agentKillProcess:
        processSelector:
          matchFields:
            exe: /bin/bash
```

Kill all deviant processes

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    - agentKillProcess:
    actions: []
```

### agentKillProcessGroup

This action kills a process an any other processes within the same process group (pgid).

| Supported Selectors |
| ------------------- |
| Cluster             |
| Machine             |
| Namespace           |
| Pod                 |
| Container           |
| Service             |
| Process             |

Examples:

Kill process group of deviant processes running the /bin/bash executable.

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    actions:
    - agentKillProcessGroup:
        processSelector:
          matchFields:
            exe: /bin/bash
```

Kill all deviant processes and their associated groups.

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    - agentKillProcessGroup:
    actions: []
```

### agentKillProcessTree

This action instructs the Spyderbat Nano Agent to kill a deviant process along with its descendants (child processes). It is used to terminate a process tree, ensuring that both the specified parent process and all its child processes are killed.

| Supported Selectors |
| ------------------- |
| Cluster             |
| Machine             |
| Namespace           |
| Pod                 |
| Container           |
| Service             |
| Process             |

Examples:

Kill a deviant process and all its descendants:

This example demonstrates how to use the `agentKillProcessTree` action to kill a deviant process along with its child processes. In this case, the process to be killed is the one running `/bin/bash`, and all descendant processes are also terminated.

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    actions:
    - agentKillProcessTree:
        processSelector:
          matchFields:
            exe: /bin/bash
```

Kill **all** deviant processes and their descendants:

This configuration will kill all deviant processes and their child processes without specifying any selectors.

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    - agentKillProcessTree:
    actions: []
```

### agentReniceProcess

This action allows the Spyderbat Nano Agent to adjust the priority of deviant processes by "renicing" them. The process's priority (or "nice value") can be changed to either increase or decrease its CPU scheduling priority.

| Supported Selectors |
| ------------------- |
| Cluster             |
| Machine             |
| Namespace           |
| Pod                 |
| Container           |
| Service             |
| Process             |

**Priority Range**: The `priority` value is an string that specifies the new priority (or "nice value") for the process. The valid range for priority is **-20 to 19**, where:

* **-20** is the highest priority (more CPU time),
* **19** is the lowest priority (less CPU time).

Note: The default nice value for a process in Linux is 0.

Examples:

Renice a deviant process by changing its priority:

To adjust the priority of a specific deviant process, use the `agentReniceProcess` action. In this example, the priority of a deviant process running the `/bin/bash` executable is set to **20**, which is a lower priority.

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    actions:
    - agentReniceProcess:
        priority: "20"
        processSelector:
          matchFields:
            exe: /bin/bash
```

Renice a specific process by name:

This example shows how to renice a process running the ping command. The priority is set to -1.

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    actions:
    - agentReniceProcess:
        priority: "-1"
        processSelector:
          name:
            - ping
```

Renice all deviant processes:

To renice all deviant processes to a specific priority you can configure the action as follows:

```yaml
response:
    default:
    - makeOpsFlag:
        severity: high
    - agentReniceProcess:
        priority: "10"
    actions: []
```

## Related Pages

* [Selectors](/reference/selectors) - Reference documentation on the various selector types.
* [Policies](/reference/policies) - The policies that use response actions.


# Rulesets

This section documents the various features of Spyderbat Rulesets. It explains how rules can be configured and scoped. It also details how rules are evaluated.

## What Are Rulesets

For a summary of Rulesets and Ruleset Policies see [Ruleset Policies Concepts](/concepts/guardian/ruleset_policies)

## Rules

Rules are defined as a list in the `rules` field of a Ruleset's `spec`.

```yaml
apiVersion: spyderbat/v1
kind: SpyderbatRuleset
metadata:
  createdBy: demo.user@spyderbat.com
  creationTimestamp: 1712787972
  lastUpdatedBy: demo.user@spyderbat.com
  lastUpdatedTimestamp: 1714162618
  name: demo-cluster-ruleset
  type: cluster
  uid: rs:xxxxxxxxxxxxxxxxxxxx
  version: 1
spec:
  rules: []
```

Each rule contains a `target`, `verb`, list of `values`, and optional selectors (for additional scoping).

* Target: what the rule is referring to within the scope of the policy. Targets are `RULE_TYPE::SELECTOR_FIELD`.
  * ex. `container::image` this means that we are allowing or denying containers using images specified in the values field.
* Verb: The currently available verbs for ruleset rules are `allow` or `deny`. Any object matching a deny rule will generate a Deviation.
* Values: This is the set of values that are allowed or denied. If the target is `container::image` then the values should be container images that are either allowed or denied.
* Selectors: Optional selectors that further define the scope of a single rule. For instance you may want a rule that defines allowed activity in a specific namespace within a cluster. *Different rule types support different selectors.*

{% hint style="info" %}
For a full breakdown of the available selectors see the [Selectors Reference Guide](/reference/selectors)
{% endhint %}

```yaml
namespaceSelector:
  matchExpressions:
  - {key: kubernetes.io/metadata.name, operator: In, values: [rsvp-svc-dev, rsvp-svc-prod]}
target: container::image
values:
- docker.io/guyduchatelet/spyderbat-demo:1
- docker.io/library/mongo:latest
verb: allow
```

### Container Rules

Container rules define which containers are allowed or denied.

| Supported Targets        |
| ------------------------ |
| container::image         |
| container::imageID       |
| container::containerName |
| container::containerID   |

| Supported Selectors |
| ------------------- |
| Cluster             |
| Machine             |
| Namespace           |
| Pod                 |
| Container           |

| Supported Verbs |
| --------------- |
| allow           |
| deny            |

Examples:

Allow the latest apache image in production and staging

```yaml
namespaceSelector:
  matchExpressions:
  - {key: kubernetes.io/metadata.name, operator: In, values: [staging, production]}
target: container::image
values:
- docker.io/apache:latest
verb: allow
```

Deny a specific image ID in production

```yaml
namespaceSelector:
  matchLabels:
    kubernetes.io/metadata.name: production
target: container::imageID
values:
- sha256@XXXXXXXXXXXXXXXXXXXXXXXX
verb: deny
```


# Selectors

Selectors are used in various places to scope policies, rules, and actions. Spyderbat's selectors based on [Kubernetes Labels and Selectors](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).

## Selector Primitives

Spyderbat's selectors offer set-based selector primitives.

|                            |                                                 |
| -------------------------- | ----------------------------------------------- |
| \*matchLabels              | user-defined key value pairs                    |
| \*matchExpressions         | contain a key, operator, and values             |
| \*\*matchFields            | pre-defined key value pairs                     |
| \*\*matchFieldsExpressions | key from pre-defined list, operator, and values |

* \* Matches the syntax from Kubernetes
* \*\* Unique to Spyderbat's Selectors

### Expressions

Expressions have 3 fields: `key`, `operator`, and `values`. They allow you to define set-based groupings.

Example:

```yaml
matchExpressions:
- key: app
  operator: In
  values: [apache, mysql]
```

In the example above whatever is being matched on, must have a label with a key `app` and the value of that label must be either `apache` or `mysql`.

#### Operators

Operators define how the set-based expression is to be evaluated.

|              |                                                           |
| ------------ | --------------------------------------------------------- |
| In           | The key must exist and the value must be in `values`      |
| NotIn        | The key must exists and the value must not be in `values` |
| Exists       | The key must exist                                        |
| DoesNotExist | The key must not exist                                    |

### Pattern Matching

Field values in `matchFields` and `matchFieldsExpressions` support Unix-style glob patterns for flexible matching.

| Pattern | Description                                      | Example                                                    |
| ------- | ------------------------------------------------ | ---------------------------------------------------------- |
| `*`     | Matches any number of characters (including `/`) | `systemd/*/curl` matches `systemd/crio/conmon/runc/6/curl` |
| `?`     | Matches exactly one character                    | `runc/?` matches `runc/6` but not `runc/runc`              |
| `[abc]` | Matches any one character in the set             | `runc/[69]` matches `runc/6` or `runc/9`                   |
| `[a-z]` | Matches any one character in the range           | `runc/[0-9]` matches `runc/6` but not `runc/runc`          |

{% hint style="warning" %}
The `*` wildcard matches across path separators (`/`). A pattern like `systemd/*/curl` will match process ancestor chains of any depth. Use specific path segments to constrain matches — for example, `systemd/crio/conmon/runc/*/curl` is safer than `systemd/*/curl`.
{% endhint %}

Values containing wildcards must be quoted in YAML:

```yaml
values:
  - "systemd/crio/conmon/runc/*/health-check/curl"
  - "*strimzi/kafka*"
```

## Pod and Namespace Selectors

Pod and Namespace selectors are defined the exact same way that Kubernetes Pod and Namespace selectors are. Both resources types can have user-defined labels that allow them to be grouped by selectors.

The labels are found within the Pod and Namespace object yaml.

| Supported Primitives |
| -------------------- |
| matchLabels          |
| matchExpressions     |

Examples:

```yaml
podSelector:
  matchLabels:
    app: apache
  matchExpressions:
  - {key: tier, operator: In, values: [frontend, backend]}
  - {key: test, operator: DoesNotExist}
```

```yaml
namespaceSelector:
  matchLabels:
    kubernetes.io/metadata.name: production
  matchExpressions:
  - {key: dedicated-node, operator: Exists}
```

## Other Selectors

The following selectors are Custom to Spyderbat's environment. They add an additional level on granularity to scoping operations.

| Supported Primitives   |
| ---------------------- |
| matchFields            |
| matchFieldsExpressions |

### Cluster Selector

The Cluster Selector allows for scoping by Kubernetes Cluster. Field values may be wildcarded with an `*` character.

| Supported Fields | Description                                                             |
| ---------------- | ----------------------------------------------------------------------- |
| name             | The name of the cluster as defined in Spyderbat                         |
| uid              | The Spyderbat-provided uid of the cluster generally begins with `clus:` |

Example:

```yaml
clusterSelector:
  matchFields:
    name: demo-cluster
```

### Machine Selector

The Machine Selector allows for scoping my Machine. A machine in this context is a device with the Spyderbat Nano Agent installed.

| Supported Fields | Description                                                             |
| ---------------- | ----------------------------------------------------------------------- |
| hostname         | The hostname of a host on the network                                   |
| uid              | The Spyderbat-provided uid of the machine generally begins with `mach:` |

Example:

```yaml
machineSelector:
  matchFieldsExpressions:
  - {key: hostname, operator: In, values: [test_node, staging_node]}
```

### Container Selector

The Container Selector allows for scoping by fields associated with containers.

| Supported Fields | Description                                                        |
| ---------------- | ------------------------------------------------------------------ |
| image            | The container's image name                                         |
| imageID          | The container's image hash                                         |
| containerName    | The name of a specific container instance (usually auto-generated) |
| containerID      | The ID of a specific container instance (usually auto-generated)   |

Example:

```yaml
containerSelector:
  matchFields:
    image: docker.io/apache
```

### Service Selector

The Service Selector allows for scoping by fields associated with Linux Services

| Supported Fields | Description                                                                                                  |
| ---------------- | ------------------------------------------------------------------------------------------------------------ |
| cgroup           | The cgroup that every process within the service falls under. Ex. `systemd:/system.slice/nano-agent.service` |
| name             | The simple name of the Linux service. Ex. `nano-agent.service`                                               |

Example:

```yaml
serviceSelector:
  matchFields:
    cgroup: systemd:/system.slice/nano-agent.service
```

### Trace Selector

The Trace Selector is used by Trace Suppression Policies to suppress Spydertraces within a specific scope.

| Supported Fields | Description                                                                    |
| ---------------- | ------------------------------------------------------------------------------ |
| triggerClass     | The class of flag that triggered the Spydertrace                               |
| triggerAncestors | The names of the ancestor processes of the flag that triggered the Spydertrace |

Field values support glob pattern matching including `*`, `?`, and character classes like `[0-9]`. See [Pattern Matching](#pattern-matching) for the full syntax.

#### Using matchFields

```yaml
traceSelector:
  matchFields:
    triggerClass: redflag/proc/command/high_severity/suspicious/netcat
    triggerAncestors: systemd/containerd-shim/runc/*/sh/python/sh/netcat
```

Use `matchFields` when you need to match a single value per field (with optional wildcards). When you need to match multiple possible values for the same field, use `matchFieldsExpressions` instead.

#### Using matchFieldsExpressions

Use `matchFieldsExpressions` when you need to match multiple patterns for the same field. This is useful when a workload runs across different container runtimes (e.g., CRI-O and containerd) that produce different process ancestry paths.

```yaml
traceSelector:
  matchFieldsExpressions:
    - key: triggerAncestors
      operator: In
      values:
        - "systemd/crio/conmon/runc/*/health-check/curl"
        - "systemd/containerd-shim/runc/*/health-check/curl"
    - key: triggerClass
      operator: In
      values:
        - "redflag/proc/command/high_severity/suspicious/curl"
```

Each value in the `values` array supports wildcards and is evaluated independently. A trace matches if its field value matches **any** of the listed values (`In` operator provides OR semantics).

The same [Operators](#operators) available for `matchExpressions` apply to `matchFieldsExpressions`: `In`, `NotIn`, `Exists`, and `DoesNotExist`.

### User Selector

The User Selector is used by Trace Suppression Policies to suppress Spydertraces triggered by a specific user or users.

| Supported Fields | Description                        |
| ---------------- | ---------------------------------- |
| user             | The username of the offending user |

Example:

```yaml
userSelector:
  matchFieldsExpressions:
  - {key: user, operator: NotIn, values: [admin, root]}
```

### Process Selector

The Process Selector is used to scope by fields associated with a Linux Process.

| Supported Fields | Description                                 |
| ---------------- | ------------------------------------------- |
| name             | The name of the process                     |
| exe              | The executable of the process               |
| euser            | The username of the process' effective user |

```yaml
processSelector:
  matchFields:
    exe: /bin/bash
```


# Notifications

This section documents the full capabilities of Notification Templates, and which fields exist to manipulate the template/behavior.

The [Conditions and Triggering Notifications](#conditions-and-triggering-notifications) section will briefly explain how the `condition` field interacts with the Spyderbat data model in order to trigger notifications.

The [Dereferencing Values](#dereferencing-values) section describes how you can inject values from the json record that triggered the notification into the notification itself.

The [Internal Functions](#internal-functions) section details the various functions you can use to add additional context to your notifications or manipulate existing values into a more desirable format.

{% hint style="info" %}
Note: The following are advanced concepts. An understanding of them is not required to get started with notifications. Follow the how-to guides to quickly setup commonly-used notifications.
{% endhint %}

## Data Model Primer

Notifications in Spyderbat are driven by the data model. The [Spyderbat Nano Agent](/installation/spyderbat-nano-agent) generates raw telemetry and sends it to the Spyderbat Analytics Engine. The Analytics Engine processes the raw data and builds the behavior web that is viewable in the Console. Additionally, the Analytics Engine analyzes data in the behavior web for security detections, operations issues, policy violations, and more.

The data emitted by the Analytics Engine comes in two flavors: models and events. Notifications are generated by evaluating these two types of records. Models are (potentially) long-lived objects that have a start, middle, and end in their lifecycle. Events represent detections or occurrences that happen at a single point in time.

Take processes as an example. Spyderbat receives process telemetry and builds models to track the state of the processes themselves. What gets emitted from Spyderbat looks like this:

### Process Model

```json
{
  "schema": "model_process::1.2.0",
  "id": "proc:_ZO7yNX2S54:ZWn4uw:753728",
  "version": 1701443785,
  "description": "bash [753728, normal] closed from 7dbab6f7-77de-494a-9490-564bc7174611",
  "cgroup": "systemd:/user.slice/user-1000.slice/session-966.scope",
  "time": 1701443785.3360424,
  "create_time": 1701443771.7009835,
  "valid_from": 1701443771.7009835,
  "muid": "mach:_ZO7yNX2S54",
  "pid": 753728,
  "ppid": 753726,
  "ppuid": "proc:_ZO7yNX2S54:ZWn4uw:753726",
  "tpuid": "proc:_ZO7yNX2S54:ZWn4uw:753726",
  "sid": "966",
  "args": [
    "/usr/bin/bash"
  ],
  "cwd": "/home/ubuntu",
  "thread": false,
  "type": "normal",
  "interactive": true,
  "environ": {},
  "duration": 9.437932014465332,
  "name": "bash",
  "title": "/usr/bin/bash",
  "auid": 1000,
  "euid": 0,
  "egid": 0,
  "container": null,
  "auser": "ubuntu",
  "euser": "root",
  "egrp": "root",
  "status": "closed",
  "data_is_complete": true,
  "ancestors": [
    "sudo",
    "bash",
    "sshd",
    "sshd",
    "systemd"
  ],
  "is_causer": false,
  "is_causee": false,
  "prev_time": 1701443781.1389155,
  "expire_at": 1701446399.999999,
  "exit": 0,
  "exe": "/usr/bin/bash",
  "valid_to": 1701443781.1389155,
  "traces": [
    "trace:_ZO7yNX2S54:AAYLdEBuJOo:753573:remote_access"
  ],
  "red_flag_count": 0,
  "red_flags": ["flag:629gia"],
  "ops_flag_count": 0,
  "red_flags": [],
  "schemaType": "model_process",
  "schemaMajorVersion": 1,
  "record_type": "model",
  "versionedId": "proc:_ZO7yNX2S54:ZWn4uw:753728:v1701443785"
}
```

The model above is for an interactive bash shell process running on a machine with the Spyderbat Nano Agent installed. It contains all the information required to add it into the behavior web. It also happens that this process is running with an effective user (`euser`) "root". That is a privileged account and we have a security detection when we see an interactive shell running as root.

## Conditions and Triggering Notifications

## Dereferencing Values

In Spyderbat's Notification Templates, you can dynamically include specific values from the JSON objects you are monitoring in the Template fields using dereferencing syntax. The syntax for dereferencing is as follows:

* For direct field access: `{{ field_name }}`
* For subfield access within a dictionary: `{{ parent_field.sub_field }}`

Let's consider an example JSON object:

Suppose you have a Saved Query for a Red Flag i.e a security detection on the root bash process above. Events generally have a `ref` field that points to the `id` of the model they're related to.

```json
{
  "id": "event_alert:_ZO7yNX2S54:ZWn4uw:753728",
  "schema": "event_redflag:root_shell:1.1.0",
  "description": "ubuntu as root ran unusual interactive shell '/usr/bin/bash'",
  "ref": "proc:_ZO7yNX2S54:ZWn4uw:753728",
  "short_name": "root_shell",
  "class": [
    "redflag",
    "proc",
    "root_shell",
    "critical_severity"
  ],
  "flag_class": "redflag/proc/root_shell/critical_severity",
  "severity": "critical",
  "time": 1701443781.1389155,
  "routing": "customer",
  "version": 2,
  "muid": "mach:_ZO7yNX2S54",
  "name": "bash",
  "auid": 1000,
  "args": [
    "/usr/bin/bash"
  ],
  "auser": "ubuntu",
  "euser": "root",
  "ancestors": [
    "sudo",
    "bash",
    "sshd",
    "sshd",
    "systemd"
  ],
  "mitre_mapping": [
    {
      "sub-technique": "T1059.004",
      "sub-technique_name": "Unix Shell",
      "url": "https://attack.mitre.org/techniques/T1059/004",
      "created": "2020-03-09T14:15:05.330Z",
      "modified": "2021-07-26T22:34:43.261Z",
      "stix": "attack-pattern--a9d4b653-6915-42af-98b2-5758c4ceee56",
      "technique": "T1059",
      "technique_name": "Command and Scripting Interpreter",
      "tactic": "TA0002",
      "tactic_name": "Execution",
      "platform": "Linux"
    }
  ],
  "impact": "A shell owned by root has a dangerous level of permissions.",
  "ppuid": "proc:_ZO7yNX2S54:ZWn4uw:753726",
  "false_positive": false,
  "traces": [
    "trace:_ZO7yNX2S54:AAYLdEBuJOo:753573:remote_access"
  ],
  "traces_suppressed": false,
  "schemaType": "event_redflag",
}
```

### **Examples of Dereferencing Values from the Object Above:**

Example with Email Template:

```
apiVersion: spyderbat/v1
kind: NotificationTemplate
metadata:
  name: email-template
  type: email
spec:
  subject: "Spyderbat Alert: {{ severity }} Severity Detected on {{ name }}"
  body_html: |
    <html>
      <body>
        <h4>Spyderbat Alert</h4>
        <p><strong>Severity:</strong> {{ severity }}</p>
        <p><strong>Description:</strong> {{ description }}</p>
        <p><strong>Process:</strong> {{ name }} (Executed by {{ auser }}, Effective user: {{ euser }})</p>
        <p><strong>Command:</strong> {{ args | join(" ") }}</p>
        <p><strong>Detection Time:</strong> {{ time }}</p>
        <p><strong>MITRE Technique:</strong> <a href="{{ mitre_mapping[0].url }}">{{ mitre_mapping[0].technique_name }}</a></p>
      </body>
    </html>
  body_text: |
    Spyderbat Alert
    Severity: {{ severity }}
    Description: {{ description }}
    Process: {{ name }} (Executed by {{ auser }}, Effective user: {{ euser }})
    Command: {{ args | join(" ") }}
    Detection Time: {{ time }}
```

These examples demonstrate how you can leverage dereferencing to dynamically include specific values from the JSON object in your notifications for more context on alert. Feel free to adjust the examples based on your specific use cases and requirements.

## Internal Functions

In Spyderbat's Notification Templates, you can enhance the template alert by using internal functions as well in each Template type. The syntax for using functions is as follows:

* `{{ __FUNCTION_NAME__ [| ARG1, ARG2, ..., ARGN] }}`

Arguments are optional, depending on the function used. The return value of the function will replace the `{{ __FUNCTION_NAME__ }}` placeholder, or an error message will be displayed if something goes wrong.

### Example JSON object used in function examples:

This metrics record is used to monitor the resource utilization of the Spyderbat Nano Agent.

```json
{
  "schema": "event_metric:agent:1.0.0",
  "id": "event_metrics:07Ax6uRpB606065sYozQ:ZWXXXX",
  "ref": "agent:07Ax6uRpB606065sXXXX",
  "version": 1,
  "muid": "mach:5sZN4f2mXXX",
  "time": 1701460978.1299076,
  "cpu_cores": 2,
  "total_mem_B": 8173600768,
  "hostname": "example_machine",
  "cluster_name": null,
  "bandwidth_1min_Bps": 1616,
  "cpu_1min_P": {
    "agent": 0.0417,
    "authUID": 0.0002,
    "bashbatUID": 0.0002,
    "grimreaperUID": 0.0034,
    "procmonUID": 0.0003,
    "scentlessUID": 0.0355,
    "snapshotUID": 0
  },
  "mem_1min_B": {
    "agent": 355326000,
    "authUID": 29440000,
    "bashbatUID": 37820000,
    "grimreaperUID": 42692000,
    "procmonUID": 38005000,
    "scentlessUID": 71352000,
    "snapshotUID": 45604000
  },
  "mem_1min_P": {
    "agent": 0.04347239485822658,
    "authUID": 0.003601839732038158,
    "bashbatUID": 0.004627091666633258,
    "grimreaperUID": 0.005223156991853704,
    "procmonUID": 0.004649725510058091,
    "scentlessUID": 0.008729567546208785,
    "snapshotUID": 0.005579425921870522
  },
  "missed_messages": {
    "scentless": {
      "pcap_dropped": 0,
      "data_drop_delayq": 0,
      "data_drop_dns": 0,
      "missed_events": 0
    }
  }
}
```

### Functions:

***

#### **{{ \_\_cluster\_\_ }}**

* *Arguments:* This function takes 0 arguments
* *Description:* Returns the name of the cluster the object is associated with or "No Cluster."

**Pagerduty Template spec Example:**

```yaml
spec:
  custom_details: 
    "cluster": "{{ __cluster__ }}"
```

***

This would result in a list displayed in the notification:

* Cluster: No Cluster

This is because in the metrics object above, the cluster\_name field is null.

***

#### **{{ \_\_hr\_time\_\_ }}**

* *Arguments:* This function takes 0 arguments
* *Description:* Returns a human-readable version of the time field found in the object.

**PagerDuty Template Spec Example:**

```yaml
spec:
  custom_details: 
    "time": "{{ __hr_time__ }}"
```

This would result in a list displayed in the notification:

* Time: 2023-12-01 20:02:58UTC

This converts the epoch time 1701460978.1299076 in the `time` field in the record above to something human readable.

***

#### **{{ \_\_linkback\_\_ }}**

* *Arguments:* This function takes 0 arguments
* *Description:* Returns a relevant URL linking back to the Spyderbat Console for the object being evaluated.

**PagerDuty Template Spec Example:**

```yaml
spec:
  custom_details: 
    "linkback": "{{ __linkback__ }}"
```

This would result in a linkback URL being generated, pointing to the Agent Health page for the agent referenced above `"ref": "agent:07Ax6uRpB606065sXXXX"`. It would display as a link "View in Spyderbat" at the bottom of your notification.

***

#### **{{ \_\_origin\_\_ }}**

* *Arguments:* This function takes 0 arguments
* *Description:* Returns a string explaining why the notification was generated.

**PagerDuty Template Spec Example:**

```yaml
spec:
  custom_details: 
    "origin": "{{ __origin__ }}"
```

This would result in a message like:

`Notification Origin:`\
`This notification was generated because an event_metric record matched the condition specified in notification config "Agent CPU Usage - notif:6voXLIYfRPmTky-XVAaXXX".`

***

#### **{{ \_\_percent\_\_ | number }}**

* *Arguments:* This function takes exactly 1 argument
  * number:
    * type: String or Number
    * description: If a String is supplied, the string must be a field in the object with a numerical value.
* *Description:* Multiplies an input number by 100, caps the precision at 2 decimal places, and appends a percent (%) symbol.

**PagerDuty Template Spec Example:**

```yaml
spec:
  custom_details: 
    "CPU Usage": "{{ __percent__ | cpu_1min_P.agent }}"
```

This would result in a list displayed in the notification:

* CPU Usage: 4.17%

**Here’s an all list of the field names along with their actual function:**

* `__hr_time__` – Human-readable timestamp of the event.
* `__time_int__` – Timestamp in integer format.
* `__linkback__` – URL linking back to the event in Spyderbat.
* `__origin__` – The origin or source of the event.
* `__cluster__` – The cluster where the event occurred.
* `__source__` – The source component or entity that generated the event.
* `__hostname__` – The hostname where the event took place.
* `__percent__` – A percentage value associated with the event.
* `__pd_severity__` – Severity level formatted specifically for PagerDuty.
* `__query_name__` – The name of the saved query that triggered the event.

Conclusion: You can use a mix of both Static values from the Object by dereferencing and using Spyderbat's Internal function to enhance templates for direct context on Notification Alert.


# Spyctl CLI


# Spyctl Commands

### Readthedocs

Spyctl's commands are documented on [readthedocs.io](https://spyctl.readthedocs.io/en/latest/commands.html)

### Spyctl CLI

If you have already installed Spyctl you can also see the command documentation by running `spyctl --help`. You can also run `spyctl <command> --help` for more information about a given command.


# Guardian Policy Management using Spyctl

This reference page details the commands used to manage Guardian Workload Policies

## Creating and Applying a Policy

See the tutorial: [How To Lock Down Your Critical Workloads With Policies using Spyctl](/tutorials/guardian/how-to-lock-down-your-critical-workloads-with-policies-spyctl)

## Updating A Policy

Over time, Policies will generate deviations. Your Linux services and containers will continue to generate activity. Some of that activity may deviate from your policy. Investigating a deviation can lead to one of two scenarios.

1. There is a legitimate threat take steps to remediate, or
2. This is additional benign activity that should be added to the policy.

This reference guide covers the second scenario.

### Viewing Deviations

Deviations come from processes or connections that deviated from your Guardian Workload Policies. They contain all of the information required to update your policy should you choose to merge them in. You can view Deviations with the `get` command:

```
spyctl get deviations [NAME_OR_UID]
```

For example:

```
spyctl get deviations
Getting policy deviations from 2024-01-15T23:06:33Z to 2024-01-16T23:06:33Z
UID                       NAME              STATUS     TYPE       CREATE_TIME           DEVIATIONS_(UNIQ/TOT)
pol:CB1fSLq4wpkFG5kWsQ2r  mongo-policy      Auditing   container  2024-01-16T15:00:43Z  2/33
```

### Viewing the Diff

To see how the merging the deviations into your policy would affect it, you can see a git-like diff with the following command:

```
spyctl diff [OPTIONS] -p [POLICY_NAME_OR_UID,POLICY_NAME_OR_UID2,...]
```

For example:

First, list the policies you have applied:

```
spyctl get policies
UID                       NAME              STATUS     TYPE       CREATE_TIME
pol:CB1fSLq4wpkFG5kWsQ2r  mongo-policy      Auditing   container  2024-01-16T15:00:43Z
```

Then select the one you want to diff:

```
spyctl diff -p pol:CB1fSLq4wpkFG5kWsQ2r
```

{% hint style="info" %}
The default diff query uses all deviations in the last 24 hours. You can use the `--latest` option to diff the policy against all deviations since the policy was last updated.
{% endhint %}

The output of the diff command will display a git-like diff of activity that doesn’t match the Policy. You can use the `merge` command to add the deviations to the Policy.

#### \[Optional] Bulk Diff

You may have many policies, and diffing each one individually might be tiresome. To systematically diff all of your policies, use the following command:

```
spyctl diff -p
```

You can also use the `-y` option to avoid any prompting.

### Merging in the Deviations

To update the your policies with known-good deviations you can use the `merge` command.

```
spyctl merge [OPTIONS] -p [POLICY_NAME_OR_UID,POLICY_NAME_OR_UID2,...]
```

For example:

```
spyctl merge -p pol:CB1fSLq4wpkFG5kWsQ2r
```

{% hint style="info" %}
The default merge query uses all deviations in the last 24 hours. You can use the `--latest` option to merge in all deviations since the policy was last updated.
{% endhint %}

You will have a chance to review any changes before they are applied.

### \[Optional] Bulk Merge

You may have many policies, merge in updates across all policies may be tiresome. To systematically merge in deviations across all of your policies, use the following command:

```
spyctl merge -p
```

You can use the `--yes-except` option to avoid all prompts except reviewing the final changes, and you can use the `-y` option to avoid all prompts entirely.

## Changing a Policy's Mode

Once your policy rarely produces deviations in `audit` mode you can change it to `enforce` mode. To change the Policy to `enforce` mode you must edit the yaml.

Use the `edit` command to edit the Policy's yaml.

```
spyctl edit RESOURCE NAME_OR_UID
```

For example:

```
spyctl edit policy pol:CB1fSLq4wpkFG5kWsQ2r
```

Change the `mode` field in the `spec`:

```
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  ...
spec:
  ...
  mode: audit
  ...
```

To:

```
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  ...
spec:
  ...
  mode: enforce
  ...
```

Then save to apply the update:

```
Successfully edited policy pol:CB1fSLq4wpkFG5kWsQ2r
```

You should now see the following when issuing the `get` command:

```
spyctl get policies
UID                       NAME              STATUS      TYPE       CREATE_TIME
pol:CB1fSLq4wpkFG5kWsQ2r  mongo-policy      Enforcing   container  2024-01-16T15:00:43Z
```

## Disabling and Re-enabling a Policy

If you notice that a Policy is too noisy, or you want to temporarily disable it, edit the yaml and update the `enabled` field:

Use the `edit` command to edit the Policy's yaml.

```
spyctl edit RESOURCE NAME_OR_UID
```

For example:

```
spyctl edit policy pol:CB1fSLq4wpkFG5kWsQ2r
```

```
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  ...
spec:
  ...
  enabled: true
  ...
```

To:

```
apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  ...
spec:
  ...
  enabled: false
  ...
```

Then save to apply the update:

```
Successfully edited policy pol:CB1fSLq4wpkFG5kWsQ2r
```

To see that the Policy is indeed disabled, issue the command:

```
spyctl get policies
UID                       NAME              STATUS     TYPE       CREATE_TIME
pol:CB1fSLq4wpkFG5kWsQ2r  mongo-policy      Disabled   container  2024-01-16T15:00:43Z
```

To re-enable a Policy you just can simply remove the `enabled` field in the `spec` or change *false* to *true* and then `apply` the Policy file again.

To see that the action was successful, issue the `get` command again:

```
spyctl get policies
UID                       NAME              STATUS      TYPE       CREATE_TIME
pol:CB1fSLq4wpkFG5kWsQ2r  mongo-policy      Enforcing   container  2024-01-16T15:00:43Z
```

## Deleting a Policy

If you wish to completely remove a Policy from the Spyderbat Environment of the organization in your current Context you can use the `delete` command:

```
spyctl delete RESOURCE [OPTIONS] NAME_OR_ID
```

For example:

```
spyctl delete policy pol:CB1fSLq4wpkFG5kWsQ2r
Successfully deleted policy pol:CB1fSLq4wpkFG5kWsQ2r
```


# Notification Template Management using Spyctl

To learn more about what Notification Templates are, see [Notification Templates](/concepts/notifications/notification-templates)

## Prerequisites

If you have never used Spyctl start [here](/installation/spyctl) to learn how to install it, then follow the [Initial Configuration](/installation/spyctl) guide.

## What are Notification Templates?

Notification Templates define the format of notifications sent via different channels such as Email, Slack, Webhook, and PagerDuty. These templates help in customizing the notification messages. You can either specify a Notification Target or a Notification Template that map specific targets to templates.

**Available Notification Template Types:**

1. email - Create an email notification template.
2. pagerduty - Create a PagerDuty notification template.
3. slack - Create a Slack notification template.
4. webhook - Create a webhook notification template.

## Managing Notification Templates

### Create

To create a new Notification Template, use the create command:

Note: This will only create a local YAML file for you to edit. It makes no immediate changes to your Spyderbat environment.

```sh
 spyctl create notification-template [TYPE] -n NAME
```

For example:

```sh
 spyctl create notification-template slack -n slack-template > template.yaml
```

This will create a default Slack Notification Template and save it to a file called template.yaml.

```yaml
apiVersion: spyderbat/v1
kind: NotificationTemplate
metadata:
  name: Example
  type: slack
spec:
  text: ''
  blocks: []
```

To get the template in JSON format use `-o json` option:

```
 spyctl create notification-template slack -n slack-template -o json > template.yaml
```

Note: Learn How to populate the Template field values [here](/concepts/notifications/notification-templates)

### Apply

To make a Notification Template available for use, apply it using the apply command:

```
 spyctl apply -f FILENAME
```

For example:

```
 spyctl apply -f template.yaml
```

### Get or Download

You can use the get command to view or download your Notification Templates.

```sh
 spyctl get notification-templates
```

For example:

```sh
 spyctl get notification-templates
```

```
Getting notification-templates
Page 1/1
NAME                 UID                        TYPE       CREATED                     DESCRIPTION
test-email-tmpl      tmpl:avgUE                 email      2024-09-30T21:06:03 UTC     Operations teams.
```

To download a Notification Template as YAML or JSON, use the -o option:

```sh
 spyctl get notification-templates -o yaml slack-template
```

Using the > character, you can save the document to a file:

```sh
 spyctl get notification-templates -o yaml slack-template > template.yaml
```

### Edit

To modify an existing Notification Template, use the edit command:

```sh
spyctl edit notification-template NAME_OR_UID
```

For example:

```sh
 spyctl edit notification-template slack-template
```

This will open the template in your configured text editor for modification.

### Delete

To remove a Notification Template from the Spyderbat system, use the delete command:

```sh
spyctl delete notification-template <NAME_OR_UID>
```

For example:

```sh
spyctl delete notification-template slack-template
```


# Notification Target Management using Spyctl

To learn more about what Notification Targets are, see:

{% content-ref url="/pages/w6sePv65ceOqTJ6cNyQW" %}
[Notification Targets](/concepts/notifications/notification-targets)
{% endcontent-ref %}

## Prerequisites

If you have never used Spyctl start [here](/installation/spyctl) to learn how to install it, then follow the [Initial Configuration](/installation/spyctl) guide.

## Managing Notification Targets

### Create

To create a new Notification Target you can use the `create` command:

{% hint style="info" %}
Note: This will only create a local yaml file for you to edit. It makes no immediate changes to your Spyderbat environment.
{% endhint %}

```bash
spyctl create notification-target -n NAME -T TYPE
```

For example:

```
spyctl create notification-target -n OperationsTeam -T emails > target.yaml
```

This will create a default Notification Target and save it to a file called `target.yaml`

{% code title="target.yaml" %}

```yaml
apiVersion: spyderbat/v1
kind: NotificationTarget
metadata:
  name: OperationsTeam
spec:
  emails:
  - example@example.com
```

{% endcode %}

### Edit

When creating new Notification Targets you will need to edit the default document to point to the proper destination. With spyctl you can use the `edit` command to ensure you don't accidentally introduce syntax errors.

```bash
spyctl edit -f FILENAME
```

If you have already [applied ](#apply)the Notification Target you may edit the resource using the following:

```bash
spyctl edit [OPTIONS] notification-target NAME_OR_UID
```

For example:

```bash
spyctl edit -f target.yaml
```

This will bring up a prompt to select a text editor unless you have already done so previously. Then, using your text editor you may fill in your desired destination or destinations.

If you save without making any changes, nothing happens to the resource or file you're editing. If you save and there were syntax errors, Spyctl will save your draft to a temporary location and re-open it with comments detailing the errors. Finally, if your changes have no syntax errors, Spyctl will update the resource or file you're editing.

{% hint style="info" %}
Note: If you edit a Notification Target in a local file, but the Target has already been [applied](#apply). You will need to apply the file again for the updates to take effect.
{% endhint %}

### Apply

In order for a Notification Target to be usable by the Spyderbat Notifications System you must first apply it using the `apply` command.

```bash
spyctl apply -f FILENAME
```

For example:

```sh
spyctl apply -f target.yaml
```

If the operation is successful, your Notification Target will be ready for use.

### Delete

To remove a Notification Target from the Spyderbat Notifications System you can use the `delete` command.

```bash
spyctl delete [OPTIONS] notification-target NAME_OR_UID
```

For example:

```bash
spyctl delete notification-target OperationsTeam
```

### View or Download

You can use the `get` command to view or download your Notification Targets.

```sh
spyctl get [OPTIONS] notification-targets [NAME_OR_UID]
```

For example:

<pre class="language-bash"><code class="lang-bash"><strong>spyctl get notification-targets
</strong></code></pre>

{% code fullWidth="false" %}

```sh
$ spyctl get notification-targets
Getting notification-targets
NAME              ID                                  AGE    TYPE      DESTINATIONS
OperationsTeam    notif_tgt:XXXXXXXXXXXXXXXXXXXXXX    7d     emails               1
```

{% endcode %}

The default output is a tabular summary of your Notification Targets. To download the Notification Target as yaml or json you can use the `-o` option

<pre class="language-bash"><code class="lang-bash"><strong>spyctl get notification-targets -o yaml OperationsTeam
</strong></code></pre>

{% code fullWidth="false" %}

```
$ spyctl get notification-targets -o yaml OperationsTeam
apiVersion: spyderbat/v1
kind: NotificationTarget
metadata:
  name: OperationsTeam
spec:
  emails:
  - engineer1@example.com
  - engineer2@example.com
```

{% endcode %}

Using the `>` character you can save the document to a file.

<pre class="language-bash"><code class="lang-bash"><strong>spyctl get notification-targets -o yaml OperationsTeam > target.yaml
</strong></code></pre>


# Search


# All Operators

## **String**

All string operators are case insensitive except for `~~=`.

| Symbol | Description                                |
| ------ | ------------------------------------------ |
| `=`    | Equal to value                             |
| `!=`   | Not equal to value                         |
| `~=`   | Matches pattern with `*` and `?` wildcards |
| `~~=`  | Matches regular expression                 |

## **Boolean**

| Symbol | Description        |
| ------ | ------------------ |
| `=`    | Equal to value     |
| `!=`   | Not equal to value |

## **Number and Integer**

| Symbol | Description                    |
| ------ | ------------------------------ |
| `=`    | Equal to value                 |
| `!=`   | Not equal to value             |
| `>`    | Greater than value             |
| `>=`   | Greater than or equal to value |
| `<`    | Less than value                |
| `<=`   | Less than or equal to value    |

## **IP Address**

| Symbol | Description                          |
| ------ | ------------------------------------ |
| `=`    | Equal to value                       |
| `!=`   | Not equal to value                   |
| `<<`   | Contained in the cidr given by value |

## **List**

| Symbol | Description             |
| ------ | ----------------------- |
| `[*]`  | Any element of the list |
| `[0]`  | Element at index 0      |

## **Dictionary**

| Symbol       | Description               |
| ------------ | ------------------------- |
| `:keys[*]`   | Any key of the dictionary |
| `:values[*]` | Any element at any key    |
| `["foo"]`    | Element at the key "foo"  |

## **Null**

| Symbol   | Description                               |
| -------- | ----------------------------------------- |
| `is`     | Checks that a field does not have a value |
| `is not` | Checks that a field does have a value     |


# All Fields

## **Redflag**

### **Ancestors**

* *Type:* List of Strings
* *Description:* If the reference object has ancestor processes, this is a list of their names.
* *Field Name:* `ancestors`

### **Arguments**

* *Type:* List of Strings
* *Description:* If referencing a process, the arguments of the process that generated the red flag.
* *Field Name:* `args`

### **Authenticated User Name**

* *Type:* String
* *Description:* If referencing a process, the authenticated user name of the process that generated the red flag.
* *Field Name:* `auser`

### **Cluster Name**

* *Type:* String
* *Description:* If red flag is associated with a cluster, or a node of a cluster, this is the name of the cluster.
* *Field Name:* `cluster_name`

### **Description**

* *Type:* String
* *Description:* The reason the red flag was generated.
* *Field Name:* `description`

### **Effective User Name**

* *Type:* String
* *Description:* If referencing a process, the effective user name of the process that generated the red flag.
* *Field Name:* `euser`

### **ID**

* *Type:* String
* *Description:* The unique ID of the red flag.
* *Field Name:* `id`

### **Is Exception**

* *Type:* Boolean
* *Description:* Is the red flag marked as an exception? If so, the red flag was generated by expected activity.
* *Field Name:* `false_positive`

### **Machine ID**

* *Type:* String
* *Description:* The unique machine ID associated with the red flag. Generally begins with "mach:".
* *Field Name:* `muid`

### **Policy Name**

* *Type:* String
* *Description:* If the red flag is associated with a Guardian policy, this is the name of the policy.
* *Field Name:* `policy_name`

### **Policy UID**

* *Type:* String
* *Description:* If the red flag is associated with a Guardian policy, this is the unique ID of the policy.
* *Field Name:* `policy_uid`

### **Reference Object**

* *Type:* String
* *Description:* The unique ID of the object that the red flag is associated with.
* *Field Name:* `ref`

### **Schema**

* *Type:* String
* *Description:* The full schema string of the red flag.
* *Field Name:* `schema`

### **Severity**

* *Type:* String
* *Description:* The security level of the red flag. One of: info, low, medium, high, critical.
* *Field Name:* `severity`

### **Spydertraces**

* *Type:* List of Strings
* *Description:* The unique IDs of the spydertraces that this red flag is a part of.
* *Field Name:* `traces`

### **Uptime**

* *Type:* Number
* *Description:* The uptime of the object referenced by the redflag.
* *Field Name:* `uptime`

## **Opsflag**

### **Agent Type**

* *Type:* String
* *Description:* The type of agent that generated an opsflag. Used with agent-related opsflags.
* *Field Name:* `agent_type`

### **Ancestors**

* *Type:* List of Strings
* *Description:* If the reference object has ancestor processes, this is a list of their names.
* *Field Name:* `ancestors`

### **Arguments**

* *Type:* List of Strings
* *Description:* If referencing a process, the arguments of the process that generated the ops flag.
* *Field Name:* `args`

### **Authenticated User Name**

* *Type:* String
* *Description:* If referencing a process, the authenticated user name of the process that generated the ops flag.
* *Field Name:* `auser`

### **Cluster Name**

* *Type:* String
* *Description:* The name of the cluster associated with an opsflag.
* *Field Name:* `cluster_name`

### **Description**

* *Type:* String
* *Description:* The reason the ops flag was generated.
* *Field Name:* `description`

### **Effective User Name**

* *Type:* String
* *Description:* If referencing a process, the effective user name of the process that generated the ops flag.
* *Field Name:* `euser`

### **False positive**

* *Type:* Boolean
* *Description:* Is the opsflag a false positive?
* *Field Name:* `false_positive`

### **Hostname**

* *Type:* String
* *Description:* The hostname of the machine associated with an opsflag.
* *Field Name:* `hostname`

### **Is Ephemeral**

* *Type:* Boolean
* *Description:* Is the reference object ephemeral? Used with agent-related opsflags.
* *Field Name:* `ephemeral`

### **Machine ID**

* *Type:* String
* *Description:* The unique machine ID associated with the ops flag. Generally begins with 'mach:'.
* *Field Name:* `muid`

### **Reference Object**

* *Type:* String
* *Description:* The unique ID of the object that the ops flag is associated with.
* *Field Name:* `ref`

### **Schema**

* *Type:* String
* *Description:* The full schema string of the ops flag.
* *Field Name:* `schema`

### **Severity**

* *Type:* String
* *Description:* The alert level of the ops flag. One of: info, low, medium, high, critical.
* *Field Name:* `severity`

### **UID**

* *Type:* String
* *Description:* The unique ID of the ops flag.
* *Field Name:* `id`

### **Uptime**

* *Type:* Number
* *Description:* The uptime of the object referenced by the ops flag.
* *Field Name:* `uptime`

## **Spydertrace**

### **Interactive Users**

* *Type:* List of Strings
* *Description:* The list of interactive users associated with the spydertrace.
* *Field Name:* `interactive_users`

### **Is Interactive**

* *Type:* Boolean
* *Description:* Is the spydertrace interactive? Interactive spydertraces are associated with interactive user processes.
* *Field Name:* `interactive`

### **Is Overtaken**

* *Type:* Boolean
* *Description:* Has the spydertrace been overtaken by another spydertrace? It is best to set this to false because the overtaking trace contains all of the overtaken trace.
* *Field Name:* `overtaken`

### **Is Suppressed**

* *Type:* Boolean
* *Description:* Is the spydertrace suppressed? Suppressed spydertraces are associated with expected activity.
* *Field Name:* `suppressed`

### **Machine UID**

* *Type:* String
* *Description:* The unique machine ID associated with the spydertrace. Generally begins with "mach:".
* *Field Name:* `muid`

### **Name**

* *Type:* String
* *Description:* The name of the spydertrace.
* *Field Name:* `name`

### **Non-Interactive Users**

* *Type:* List of Strings
* *Description:* The list of non-interactive users associated with the spydertrace.
* *Field Name:* `non_interactive_users`

### **Root Process Name**

* *Type:* String
* *Description:* Name of the root process of the spydertrace.
* *Field Name:* `root_proc_name`

### **Schema**

* *Type:* String
* *Description:* The full schema string of the spydertrace.
* *Field Name:* `schema`

### **Score**

* *Type:* Integer
* *Description:* A score ranking the severity of the spydertrace.
* *Field Name:* `score`

### **Status**

* *Type:* String
* *Description:* Status of the spydertrace: closed or active.
* *Field Name:* `status`

### **Trigger**

* *Type:* String
* *Description:* The unique ID for the object that triggered the spydertrace's creation.
* *Field Name:* `trigger`

### **Trigger Short Name**

* *Type:* String
* *Description:* Short name for the object that triggered the spydertrace.
* *Field Name:* `trigger_short_name`

### **UID**

* *Type:* String
* *Description:* The unique ID of the spydertrace.
* *Field Name:* `id`

## **Container**

[Link to Related Objects](/reference/search/search-joins#container)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the container is a part of
* *Field Name:* `clustername`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the kubernetes cluster the container is a part of
* *Field Name:* `cluster_uid`

### **Container ID**

* *Type:* String
* *Description:* The long identifier of the container as reported by the container runtime
* *Field Name:* `container_id`

### **Container Name**

* *Type:* String
* *Description:* The name of the container as reported by the container runtime
* *Field Name:* `container_name`

### **Image**

* *Type:* String
* *Description:* The fully qualified name of the image used to create the container
* *Field Name:* `image`

### **Image ID**

* *Type:* String
* *Description:* The identifier of the image used to create the container
* *Field Name:* `image_id`

### **Machine UID**

* *Type:* String
* *Description:* The unique spyderbat machine ID the container is running on
* *Field Name:* `muid`

### **Pod Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the pod the container is a part of
* *Field Name:* `pod_labels`

### **Pod Name**

* *Type:* String
* *Description:* The name of the kubernetes pod the container is a part of
* *Field Name:* `pod_name`

### **Pod Namespace**

* *Type:* String
* *Description:* The namespace of the kubernetes pod the container is a part of
* *Field Name:* `pod_namespace`

### **Pod Namespace Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The labels for the namespace of the kubernetes pod the container is a part of
* *Field Name:* `pod_namespace_labels`

### **Pod UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the kubernetes pod the container is a part of
* *Field Name:* `pod_uid`

### **Root process UID**

* *Type:* String
* *Description:* The spyderbat ID of the root process running in the container
* *Field Name:* `root_puid`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the container model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the container
* *Field Name:* `id`

### **node\_uid**

* *Type:* String
* *Field Name:* `node_uid`

## **Cluster**

[Link to Related Objects](/reference/search/search-joins#cluster-1)

### **Name**

* *Type:* String
* *Description:* The name assigned to the cluster at spyderbat provisioning time
* *Field Name:* `name`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the cluster model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the cluster
* *Field Name:* `id`

## **Node**

[Link to Related Objects](/reference/search/search-joins#node-1)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the node belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the kubernetes cluster the node belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the node as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the node as reported in the metadata
* *Field Name:* `metadata.labels`

### **Machine UID**

* *Type:* String
* *Description:* The unique Spyderbat machine ID for the node
* *Field Name:* `muid`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the node as reported in the metadata
* *Field Name:* `metadata.name`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the node model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for this model
* *Field Name:* `id`

## **Deployment**

[Link to Related Objects](/reference/search/search-joins#deployment)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the deployment belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the deployment belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the deployment as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the deployment as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the deployment as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the deployment as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the deployment model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the deployment
* *Field Name:* `id`

## **Replicaset**

[Link to Related Objects](/reference/search/search-joins#replicaset)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the replicaset belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the replicaset belongs to
* *Field Name:* `cluster_uid`

### **Deployment name**

* *Type:* String
* *Description:* The name for the deployment the replicaset is owned by (if replicaset is owned by a deployment)
* *Field Name:* `deployment_name`

### **Deployment uid**

* *Type:* String
* *Description:* The Spyderbat unique id for the deployment the replicaset is owned by (if replicaset is owned by a deployment)
* *Field Name:* `deployment_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the replicaset as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the replicaset as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the replicaset as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the replicaset as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the replicaset model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the replicaset
* *Field Name:* `id`

## **Daemonset**

[Link to Related Objects](/reference/search/search-joins#daemonset)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the daemonset belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the daemonset belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the daemonset as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the daemonset as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the daemonset as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the daemonset as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the daemonset model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the daemonset
* *Field Name:* `id`

## **Job**

[Link to Related Objects](/reference/search/search-joins#job)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the job belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the job belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the job as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the job as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the job as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the job as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the job model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the job
* *Field Name:* `id`

## **Cronjob**

[Link to Related Objects](/reference/search/search-joins#cronjob)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the cronjob belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the cronjob belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the cronjob as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the cronjob as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the cronjob as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the cronjob as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the cronjob model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the cronjob
* *Field Name:* `id`

## **Statefulset**

[Link to Related Objects](/reference/search/search-joins#statefulset)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the statefulset belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the statefulset belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the statefulset as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the statefulset as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the statefulset as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the statefulset as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the statefulset model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the statefulset
* *Field Name:* `id`

## **Service**

[Link to Related Objects](/reference/search/search-joins#service)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the service belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the service belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the service as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the service as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the service as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the service as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the service model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the service
* *Field Name:* `id`

## **Pod**

[Link to Related Objects](/reference/search/search-joins#pod-1)

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the pod belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the pod belongs to
* *Field Name:* `cluster_uid`

### **Deployment UID**

* *Type:* String
* *Description:* The spyderbat unique id for the deployment the pod is associated with
* *Field Name:* `deployment_uid`

### **Deployment name**

* *Type:* String
* *Description:* The name of the deployment the pod is associated with
* *Field Name:* `deployment_name`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the pod as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the pod as reported in the metadata
* *Field Name:* `metadata.labels`

### **Machine UID**

* *Type:* String
* *Description:* The unique machine ID associated with this pod
* *Field Name:* `muid`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the pod as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the pod as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Node UID**

* *Type:* String
* *Description:* The spyderbat unique id for the node the pod is running on
* *Field Name:* `node_uid`

### **Owner Kind**

* *Type:* String
* *Description:* The kind of the resource that owns the pod
* *Field Name:* `owner_kind`

### **Owner Name**

* *Type:* String
* *Description:* The name of the resource that owns the pod
* *Field Name:* `owner_name`

### **Owner UID**

* *Type:* String
* *Description:* The Spyderbat unique uid of the resource that owns the pod
* *Field Name:* `owner_uid`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the pod model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the pod
* *Field Name:* `id`

## **Role**

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the role belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the role belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the role as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the role as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the role as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the role as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the role model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the role
* *Field Name:* `id`

## **Cluster Role**

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the role belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the role belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the role as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the role as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the role as reported in the metadata
* *Field Name:* `metadata.name`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the role model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the role
* *Field Name:* `id`

## **Service Account**

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the service account belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the service account belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the service account as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the service account as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the service account as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the service account as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the service account model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the service account
* *Field Name:* `id`

## **Role Binding**

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the rolebinding belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the rolebinding belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the rolebinding as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the rolebinding as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the rolebinding as reported in the metadata
* *Field Name:* `metadata.name`

### **Namespace**

* *Type:* String
* *Description:* The kubernetes namespace for the rolebinding as reported in the metadata
* *Field Name:* `metadata.namespace`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the rolebinding model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the rolebinding
* *Field Name:* `id`

## **Cluster Role Binding**

### **Cluster Name**

* *Type:* String
* *Description:* The name of the kubernetes cluster the clusterrolebinding belongs to
* *Field Name:* `cluster_name`

### **Cluster UID**

* *Type:* String
* *Description:* The unique Spyderbat id for the kubernetes cluster the clusterrolebinding belongs to
* *Field Name:* `cluster_uid`

### **Kubernetes uid**

* *Type:* String
* *Description:* The kubernetes unique id for the clusterrolebinding as reported in the metadata
* *Field Name:* `metadata.uid`

### **Labels**

* *Type:* Dictionary of Strings to Strings
* *Description:* The kubernetes labels for the clusterrolebinding as reported in the metadata
* *Field Name:* `metadata.labels`

### **Name**

* *Type:* String
* *Description:* The kubernetes name for the clusterrolebinding as reported in the metadata
* *Field Name:* `metadata.name`

### **Schema**

* *Type:* String
* *Description:* The Spyderbat schema for the clusterrolebinding model
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the clusterrolebinding
* *Field Name:* `id`

## **Listening Socket**

### **Duration**

* *Type:* Number
* *Description:* The duration of the model in seconds
* *Field Name:* `duration`

### **Local IP**

* *Type:* IP Address
* *Description:* The local IP address, or originating address of the connection
* *Field Name:* `local_ip`

### **Local port**

* *Type:* Integer
* *Description:* The local port of the connection
* *Field Name:* `local_port`

### **Machine UID**

* *Type:* String
* *Description:* The unique machine ID associated with this model or event
* *Field Name:* `muid`

### **Process UIDs**

* *Type:* List of Strings
* *Description:* The unique Spyderbat IDs for the associated processes to this socket
* *Field Name:* `puids`

### **Schema**

* *Type:* String
* *Description:* The full schema string of the listening socket
* *Field Name:* `schema`

### **Status**

* *Type:* String
* *Description:* Status of this model: closed or active
* *Field Name:* `status`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for the listening socket.
* *Field Name:* `id`

## **Connection**

[Link to Related Objects](/reference/search/search-joins#connection)

### **Bytes Received**

* *Type:* Integer
* *Description:* The number of bytes received on the local side of the connection.
* *Field Name:* `bytes_rx`

### **Bytes Sent**

* *Type:* Integer
* *Description:* The number of bytes sent on to the remote side of the connection.
* *Field Name:* `bytes_tx`

### **Cgroup**

* *Type:* String
* *Description:* The latest cgroup associated with the connection.
* *Field Name:* `cgroup`

### **Container UID**

* *Type:* String
* *Description:* The unique ID of the container associated with the connection.
* *Field Name:* `container_uid`

### **Destination**

* *Type:* List of Strings
* *Description:* The destinations of the connection (max 100 array). "ipv4|ipv6:remote\_ip:remote\_port".
* *Field Name:* `dsts`

### **Direction**

* *Type:* String
* *Description:* The direction of the connection: "inbound", "outbound", or "unknown".
* *Field Name:* `direction`

### **Duration**

* *Type:* Number
* *Description:* The duration of the connection model in seconds at time of last update.
* *Field Name:* `duration`

### **Family**

* *Type:* String
* *Description:* Family: IPV4 or IPV6.
* *Field Name:* `family`

### **Local IP**

* *Type:* IP Address
* *Description:* The local IP address, or originating address of the connection
* *Field Name:* `local_ip`

### **Local port**

* *Type:* Integer
* *Description:* The local port of the connection
* *Field Name:* `local_port`

### **Machine UID**

* *Type:* String
* *Description:* The unique ID of the machine associated with the connection.
* *Field Name:* `muid`

### **Payload**

* *Type:* String
* *Description:* A string representation of the payload of the connection. For example, the domain name of a DNS request response.
* *Field Name:* `payload`

### **Peer connection UID**

* *Type:* String
* *Description:* The unique ID of the peer remote connection if seen by Spyderbat.
* *Field Name:* `peer_cuid`

### **Peer machine UID**

* *Type:* String
* *Description:* The unique ID of the peer connection's machine if seen by Spyderbat.
* *Field Name:* `peer_muid`

### **Peer process UID**

* *Type:* String
* *Description:* The unique ID of the peer connection's process if seen by Spyderbat.
* *Field Name:* `peer_puid`

### **Process UID**

* *Type:* String
* *Description:* The unique ID of the latest process associated with the connection.
* *Field Name:* `puid`

### **Process UIDs**

* *Type:* List of Strings
* *Description:* The unique IDs of the process(es) associated with the connection.
* *Field Name:* `puids`

### **Process name**

* *Type:* String
* *Description:* The name of the process associated with the connection.
* *Field Name:* `proc_name`

### **Remote IP**

* *Type:* IP Address
* *Description:* The IP address on the remote side of the connection.
* *Field Name:* `remote_ip`

### **Remote hostname**

* *Type:* String
* *Description:* The hostname on the remote side of the connection.
* *Field Name:* `remote_hostname`

### **Remote port**

* *Type:* Integer
* *Description:* The port number on the remote side of the connection.
* *Field Name:* `remote_port`

### **Schema**

* *Type:* String
* *Description:* The full schema of the connection.
* *Field Name:* `schema`

### **Sources**

* *Type:* List of Strings
* *Description:* The objects that are the source of the connection (max 100 array).
* *Field Name:* `srcs`

### **Spydertraces**

* *Type:* List of Strings
* *Description:* The unique IDs of the spydertraces this connection is a part of.
* *Field Name:* `traces`

### **Status**

* *Type:* String
* *Description:* Status of the connection: closed or active.
* *Field Name:* `status`

### **UID**

* *Type:* String
* *Description:* The unique ID for this connection.
* *Field Name:* `id`

## **Machine**

[Link to Related Objects](/reference/search/search-joins#machine-3)

### **Boot Time**

* *Type:* Number
* *Description:* The time at which the machine was booted.
* *Field Name:* `boot_time`

### **CPU Architecture**

* *Type:* String
* *Description:* The architecture of the CPU that is installed in the machine.
* *Field Name:* `machine_processor`

### **CPU Model**

* *Type:* String
* *Description:* The model of the CPU that is installed in the machine.
* *Field Name:* `cpu_model`

### **Cloud Image ID**

* *Type:* String
* *Description:* If from a cloud provider, the image ID.
* *Field Name:* `cloud_image_id`

### **Cloud Instance ID**

* *Type:* String
* *Description:* If from a cloud provider, the instance ID of the virtual machine.
* *Field Name:* `cloud_instance_id`

### **Cloud Region ID**

* *Type:* String
* *Description:* If from a cloud provider, the region ID.
* *Field Name:* `cloud_region`

### **Cloud Tags**

* *Type:* Dictionary of Strings to Strings
* *Description:* If from a cloud provider, the tags associated with the machine.
* *Field Name:* `cloud_tags`

### **Cloud Type**

* *Type:* String
* *Description:* If from a cloud provider, the type of cloud provider.
* *Field Name:* `cloud_type`

### **Cluster Name**

* *Type:* String
* *Description:* The name of the cluster the machine is associated with.
* *Field Name:* `cluster_name`

### **Duration**

* *Type:* Number
* *Description:* The amount of time the machine has been running in seconds.
* *Field Name:* `duration`

### **Hostname**

* *Type:* String
* *Description:* The hostname of the machine.
* *Field Name:* `hostname`

### **Kernel Modules**

* *Type:* List of Strings
* *Description:* The list of kernel modules that are installed on the machine.
* *Field Name:* `kernel_mods`

### **OS Release**

* *Type:* String
* *Description:* The release of the operating system installed on the machine.
* *Field Name:* `os_release`

### **OS System**

* *Type:* String
* *Description:* The system of the operating system installed on the machine. Generally "linux".
* *Field Name:* `os_system`

### **OS Version**

* *Type:* String
* *Description:* The version of the operating system installed on the machine.
* *Field Name:* `os_version`

### **OS name**

* *Type:* String
* *Description:* The name of the operating system installed on the machine.
* *Field Name:* `os_name`

### **Private IP Address**

* *Type:* List of Strings
* *Description:* The private IP addresses associated with the machine.
* *Field Name:* `private_ip`

### **Public IP Address**

* *Type:* List of Strings
* *Description:* The public IP addresses associated with the machine.
* *Field Name:* `public_ip`

### **Schema**

* *Type:* String
* *Description:* The full schema of the machine.
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique ID for this machine.
* *Field Name:* `id`

## **Fingerprint**

### **status**

* *Type:* String
* *Field Name:* `status`

### **cgroup**

* *Type:* String
* *Field Name:* `cgroup`

### **service\_name**

* *Type:* String
* *Field Name:* `service_name`

### **image**

* *Type:* String
* *Field Name:* `image`

### **image\_id**

* *Type:* String
* *Field Name:* `image_id`

### **container\_name**

* *Type:* String
* *Field Name:* `container_name`

### **container\_id**

* *Type:* String
* *Field Name:* `container_id`

### **Machine UID**

* *Type:* String
* *Field Name:* `muid`

### **Root Process UID**

* *Type:* String
* *Field Name:* `root_puid`

### **Schema**

* *Type:* String
* *Field Name:* `schema`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for this model
* *Field Name:* `id`

## **Process**

[Link to Related Objects](/reference/search/search-joins#process)

### **src\_uid**

* *Type:* String
* *Field Name:* `src_uid`

### **Ancestors**

* *Type:* List of Strings
* *Description:* A list of the names of the ancestor processes
* *Field Name:* `ancestors`

### **Arguments**

* *Type:* List of Strings
* *Description:* The arguments specified when the process is started
* *Field Name:* `args`

### **Authenticated user**

* *Type:* String
* *Description:* The authenticated user name
* *Field Name:* `auser`

### **CGroup**

* *Type:* String
* *Description:* The Cgroup, if any, associated with the process
* *Field Name:* `cgroup`

### **Container**

* *Type:* String
* *Description:* The container ID
* *Field Name:* `container`

### **Container UID**

* *Type:* String
* *Description:* The spyderbat ID for the container model, if any
* *Field Name:* `container_uid`

### **Duration**

* *Type:* Number
* *Description:* The duration of the model in seconds
* *Field Name:* `duration`

### **Effective user**

* *Type:* String
* *Description:* The effective user who created the process
* *Field Name:* `euser`

### **Environment Variables**

* *Type:* Dictionary of Strings to Strings
* *Description:* A map with the name and value of all environment variables set at the time of process creation
* *Field Name:* `environ`

### **Executable**

* *Type:* String
* *Description:* The pathname of the executable associated with the process
* *Field Name:* `exe`

### **Interactive**

* *Type:* Boolean
* *Description:* Specifies if the process is associated with a terminal, and indicates if there is a human user who likely created the process
* *Field Name:* `interactive`

### **Machine UID**

* *Type:* String
* *Description:* The unique ID of the associated machine
* *Field Name:* `muid`

### **Name**

* *Type:* String
* *Description:* The name of the process
* *Field Name:* `name`

### **Organization UID**

* *Type:* String
* *Description:* The unique ID of the Spyderbat organization that owns this data
* *Field Name:* `org_uid`

### **PID**

* *Type:* Integer
* *Description:* The Unix process ID for this process
* *Field Name:* `pid`

### **Parent PID**

* *Type:* Integer
* *Description:* Unix process ID for the parent of this process
* *Field Name:* `ppid`

### **Parent process UID**

* *Type:* String
* *Description:* The unique Spyderbat ID of the parent process object
* *Field Name:* `ppuid`

### **Schema**

* *Type:* String
* *Description:* The string model\_process:...
* *Field Name:* `schema`

### **Session UID**

* *Type:* String
* *Description:* The Spyderbat UID for the associated session
* *Field Name:* `suid`

### **Status**

* *Type:* String
* *Description:* Status of this model: closed or active
* *Field Name:* `status`

### **Thread**

* *Type:* Boolean
* *Description:* Indicates that this process is a thread
* *Field Name:* `thread`

### **Traces**

* *Type:* List of Strings
* *Description:* An array of Spyderbat UID for traces associated with this process
* *Field Name:* `traces`

### **UID**

* *Type:* String
* *Description:* The unique Spyderbat ID for this model
* *Field Name:* `id`


# All Related Objects

## **Container**

[Link to Fields](/reference/search/search-fields#container)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

### **connections\[\*]**

* *New Schema:* [Connection](#connection)

### **machine**

* *New Schema:* [Machine](#machine-3)

### **node**

* *New Schema:* [Node](#node-1)

### **pod**

* *New Schema:* [Pod](#pod-1)

### **processes\[\*]**

* *New Schema:* [Process](#process)

### **root\_process**

* *New Schema:* [Process](#process)

## **Cluster**

[Link to Fields](/reference/search/search-fields#cluster)

### **containers\[\*]**

* *New Schema:* [Container](#container)

### **cronjobs\[\*]**

* *New Schema:* [Cronjob](#cronjob)

### **daemonsets\[\*]**

* *New Schema:* [Daemonset](#daemonset)

### **deployments\[\*]**

* *New Schema:* [Deployment](#deployment)

### **jobs\[\*]**

* *New Schema:* [Job](#job)

### **nodes\[\*]**

* *New Schema:* [Node](#node-1)

### **pods\[\*]**

* *New Schema:* [Pod](#pod-1)

### **replicasets\[\*]**

* *New Schema:* [Replicaset](#replicaset)

### **services\[\*]**

* *New Schema:* [Service](#service)

### **statefulsets\[\*]**

* *New Schema:* [Statefulset](#statefulset)

## **Node**

[Link to Fields](/reference/search/search-fields#node)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

### **pods\[\*]**

* *New Schema:* [Pod](#pod-1)

## **Deployment**

[Link to Fields](/reference/search/search-fields#deployment)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

### **pods\[\*]**

* *New Schema:* [Pod](#pod-1)

## **Replicaset**

[Link to Fields](/reference/search/search-fields#replicaset)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

### **pods\[\*]**

* *New Schema:* [Pod](#pod-1)

## **Daemonset**

[Link to Fields](/reference/search/search-fields#daemonset)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

### **pods\[\*]**

* *New Schema:* [Pod](#pod-1)

## **Job**

[Link to Fields](/reference/search/search-fields#job)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

### **pods\[\*]**

* *New Schema:* [Pod](#pod-1)

## **Cronjob**

[Link to Fields](/reference/search/search-fields#cronjob)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

## **Statefulset**

[Link to Fields](/reference/search/search-fields#statefulset)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

### **pods\[\*]**

* *New Schema:* [Pod](#pod-1)

## **Service**

[Link to Fields](/reference/search/search-fields#service)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

## **Pod**

[Link to Fields](/reference/search/search-fields#pod)

### **cluster**

* *New Schema:* [Cluster](#cluster-1)

### **connections\[\*]**

* *New Schema:* [Connection](#connection)

### **containers\[\*]**

* *New Schema:* [Container](#container)

### **daemonset**

* *New Schema:* [Daemonset](#daemonset)

### **deployment**

* *New Schema:* [Deployment](#deployment)

### **job**

* *New Schema:* [Job](#job)

### **machine**

* *New Schema:* [Machine](#machine-3)

### **node**

* *New Schema:* [Node](#node-1)

### **replicaset**

* *New Schema:* [Replicaset](#replicaset)

### **statefulset**

* *New Schema:* [Statefulset](#statefulset)

## **Connection**

[Link to Fields](/reference/search/search-fields#connection)

### **container**

* *New Schema:* [Container](#container)

### **machine**

* *New Schema:* [Machine](#machine-3)

### **peer\_connection**

* *New Schema:* [Connection](#connection)

### **peer\_machine**

* *New Schema:* [Machine](#machine-3)

### **peer\_process**

* *New Schema:* [Process](#process)

### **pod**

* *New Schema:* [Pod](#pod-1)

### **processes\[\*]**

* *New Schema:* [Process](#process)

## **Machine**

[Link to Fields](/reference/search/search-fields#machine)

### **connections\[\*]**

* *New Schema:* [Connection](#connection)

### **connections\_as\_peer\[\*]**

* *New Schema:* [Connection](#connection)

### **processes\[\*]**

* *New Schema:* [Process](#process)

## **Process**

[Link to Fields](/reference/search/search-fields#process)

### **children\[\*]**

* *New Schema:* [Process](#process)

### **connections\[\*]**

* *New Schema:* [Connection](#connection)

### **connections\_as\_peer\[\*]**

* *New Schema:* [Connection](#connection)

### **container**

* *New Schema:* [Container](#container)

### **container\_as\_root**

* *New Schema:* [Container](#container)

### **machine**

* *New Schema:* [Machine](#machine-3)

### **parent**

* *New Schema:* [Process](#process)


