Notifications

This section documents the full capabilities of Notification Templates, and which fields exist to manipulate the template/behavior.

The Conditions and Triggering Notifications section will briefly explain how the condition field interacts with the Spyderbat data model in order to trigger notifications.

The Dereferencing Values section describes how you can inject values from the json record that triggered the notification into the notification itself.

The Internal Functions section details the various functions you can use to add additional context to your notifications or manipulate existing values into a more desirable format.

Note: The following are advanced concepts. An understanding of them is not required to get started with notifications. Follow the how-to guides to quickly setup commonly-used notifications.

Data Model Primer

Notifications in Spyderbat are driven by the data model. The Spyderbat Nano Agent generates raw telemetry and sends it to the Spyderbat Analytics Engine. The Analytics Engine processes the raw data and builds the behavior web that is viewable in the Console. Additionally, the Analytics Engine analyzes data in the behavior web for security detections, operations issues, policy violations, and more.

The data emitted by the Analytics Engine comes in two flavors: models and events. Notifications are generated by evaluating these two types of records. Models are (potentially) long-lived objects that have a start, middle, and end in their lifecycle. Events represent detections or occurrences that happen at a single point in time.

Take processes as an example. Spyderbat receives process telemetry and builds models to track the state of the processes themselves. What gets emitted from Spyderbat looks like this:

Process Model

{
  "schema": "model_process::1.2.0",
  "id": "proc:_ZO7yNX2S54:ZWn4uw:753728",
  "version": 1701443785,
  "description": "bash [753728, normal] closed from 7dbab6f7-77de-494a-9490-564bc7174611",
  "cgroup": "systemd:/user.slice/user-1000.slice/session-966.scope",
  "time": 1701443785.3360424,
  "create_time": 1701443771.7009835,
  "valid_from": 1701443771.7009835,
  "muid": "mach:_ZO7yNX2S54",
  "pid": 753728,
  "ppid": 753726,
  "ppuid": "proc:_ZO7yNX2S54:ZWn4uw:753726",
  "tpuid": "proc:_ZO7yNX2S54:ZWn4uw:753726",
  "sid": "966",
  "args": [
    "/usr/bin/bash"
  ],
  "cwd": "/home/ubuntu",
  "thread": false,
  "type": "normal",
  "interactive": true,
  "environ": {},
  "duration": 9.437932014465332,
  "name": "bash",
  "title": "/usr/bin/bash",
  "auid": 1000,
  "euid": 0,
  "egid": 0,
  "container": null,
  "auser": "ubuntu",
  "euser": "root",
  "egrp": "root",
  "status": "closed",
  "data_is_complete": true,
  "ancestors": [
    "sudo",
    "bash",
    "sshd",
    "sshd",
    "systemd"
  ],
  "is_causer": false,
  "is_causee": false,
  "prev_time": 1701443781.1389155,
  "expire_at": 1701446399.999999,
  "exit": 0,
  "exe": "/usr/bin/bash",
  "valid_to": 1701443781.1389155,
  "traces": [
    "trace:_ZO7yNX2S54:AAYLdEBuJOo:753573:remote_access"
  ],
  "red_flag_count": 0,
  "red_flags": ["flag:629gia"],
  "ops_flag_count": 0,
  "red_flags": [],
  "schemaType": "model_process",
  "schemaMajorVersion": 1,
  "record_type": "model",
  "versionedId": "proc:_ZO7yNX2S54:ZWn4uw:753728:v1701443785"
}

The model above is for an interactive bash shell process running on a machine with the Spyderbat Nano Agent installed. It contains all the information required to add it into the behavior web. It also happens that this process is running with an effective user (euser) "root". That is a privileged account and we have a security detection when we see an interactive shell running as root.

Conditions and Triggering Notifications

Dereferencing Values

In Spyderbat's Notification Templates, you can dynamically include specific values from the JSON objects you are monitoring in the Template fields using dereferencing syntax. The syntax for dereferencing is as follows:

• For direct field access: {{ field_name }}

• For subfield access within a dictionary: {{ parent_field.sub_field }}

Let's consider an example JSON object:

Suppose you have a Saved Query for a Red Flag i.e a security detection on the root bash process above. Events generally have a ref field that points to the id of the model they're related to.

{
  "id": "event_alert:_ZO7yNX2S54:ZWn4uw:753728",
  "schema": "event_redflag:root_shell:1.1.0",
  "description": "ubuntu as root ran unusual interactive shell '/usr/bin/bash'",
  "ref": "proc:_ZO7yNX2S54:ZWn4uw:753728",
  "short_name": "root_shell",
  "class": [
    "redflag",
    "proc",
    "root_shell",
    "critical_severity"
  ],
  "flag_class": "redflag/proc/root_shell/critical_severity",
  "severity": "critical",
  "time": 1701443781.1389155,
  "routing": "customer",
  "version": 2,
  "muid": "mach:_ZO7yNX2S54",
  "name": "bash",
  "auid": 1000,
  "args": [
    "/usr/bin/bash"
  ],
  "auser": "ubuntu",
  "euser": "root",
  "ancestors": [
    "sudo",
    "bash",
    "sshd",
    "sshd",
    "systemd"
  ],
  "mitre_mapping": [
    {
      "sub-technique": "T1059.004",
      "sub-technique_name": "Unix Shell",
      "url": "https://attack.mitre.org/techniques/T1059/004",
      "created": "2020-03-09T14:15:05.330Z",
      "modified": "2021-07-26T22:34:43.261Z",
      "stix": "attack-pattern--a9d4b653-6915-42af-98b2-5758c4ceee56",
      "technique": "T1059",
      "technique_name": "Command and Scripting Interpreter",
      "tactic": "TA0002",
      "tactic_name": "Execution",
      "platform": "Linux"
    }
  ],
  "impact": "A shell owned by root has a dangerous level of permissions.",
  "ppuid": "proc:_ZO7yNX2S54:ZWn4uw:753726",
  "false_positive": false,
  "traces": [
    "trace:_ZO7yNX2S54:AAYLdEBuJOo:753573:remote_access"
  ],
  "traces_suppressed": false,
  "schemaType": "event_redflag",
}

Examples of Dereferencing Values from the Object Above:

Example with Email Template:

apiVersion: spyderbat/v1
kind: NotificationTemplate
metadata:
  name: email-template
  type: email
spec:
  subject: "Spyderbat Alert: {{ severity }} Severity Detected on {{ name }}"
  body_html: |
    <html>
      <body>
        <h4>Spyderbat Alert</h4>
        <p><strong>Severity:</strong> {{ severity }}</p>
        <p><strong>Description:</strong> {{ description }}</p>
        <p><strong>Process:</strong> {{ name }} (Executed by {{ auser }}, Effective user: {{ euser }})</p>
        <p><strong>Command:</strong> {{ args | join(" ") }}</p>
        <p><strong>Detection Time:</strong> {{ time }}</p>
        <p><strong>MITRE Technique:</strong> <a href="{{ mitre_mapping[0].url }}">{{ mitre_mapping[0].technique_name }}</a></p>
      </body>
    </html>
  body_text: |
    Spyderbat Alert
    Severity: {{ severity }}
    Description: {{ description }}
    Process: {{ name }} (Executed by {{ auser }}, Effective user: {{ euser }})
    Command: {{ args | join(" ") }}
    Detection Time: {{ time }}

These examples demonstrate how you can leverage dereferencing to dynamically include specific values from the JSON object in your notifications for more context on alert. Feel free to adjust the examples based on your specific use cases and requirements.

Internal Functions

In Spyderbat's Notification Templates, you can enhance the template alert by using internal functions as well in each Template type. The syntax for using functions is as follows:

• {{ __FUNCTION_NAME__ [| ARG1, ARG2, ..., ARGN] }}

Arguments are optional, depending on the function used. The return value of the function will replace the {{ __FUNCTION_NAME__ }} placeholder, or an error message will be displayed if something goes wrong.

Example JSON object used in function examples:

This metrics record is used to monitor the resource utilization of the Spyderbat Nano Agent.

{
  "schema": "event_metric:agent:1.0.0",
  "id": "event_metrics:07Ax6uRpB606065sYozQ:ZWXXXX",
  "ref": "agent:07Ax6uRpB606065sXXXX",
  "version": 1,
  "muid": "mach:5sZN4f2mXXX",
  "time": 1701460978.1299076,
  "cpu_cores": 2,
  "total_mem_B": 8173600768,
  "hostname": "example_machine",
  "cluster_name": null,
  "bandwidth_1min_Bps": 1616,
  "cpu_1min_P": {
    "agent": 0.0417,
    "authUID": 0.0002,
    "bashbatUID": 0.0002,
    "grimreaperUID": 0.0034,
    "procmonUID": 0.0003,
    "scentlessUID": 0.0355,
    "snapshotUID": 0
  },
  "mem_1min_B": {
    "agent": 355326000,
    "authUID": 29440000,
    "bashbatUID": 37820000,
    "grimreaperUID": 42692000,
    "procmonUID": 38005000,
    "scentlessUID": 71352000,
    "snapshotUID": 45604000
  },
  "mem_1min_P": {
    "agent": 0.04347239485822658,
    "authUID": 0.003601839732038158,
    "bashbatUID": 0.004627091666633258,
    "grimreaperUID": 0.005223156991853704,
    "procmonUID": 0.004649725510058091,
    "scentlessUID": 0.008729567546208785,
    "snapshotUID": 0.005579425921870522
  },
  "missed_messages": {
    "scentless": {
      "pcap_dropped": 0,
      "data_drop_delayq": 0,
      "data_drop_dns": 0,
      "missed_events": 0
    }
  }
}

Functions:

---

{{ __cluster__ }}

• Arguments: This function takes 0 arguments

• Description: Returns the name of the cluster the object is associated with or "No Cluster."

Pagerduty Template spec Example:

spec:
  custom_details: 
    "cluster": "{{ __cluster__ }}"

---

This would result in a list displayed in the notification:

• Cluster: No Cluster

This is because in the metrics object above, the cluster_name field is null.

---

{{ __hr_time__ }}

• Arguments: This function takes 0 arguments

• Description: Returns a human-readable version of the time field found in the object.

PagerDuty Template Spec Example:

spec:
  custom_details: 
    "time": "{{ __hr_time__ }}"

This would result in a list displayed in the notification:

• Time: 2023-12-01 20:02:58UTC

This converts the epoch time 1701460978.1299076 in the time field in the record above to something human readable.

---

{{ __linkback__ }}

• Arguments: This function takes 0 arguments

• Description: Returns a relevant URL linking back to the Spyderbat Console for the object being evaluated.

PagerDuty Template Spec Example:

spec:
  custom_details: 
    "linkback": "{{ __linkback__ }}"

This would result in a linkback URL being generated, pointing to the Agent Health page for the agent referenced above "ref": "agent:07Ax6uRpB606065sXXXX". It would display as a link "View in Spyderbat" at the bottom of your notification.

---

{{ __origin__ }}

• Arguments: This function takes 0 arguments

• Description: Returns a string explaining why the notification was generated.

PagerDuty Template Spec Example:

spec:
  custom_details: 
    "origin": "{{ __origin__ }}"

This would result in a message like:

Notification Origin: This notification was generated because an event_metric record matched the condition specified in notification config "Agent CPU Usage - notif:6voXLIYfRPmTky-XVAaXXX".

---

{{ __percent__ | number }}

• Arguments: This function takes exactly 1 argument

  • number:

    • type: String or Number

    • description: If a String is supplied, the string must be a field in the object with a numerical value.

• Description: Multiplies an input number by 100, caps the precision at 2 decimal places, and appends a percent (%) symbol.

PagerDuty Template Spec Example:

spec:
  custom_details: 
    "CPU Usage": "{{ __percent__ | cpu_1min_P.agent }}"

This would result in a list displayed in the notification:

• CPU Usage: 4.17%

Here’s an all list of the field names along with their actual function:

• __hr_time__ – Human-readable timestamp of the event.

• __time_int__ – Timestamp in integer format.

• __linkback__ – URL linking back to the event in Spyderbat.

• __origin__ – The origin or source of the event.

• __cluster__ – The cluster where the event occurred.

• __source__ – The source component or entity that generated the event.

• __hostname__ – The hostname where the event took place.

• __percent__ – A percentage value associated with the event.

• __pd_severity__ – Severity level formatted specifically for PagerDuty.

• __query_name__ – The name of the saved query that triggered the event.

Conclusion: You can use a mix of both Static values from the Object by dereferencing and using Spyderbat's Internal function to enhance templates for direct context on Notification Alert.