Spyderbat
Book a DemoStart FreeContact Us
  • Spyderbat Product Docs
  • Getting Started
    • Create an Organization
    • Install the Nano Agent
    • Three Things to Try with Spyderbat Community Edition
    • Manage Users and Roles
  • Installation
    • Spyderbat Nano Agent
      • Kubernetes
        • Pre Deployment Environment Data Collection Script
      • Linux Standalone
      • AWS Unattended Install
        • Secure your Registration Code with AWS Secrets Manager
      • Create a Golden Image with the Nano Agent Pre-Installed
    • Spyderbat AWS Agent
      • AWS Linux VM
      • Kubernetes
      • Configuration Guide - AWS Linux VM
      • Configuration Guide - Kubernetes
    • Install Spyctl CLI
      • Initial Configuration
    • Install Spydertop CLI
    • Install the Spyderbat Event Forwarder
      • Helm Chart
      • Traditional Installer
  • Concepts
    • Guardian & Interceptor
      • Ruleset Policies
      • Workload Policies
    • Flashback (Go Back In Time)
      • Investigations
    • Search
      • Saved Searches
    • Summarize
      • Spydertrace Summarize
    • Dashboards
      • Dashboard Categories
    • Reports
    • Notifications
      • Notification Targets
      • Notification Templates
    • Actions
    • Integrations
      • AWS Integration
      • Spyderbat Event Forwarder
    • Suppression & Tuning
    • Scout (Detections)
      • Custom Flags
  • Tutorials
    • Flashback
      • How to Use the Investigations Feature in Spyderbat
    • Guardian
      • How to Lock Down Your Workloads With Guardian Policies Using Spyctl
      • How to Put Guardrails Around Your K8s Clusters Using Spyctl
    • Integrations
      • How to Configure Event Forwarder Webhook for Panther
      • How to Set Up Spyderbat to Ingest Falco Alerts
      • How to Create and Use a Spyderbat API Key
    • Notifications
      • How to Set Up Notifications Using Spyctl
      • How to Set up Agent-Health Notifications Using Spyctl
    • Dashboards
    • Miscellaneous
      • How to Set Up Spyderbat to Monitor Systems From vulnhub.com
    • Scout (Detections)
      • How to Set Up Custom Flags Using Spyctl CLI
  • Reference
    • Policies
      • Response Actions
    • Rulesets
    • Selectors
    • Notifications
    • Spyctl CLI
      • Spyctl Commands
      • Guardian Policy Management using Spyctl
      • Notification Template Management using Spyctl
      • Notification Target Management using Spyctl
    • Search
      • All Operators
      • All Fields
      • All Related Objects
  • Quick Links
    • Contact Us
    • Try Spyderbat for Free
    • Book a Demo
Powered by GitBook

© SPYDERBAT, Inc., All Rights Reserved

On this page
  • Spyderbat Event Forwarder Overview
  • Filtering Data Using Hard-Coded Expressions
  • Event Forwarder Validation

Was this helpful?

Export as PDF
  1. Concepts
  2. Integrations

Spyderbat Event Forwarder

Export Spyderbat findings (runtime security flags, Spydertraces) for ingestion into third-party SIEM tools like Splunk, Panther, or Sumo Logic.

Last updated 1 year ago

Was this helpful?

Spyderbat Event Forwarder Overview

The event forwarder is a custom utility that consumes Spyderbat events from the API and emits files containing one of the two types of data, security flags or spydertraces. The event forwarder can be integrated with the third-party systems, such as SIEMs and other alert aggregation points, to forward those files via syslog to be consumed outside of Spyderbat platform to further enhance the security content and improve overall observability and security awareness.

For guides detailing the Spyderbat Event Forwarder installation process see .

How Does the Spyderbat Event Forwarder Work?

Let’s take a closer look at how the event forwarder collects and transports the data.

Spyderbat analytics system has a specific data store just for redflag events and spydertraces. When a redflag is created on any machine or a spydertrace is kicked off, they get sent to this store. After an agent sees an event, which would create a redflag or trigger a spydertrace, it is stored in the store.

The Spyderbat event forwarder polls the store using Spyderbat API to access these stored events, using a time window of start time and end time in epoch format. It also queries the list of known hosts to augment the events with the details of the known machines.

When the event forwarder is started for the first time, it will read back one hour in time to get the last hour of events, but when it is restarted it will read its logs to figure out the last event it emitted and will use that as its start time. It will read all files which match the file name spyderbat_events*.log in the directory specified by the log_path configuration setting, reading in all the events and keeping the latest time, to use as its starting time.

Once the event forwarder has started, it will either start with reading the last hour of data or start from the last event discovered in the logs (the files named spyderbat_event*.logs). It will set the end time to the current time, so that the query encapsulates a range to ensure that no events are missed.

Once the data comes back from the query, each event is examined to make sure it isn't duplicated and it is augmented to include the details of the associated machines, and the latest event time seen is captured to use as the next start time for the next query.

The event forwarder will then sleep for 30 seconds and then repeats the loop using the latest event time as the start time, but because there is a check to always expand the time window to encapsulate a full 5 minute window, it will always look 4 minutes and 30 seconds further into the past beyond the last event, to ensure that no records have been missed, and will use its LRU (Last Recently Used value) to de-duplicate the events it is capturing.

In the case when there are no events, the start time for the query will not be moved forward until either more events appear, or it reaches its maximum of the 4-hour boundary. This ensures that in the case of an outage on the backend these events will not be missed until they exceed 4 hours of age.

Filtering Data Using Hard-Coded Expressions

It is possible to configure the event forwarder to only log the events that meet certain criteria. This can be achieved through optional filtering using an expression syntax. The expression must evaluate to a bool; If it is true, the event will be logged.

The expression syntax is documented here:

If the expression fails to compile, the event forwarder will exit with an error at startup. If the expression fails to evaluate, the event will be logged and the forwarder will continue.

The most common reason for an expression to fail to evaluate is that the event does not contain the field(s) referenced in the expression. To avoid this problem, check that the fields you are referencing are not nil, or use the short-circuit "??" operator. Schema is guaranteed to be present.

Here is an example:

expr: |
      (
       schema startsWith "model_spydertrace:"
       and
       (score ?? 0) > 1000
      )
      or
      (
       not
       (
        schema startsWith "model_spydertrace:"
        or
        schema startsWith "event_redflag:bogons:"
        or
        (severity ?? "") in ["info", "low", "medium"]
       )
      )

In this example, the expression will log all events with a schema starting with "model_spydertrace:" and a score greater than 1000. It will log everything else except events with a schema starting with "event_redflag:bogons:" or a severity of "low" or "medium".

Event Forwarder Validation

To ensure that the event forwarder has been deployed and configured correctly and is working as expected, the following conditions must be met and observed:

  1. Event forwarder should emit logs to standard out that looks like this: "5 new records, most recent 23s ago". Here is the code reference in the forwarder:\

    log.Printf("%d new records, most recent %v ago", newRecords, et.Sub(lastTime.Time()).Round(time.Second))\

  2. Event forwarder should be writing to the specified output in the config file, either to the specified syslog end point or the specified log_path directory with the filename spyderbat_events*.log. You can run 'tail -f spyderbat_events*.log' in that directory to watch the logs being written in real time, you should see events appear within a 10-minute window.

this section
https://expr.medv.io/docs/Language-Definition