Search
Detailed overview of the Search tab in the console and the Spyderbat search language it uses.
The Spyderbat Search Language is a tool designed to easily query and extract actionable insights from your data within the Spyderbat console. This guide is designed to introduce the language and to help you understand the search functionalities and their practical applications for your organization.
The Search section of the Spyderbat UI is located second from the top of the left hand navigation panel. When entering the page, you will be presented with a search bar, some example searches, and a list of recent searches if you have run any previously.
To begin searching, click on the search bar. You’ll need to pick a schema, then enter a query expression in the Spyderbat search language, and finally select a time range. All of these steps will be elaborated on below in more detail. The schema and expression can also be filled by clicking on an example or recent search from below the bar. From there, you can click on the search button to begin loading results, or see a syntax error if the query was invalid.
Schemas
In Spyderbat, a schema represents a type of data collected by Spyderbat. For example, the schemas shown in the search UI include Process and Connection, which represent data associated to processes and network connections, respectively.
There are a number of schemas available to search on, each with their own fields that can be queried, which can be found categorized in the schema selector. These schemas are not part of the text of your query, but are selected on the search page. Additionally, a full list of them can be found in the Search Reference.
The Query Builder
After selecting a schema, you can either begin to type an expression, or open the Query Builder. The Query Builder is a useful tool for crafting expressions that contains all the information about the schemas, instead of requiring you to check the reference and documentation.
It contains the full set of fields for each schema, descriptions of the fields, and matches them to comparisons for you, allowing you to easily create effective and correct searches.
The Query Builder will open with the same schema as you had selected in the search page, but you can also select a different schema using the dropdown next to the query preview. Additionally, clicking on the query preview will show a list of recent queries constructed using the Query Builder. Clicking on one will populate the Query Builder and allow you to modify it or send it to the search page.
To modify a query in the Query Builder, use the selector boxes to select a field and what to compare it with. The selectors also contain short descriptions of the fields and operators.
Expressions
After selecting the schema, you’ll need to specify the expression for your search. The expression tells the query engine exactly what criteria to use for filtering data. Every schema has its own set of fields that can be used in the expression, and each field has a type such as String or Number that determines the comparisons available. In addition to these simple types, the List and Map types are also available as composite types, which are discussed below.
Comparisons
The simplest type of expression is a comparison of a field and a value. For example, given that the Process schema has the Executable field, we can use its shortened name exe in an expression to find all processes with the executable "/usr/bin/bash":
This expression uses the = operator, which simply checks for equality with the value. Each field type has a set of available operators in addition to =, allowing for more advanced searches. Additionally, inside an expression, fields must always use their shortened name, and they can only be compared with constant values in the expression itself, not the values of other fields.
A full list of the available comparison operators is available in the Search Reference.
Pattern Matching
String fields have two unique operators: ~= and ~~=, which are Unix glob style pattern matching and regular expression matching, respectively. Regular expressions can be researched elsewhere, but glob pattern matching in this context refers to a string with the characters * and ? used as wildcards. They can both be escaped in a pattern to match their literal characters instead of wildcards using the standard escape character \.
As wildcards, * can match any number of characters, while ? matches exactly one unknown character. For example, we can modify the previous example to check for any bash executable, instead of a specific path:
CIDR Matching
The IP address type also has a CIDR matching operator: <<. CIDR blocks can be researched elsewhere, but the operator is essentialy a way to compare against a range of IP addresses instead of individual ones. For example, we can check for connections from any IP address between 192.168.1.0 and 192.168.1.255:
Logic
Multiple field comparisons can be combined together using the basic logical operations and, or, and not. They can use any capitalization and may be combined with parentheses to specify precedence. For example:
This search uses parentheses to guarantee that the Duration field must always be greater than 60 seconds, regardless of the other conditions.
Field Types
In addition to the available operators, certain types of fields have special rules. For all types, the value used with any comparisons must be of the same type, although Number and Integer are effectively the same. In addition, all String values must be in either single or double quotes, and may escape any ending quotes inside the string using the standard escape character \.
There are two outlier types that fields can have: List and Map. These types both have elements of another type, most commonly String. A List represents an ordered collections of elements, while a Map represents key and element pairs. Interacting with these data types is relatively straightforward and can be combined with other comparisons for more complex queries. To use a List, you can either access elements by index – for example, you would use [0] to access the first element in the list – or search for a value occurring anywhere using the [*] syntax. Elements in a Map can be queried by referencing the key in brackets or with the :keys[*] and :values[*] syntax to search for a key or value anywhere in the map, respectively.
For example, this search finds Process objects with a first argument of “-i”:
Process objects where any argument is “-i”:
Kubernetes Pod objects serving the PostgreSQL database in the "production" namespace:
A full list of fields and their types is available in the Search Reference.
Filtering with related objects
An advanced feature of the Spyderbat search language is related object queries - the ability to reference other objects of different schemas in addition to the main object and query fields on those objects. Every schema has a set of related objects alongside its fields, which point to specific other objects of either the same or a different schema.
After a related object, any field or even related object on the related object's new schema can be chained together using the normal query syntax. This is useful when you want to filter based on information that isn’t contained in the original schema.
The capabilities are best illustrated with an example, such as the machine reference in the Process object, which points to a related object with the Machine schema - in this case, it would be the machine that the process is running on. Using the Cloud Region field in the Machine schema, we can find all bash processes in the "us-east-1" cloud region, even though we do not gather any region data in the Process schema:
In the example above, each process has exactly one associated machine. Some schemas have multiple other objects that can be associated, however, such as the children of a process. In that case, the reference must always be followed by [*], similar to how any element of a List can be used in a comparison. For example, the mentioned children[*] reference in the Process schema can be used to find all bash processes that ran "sudo" in a child process:
Related objects of this type can only be queried with the [*] syntax, and attempting to use an index or a string key will cause a syntax error.
Time Range
After selecting a schema and creating an expression, the last part necessary for a search is to select the time range for the query. When doing so, this will return all data that existed at any point during that time range. For example, a search with the Process schema in a time range of the previous 15 minutes will return all matching processes that were running in the past 15 minutes, but could have been started five minutes or a day ago.
The Spyderbat UI includes a time picker with preset relative times and a custom duration picker. Relative times become constant for a search once it has been run, as opposed to the search continuing to update as time goes by. For a search to continue reporting results, save it to a Dashboard card.
Last updated
