Comment on page
Basics Tutorial

This tutorial will teach you how to configure Spyctl. It will also provide basic instructions for viewing your Spyderbat Resources and baselining your services & containers.
Prerequisites
If you haven’t already done so, follow the instructions for installing Spyctl: Installation
In order to properly utilize Spyctl you must:
Have a Spyderbat account
Have installed at least one Spyderbat Nano Agent installed on a machine of your choosing
Have generated a key to access the Spyderbat API​
Initial Configuration
In this section you will learn how to configure Spyctl to enable data retrieval from across your entire organization. To do so, you must first create an APISecret and then use that APISecret to set a Context. An APISecret encapsulates your Spyderbat API credentials; the Context specifies where Spyctl should look for data when interacting with the Spyderbat API (e.g., organization, cluster, machine, service, or container image).
Create an APISecret
An APISecret encapsulates your Spyderbat API credentials. You must create at least one APISecret in order for Spyctl to access your data via theSpyderbat API.
To create an APISecret, use an api key generated from the Spyderbat Console:
$ spyctl config set-apisecret -k <apikey> -u "https://api.spyderbat.com" NAME
For example:
$ spyctl config set-apisecret -k ZXlKaGJHY2lPaUpJVXpJMU5pSXNJbXRwWkNJNkluTm\
lJaXdpZEhsd0lqb2lTbGRVSW4wLmV5SmxlSEFpT2pFM01EUTVPVGM1TWpBc0ltbGhkQ0k2TVRZM\
016UTJNVGt4T1N3aWFYTnpJam9pYTJGdVoyRnliMjlpWVhRdWJtVjBJaXdpYzNWaUlqb2ljSGhX\
YjBwMlVFeElXakJIY1VJd2RXMTNTMEVpZlEuZGpxWkRCOTNuUnB4RUF0UU4yQ0ZrOU5zblQ5Z2Q\
tN0tYT081TEZBZC1GSQ== -u "https://api.spyderbat.com" my_secret
​
Set new apisecret 'my_secret' in '/home/demouser/.spyctl/.secrets/secrets'
Spyctl saves APISecrets in $HOME/.spyctl/.secrets/secrets
Set a Context
Contexts will let Spyctl know where to look for data. The broadest possible Context is organization-wide. This means that when you run Spyctl commands, the Spyderbat API will return results relevant to your entire organization.
For the --org field in the following command you may supply the name of your organization which can be found in the top right of the Spyderbat Console or the organization UID which can be found in your web browser’s url when logged into the Spyderbat Console: https://app.spyderbat.com/app/org/UID/dashboard.
$ spyctl config set-context --org <ORG NAME or UID> --secret <SECRET NAME> NAME
For example:
$ spyctl config set-context --org "John's Org" --secret my_secret my_context
Set new context 'my_context' in configuration file '/home/demouser/.spyctl/config'.
You can view your configuration by issuing the following command:
$ spyctl config view
You should see something like this:
apiVersion: spyderbat/v1
kind: Config
contexts:
- name: my_context
  secret: my_secret
  context:
    organization: John's Org
current-context: my_context
The global configuration file is located at $HOME/.spyctl/config
It is possible to create more specific contexts, such as a group of machines or a specific container image. You can think of the fields in your context as filters to limit your scope. Follow this link to learn more about contexts: Contexts
Basic Usage
Now that you have configured a Context for your organization you can use Spyctl to view and manage your Spyderbat Resources.
Spyctl has numerous useful commands like get, delete, create, diff etc. 
In this section we will talk about one of these commands, specifically the get command.
The ‘get’ Command
To retrieve data from the Spyderbat API, you can use the get command:
$ spyctl get RESOURCE [OPTIONS] [NAME_OR_ID]
For some resources retrieved from the databases, a time range can be specified:
Connections
Containers
Agents
Example:
$ spyctl get connections -t 3h -e 1h
The t flag specifies the start time, whereas the e specifies the end time for the resource data query.
For other resources, time ranges are not applicable:
Clusters
Machines
Policies
To get a full list of resources with some usage examples use the spyctl get help flag:
$ spyctl get --help
To retrieve the list of machines with the Spyderbat Nano Agent installed, use the command below. By default, this displays a table of information about the resources you retrieved.
$ spyctl get machines
It is possible to output these resources in other formats:
$ spyctl get machines -o yaml
The JSON output formatter is organized as a collection of key-value pairs using json option.
$ spyctl get machines -o json
Using the ndjson, each JSON object is separated by a newline character, allowing for easy parsing and streaming of JSON data one object at a time.
$ spyctl get machines -o json --ndjson
For some of the resources like spydertraces you can also get the wide output with additional fields using the wide option.
$ spyctl get machines -o json NAME_OR_ID
If you wish to retrieve a specific object you may also supply a name or id with the command:
To know about all the available options for various commands, you can use the spyctl <command> --help
Baselining Workflow
In this section you will learn about how auto-generated Spyderbat Fingerprints are viewed and how they are used to Baseline your services and containers. You will also learn how to manage Baselines once you’ve created them.
Viewing Fingerprints
When you install the Spyderbat Nano Agent, Spyderbat immediately starts building up Fingerprints for the services and containers running on the machine. Fingerprints are used to create Baselines. Fingerprints are a compact representation of process and network activity for a given instance of a service or container, and can update over time.
To see a tabular summary of the Fingerprints in your current Context issue the command:
$ spyctl get fingerprints
When you retrieve Fingerprints from the Spyderbat API, you are actually retrieving are Fingerprint Groups. Container Fingerprints are grouped by image ID, and Linux Service Fingerprints are grouped by cgroup. This means that if the same service is running on multiple machines, all of the fingerprints across those machines get grouped together. The reason for this will become clear you move through the baselining process.
Here is an example of a Fingerprint Group:
apiVersion: spyderbat/v1
kind: FingerprintGroup
metadata:
  firstTimestamp: 1670000035
  lastTimestamp: 1670001133
data:
  fingerprints:
  - apiVersion: spyderbat/v1
    kind: SpyderbatFingerprint
    metadata:
      ...
    spec:
      containerSelector:
        image: python_webserver:latest
        imageID: sha256:6e2e1bce440ec41f53e849e56d5c6716ed7f1e1fa614d8dca2bbda49e5cde29e
      processPolicy:
      - name: sh
        exe:
        - /bin/dash
        id: sh_0
        euser:
        - root
        children:
        - name: python
          exe:
          - /usr/local/bin/python3.7
          id: python_0
      networkPolicy:
        ingress: []
        egress:
        - to:
          - dnsSelector:
            - mongodb.my_app.svc.cluster.local
          processes:
          - python_0
          ports:
          - protocol: TCP
            port: 27018
  - apiVersion: spyderbat/v1
    kind: SpyderbatFingerprint
    metadata:
    - ...
    spec:
      containerSelector:
        image: python_webserver:latest
        imageID: sha256:6e2e1bce440ec41f53e849e56d5c6716ed7f1e1fa614d8dca2bbda49e5cde29e
      processPolicy:
      - ...
      networkPolicy:
        ingress:
        - from:
          - ipBlock:
              cidr: 192.168.1.10/32
          processes:
          - python_0
          ports:
          - protocol: TCP
            port: 8080
        egress: []
Every Fingerprint will have the same four fields, apiVersion, kind, metadata, and spec. The Fingerprint Group shown above is for a specific container image. In the spec of every Fingerprint you will find one or more Selector fields. For now, just know that the containerSelector is used to group container Fingerprints together and the serviceSelector is used to group service Fingerprints together. In a separate tutorial you will learn how Selectors are used with Policies.
Creating a Baseline
Baselines are created from 1 or more Fingerprint Groups merged into a single document. The purpose of a Baseline is to represent the expected activity of a service or container image.
The first step to create a Baseline is to retrieve a Fingerprint Group and save it to a file. To do this, you use the get fingerprints command mentioned above. This will show you a table view of the available Fingerprint Groups.
For containers you can use the image or the image ID to retrieve a specific one:
$ spyctl get fingerprints -o yaml IMAGE_OR_IMAGE_ID > fprint_grp.yaml
For services you can use the cgroup:
$ spyctl get fingerprints -o yaml CGROUP > fprint_grp.yaml
For example, we want to save the Fingerprint Group for a container image python_webserver:latest:
$ spyctl get fingerprints -o yaml "python_webserver:latest" > python_srv_fprints.yaml
We just saved the auto-generated Fingerprints for all instances of the container image to a single yaml file.
The next step is to create a Baseline from that Fingerprint Group. The command to create a Baseline is:
$ spyctl create baseline --from-file FILENAME > baseline.yaml
Continuing the example from above, we would issue this command:
$ spyctl create baseline --from-file python_srv_fprints.yaml > python_srv_baseline.yaml
The resulting Baseline would look something like this:
apiVersion: spyderbat/v1
kind: SpyderbatBaseline
metadata:
  name: webserver_baseline
  type: container
  latestTimestamp: 1670001133
spec:
  containerSelector:
    image: "python_webserver:latest"
  processPolicy:
  - name: sh
    exe:
    - /bin/dash
    id: sh_0
    euser:
    - root
    children:
    - name: python
      exe:
      - /usr/local/bin/python3.7
      id: python_0
  networkPolicy:
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.1.10/32
      processes:
      - python_0
      ports:
      - protocol: TCP
        port: 8080
    egress:
    - to:
      - dnsSelector:
        - mongodb.my_app.svc.cluster.local
      processes:
      - python_0
      ports:
      - protocol: TCP
        port: 27017
In this example the root process of the container is sh run as root with a child python process. The ingress traffic is coming from 192.168.1.10/32 and the only egress traffic is going to a database with the dns name mongodb.my_app.svc.cluster.local.
Generalizing a Baseline
Fingerprints only capture activity that has occurred, so if you want your Baselines to include other expected activity, you can take steps to generalize the document. This can be done by simply editing the baseline document with your favorite text editor.
For example:
$ vim python_srv_baseline.yaml
Some ways to generalize a Baseline are to:
add wildcards to text fields (e.g. updating the image to incorporate all versions):
image: python_webserver:*
expand an ip block’s cidr range (e.g. say there is a /16 network that we expect traffic from):
Managing A Baseline
We now have a Baseline python_srv_baseline.yaml that we have generalized. The goal now is to stabilize the Baseline. Your services and containers will continue to generate updated Fingerprints which may contain activity that deviates from the Baseline. The way to detect this is with the diff command:
$ spyctl diff -f BASELINE_FILE --latest
For example:
$ spyctl diff -f python_srv_baseline.yaml --latest
The output of the diff command will display all activity that doesn’t match the Baseline. If there are deviations, and those deviations should be added to the Baseline, you can use the merge command to add them to the Baseline:
$ spyctl merge -f BASELINE_FILE --latest > merged_baseline.yaml
For example:
$ spyctl merge -f python_srv_baseline.yaml --latest > python_srv_merged_baseline.yaml
Never redirect output to the same file you are using as input, the file will be wiped before Spyctl can read it.
At this point you may want to edit the file again to generalize more fields. Repeat these management steps until you’re satisfied that your Baseline has stabilized.
Our stable (for now) baseline now looks as follows:
apiVersion: spyderbat/v1
kind: SpyderbatBaseline
metadata:
  name: webserver_baseline
  type: container
  latestTimestamp: 1670001133
spec:
  containerSelector:
    image: "python_webserver:*"
  processPolicy:
  - name: sh
    exe:
    - /bin/dash
    id: sh_0
    euser:
    - root
    children:
    - name: python
      exe:
      - /usr/local/bin/python3.7
      id: python_0
  networkPolicy:
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.0/16
      processes:
      - python_0
      ports:
      - protocol: TCP
        port: 8080
    egress:
    - to:
      - dnsSelector:
        - mongodb.my_app.svc.cluster.local
      processes:
      - python_0
      ports:
      - protocol: TCP
        port: 27017
Spyctl: Getting Started
Policy Management Tutorial
Last modified 1mo ago