How to Lock Down Your Workloads With Guardian Policies Using Spyctl

This page will teach you about Guardian Workload Policies. It will explain what they are, how to create them, how to apply them, and how to manage them.

Prerequisites

• Install Spyctl

• Configure Spyctl with a Context

• Have installed at least one Spyderbat Nano Agent installed on a machine of your choosing

What is a Guardian Workload Policy

Spyderbat's Guardian Workload Policy feature empowers users to define a known-good whitelist of process and network behavior for containers and Linux services. These policies serve as a proactive measure, enabling users to receive notifications when Spyderbat detects deviant activity within their workloads. Not only does the Guardian Workload Policy provide alerts, but it also offers the capability to take decisive actions by terminating deviant processes and Kubernetes pods. This comprehensive approach ensures a robust security framework, allowing users to maintain a vigilant stance against potential threats and unauthorized activities within their environments.

Workload Policies also serve as a way to tune out certain Red Flags & Spydertraces. Red Flags and Spydertraces that would have been generated by your workloads will be marked as exceptions and reduce clutter in your security dashboards.

Retrieving Fingerprints

Policies are created from Fingerprints. Fingerprints are auto-generated documents with the process and network activity for a single container instance or instance of a Linux Service.

To view all Fingerprints generated across your organization, issue the following command:

$ spyctl get fingerprints --type FINGERPRINT_TYPE [NAME_OR_UID]

For example:

$ spyctl get fingerprints --type container docker
Getting fingerprints from 2024-01-16T13:52:51Z to 2024-01-16T15:22:51Z
IMAGE_NAME:TAG        IMAGEID       REPO                        COVERED_BY_POLICY    LATEST_TIMESTAMP                                                                                                                                                                                                                                                                 
mongo:latest          8248f2793e07  docker.io/library           0/2                  2024-01-16T15:00:43Z
nginx:latest          10d1f5b58f74  docker.io/library           0/14                 2024-01-16T15:01:08Z
node:v3.23.5          b7f4f7a0ce46  docker.io/calico            0/1                  2024-01-16T15:01:08Z

By default, Spyderbat queries for all Fingerprints in your organization for the last 1.5 hours. This means that it will retrieve Fingerprints for any container or Linux service instance running during that time window (Even if the instances started well before the time window). You can increase the time range with the -t option.

In this example organization we have 3 workloads running across multiple instances. Spyderbat has Fingerprints for two instances of mongo, 14 instances of nginx, and one instance of node that were online during the query's time window. None of these Fingerprints are covered by a policy as seen in the COVERED_BY_POLICY column.

Download Fingerprints to a File

To create a policy we must first download the fingerprints we wish to use to build the policy.

spyctl get fingerprints --type TYPE --output OUTPUT NAME_OR_UID > FILENAME

For example:

spyctl get fingerprints --type container -o yaml mongo:latest > fprints.yaml

[Optional] Downloading Fingerprints from a K8s Namespace

In certain instances you may have the same container image running in different Kubernetes namespaces with different allowed network activity. To separate allowed network activity by namespace you can use multiple policies for the same image. Using only Fingerprints from the same namespace will automatically scope the policy to that namespace.

spyctl get fingerprints --type container --output OUTPUT [--cluster CLUSTER_NAME_OR_UID] --namespace NAMESPACE NAME_OR_UID > FILENAME

for example:

spyctl get fingerprints --type container -o yaml --namespace dev mongo:latest > fprints-dev.yaml
spyctl get fingerprints --type container -o yaml --namespace prod mongo:latest > fprints-prod.yaml

This will only download Fingerprints tied to a the namespace specified in the command.

Create the Policy

Once you have the Fingerprints to create the policy from, issue the following command:

$ spyctl create policy --from-file FILENAME --name NAME_FOR_POLICY --mode MODE > policy.yaml

For example:

$ spyctl create policy --from-file fprints.yaml --name mongo-policy --mode audit > policy.yaml

Running this command does not make any changes to your Spyderbat Environment. It is not until you have applied a Policy, that enforcement takes effect.

The Policy file we just created policy.yaml now has a new resource, the kind field is now "SpyderbatPolicy". This document is a merged version of the Fingerprints that went into it:

apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  latestTimestamp: 1705435213.327981
  name: mongo-policy
  type: container
spec:
  containerSelector:
    image: docker.io/library/mongo:latest
    imageID: sha256:68248f2793e077e818710fc5d6f6f93f1ae5739b694d541b7e0cd114e064fa11
  mode: audit
  processPolicy:
  - name: mongod
    exe:
    - /usr/bin/mongod
    id: mongod_0
    euser:
    - mongo
  networkPolicy:
    egress: []
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.229/32
      - ipBlock:
          cidr: 192.168.1.146/32
      - ipBlock:
          cidr: 192.168.2.221/32
      - ipBlock:
          cidr: 192.168.4.31/32
      processes:
      - mongod_0
      ports:
      - protocol: TCP
        port: 27017
  response:
    default:
    - makeRedFlag:
        severity: high
    actions: []

Policies are created in audit mode by default. If you apply a Policy in audit mode it will not take response actions, but will log the activity it would have taken. You can use the command spyctl logs policy POLICY_UID to monitor those log.

Generalize the Policy

In its current form, this policy will only apply to mongo containers with the latest tag and only with the image ID sha256:68248f2793e077e818710fc5d6f6f93f1ae5739b694d541b7e0cd114e064fa11

We can remove selector fields and wildcard values to broaden the Policy's scope.

Using the edit command will open your favorite text-editor and perform syntax checking when you save.

spyctl edit policy.yaml

Then we can generalize the containerSelector and the IP blocks

apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  latestTimestamp: 1705435213.327981
  name: mongo-policy
  type: container
spec:
  containerSelector:
    image: docker.io/library/mongo:*
  mode: audit
  processPolicy:
  - name: mongod
    exe:
    - /usr/bin/mongod
    id: mongod_0
    euser:
    - mongo
  networkPolicy:
    egress: []
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.0/16
      processes:
      - mongod_0
      ports:
      - protocol: TCP
        port: 27017
  response:
    default:
    - makeRedFlag:
        severity: high
    actions: []

In the above policy we removed the imageID field in the containerSelector we also increase the scope of the ipBlock in ingest from multiple /32 CIDRs to a single /16 CIDR.

Applying the Policy

To apply a Policy you must use the apply command:

$ spyctl apply -f FILENAME

The apply command will recognize the kind of the file, perform validation, and attempt to apply the resource to the policy database for the organization in your current Context. It accomplishes this via the Spyderbat API.

For example, to apply the Policy we created above:

$ spyctl apply -f policy.yaml

This will apply the Policy to the organization in your current Context.

To view the applied Policies in your current Context you can use the get command:

$ spyctl get RESOURCE [OPTIONS] [NAME_OR_ID]

For example:

$ spyctl get policies
UID                       NAME              STATUS     TYPE       CREATE_TIME
pol:CB1fSLq4wpkFG5kWsQ2r  mongo-policy      Auditing   container  2024-01-16T15:00:43Z

To view the yaml of the Policy you just applied, issue the command:

$ spyctl get policies -o yaml CB1fSLq4wpkFG5kWsQ2r

The Policy will look something like this:

apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  name: mongo-policy
  type: container
  uid: pol:CB1fSLq4wpkFG5kWsQ2r
  creationTimestamp: 1673477668
  latestTimestamp: 1670001133
spec:
  containerSelector:
    image: docker.io/library/mongo:*
  mode: audit
  processPolicy:
  - name: mongod
    exe:
    - /usr/bin/mongod
    id: mongod_0
    euser:
    - mongo
  networkPolicy:
    egress: []
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.0/16
      processes:
      - mongod_0
      ports:
      - protocol: TCP
        port: 27017
  response:
    default:
    - makeRedFlag:
        severity: high
    actions: []

[Optional] Adding "Interceptor" Response Actions

When a new Policy is created it will have a default Actions list, and an empty list of actions. The default Actions are taken when a policy is violated and no Actions of the same type in the actions list are taken.

response:
  default:
  - makeRedFlag:
      severity: high
  actions: []

By default, spyctl includes a makeRedFlag Action in the default section of the policy’s response field. This tells the Spyderbat backend to generate a redflag of high severity which will show up in the Spyderbat Console. The full list of redflag severities, in increasing severity, is as follows:

• info

• low

• medium

• high

• critical

The Actions in the actions field are taken when certain criteria are met. Every Action in the actions field must include a Selector. Selectors are a way of limiting the scope of an Action. For example, you can tell Spyderbat to kill a bash process that deviates from the Policy by using the processSelector:

actions:
- agentKillProcess:
    processSelector:
      name:
      - bash

If you are in a Kubernetes environment you can also set up an Action to kill a pod when a Policy violation occurs. Let's say you want to kill a pod in your staging environment, the action would look like so:

actions:
- agentKillPod:
    podSelector:
      matchLabels:
        env: staging

To add a kill process action, edit your policy file. For example:

$ spyctl edit policy pol:CB1fSLq4wpkFG5kWsQ2r

And add a kill process Action to the actions list.

response:
  default:
  - makeRedFlag:
      severity: high
  actions:
  - agentKillProcess:
      processSelector:
        name:
        - bash

Our Policy now looks like this:

apiVersion: spyderbat/v1
kind: SpyderbatPolicy
metadata:
  name: mongo-policy
  type: container
  uid: pol:CB1fSLq4wpkFG5kWsQ2r
  creationTimestamp: 1673477668
  latestTimestamp: 1670001133
spec:
  containerSelector:
    image: docker.io/library/mongo:*
  mode: audit
  processPolicy:
  - name: mongod
    exe:
    - /usr/bin/mongod
    id: mongod_0
    euser:
    - mongo
  networkPolicy:
    egress: []
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.0/16
      processes:
      - mongod_0
      ports:
      - protocol: TCP
        port: 27017
  response:
    default:
    - makeRedFlag:
        severity: high
    actions:
    - agentKillProcess:
        processSelector:
          name:
          - bash

Summary and Next Steps

At this point you should have an applied policy in audit mode. You'll find that if you run spyctl get fingerprints --type container or spyctl get fingerprints --type linux-service the fingerprints you included in the policy will now be covered.

$ spyctl get fingerprints --type container docker
Getting fingerprints from 2024-01-16T13:52:51Z to 2024-01-16T15:22:51Z
IMAGE_NAME:TAG        IMAGEID       REPO                        COVERED_BY_POLICY    LATEST_TIMESTAMP                                                                                                                                                                                                                                                                 
mongo:latest          8248f2793e07  docker.io/library           2/2                  2024-01-16T15:00:43Z
nginx:latest          10d1f5b58f74  docker.io/library           0/14                 2024-01-16T15:01:08Z
node:v3.23.5          b7f4f7a0ce46  docker.io/calico            0/1                  2024-01-16T15:01:08Z

While in audit mode your policy will generate logs of the actions it would have taken, in addition to any deviations it detects. To learn how to manage and update your policies refer to Guardian Policy Management in Spyctl. It explains how to edit, update, and diff your policies. It will also explain when to graduate your policies from audit to enforce mode.

When all of your policies are in enforce mode and every fingerprint is covered by a policy you will have locked down your environment using Guardian. You will have established a whitelist of activity for your all of your critical workloads which simultaneously allows you to quickly be notified when real threats occur, and reduce red flag and spydertrace noise in your Spyderbat Dashboards.