# Configuration Guide - AWS Linux VM This guide explains how to configure the Spyderbat AWS Agent to collect information from an AWS account and send it to the Spyderbat platform. It provides detailed instructions for locating the configuration file, managing AWS credentials, and configuring all available settings. *** ## 1. Managing the configuration The Spyderbat AWS Agent's configuration file is a YAML file named `aws-agent.yaml`. It is used to control the behavior of the agent, such as which AWS services to monitor, where to send data, and how to manage credentials. ### Locating the Configuration File By default, the configuration file is located at: ``` /opt/spyderbat/etc/aws-agent.yaml ``` This file can be edited using any text editor with root privileges. For example: ```bash sudo vi /opt/spyderbat/etc/aws-agent.yaml ``` ### Applying Changes After making changes to the configuration file, the AWS Agent service must be restarted to apply the updates. Use the following command to restart the service: ```bash sudo systemctl restart aws_agent.service ``` ### Validating Configuration To ensure the configuration file is valid, check the service status after restarting: ```bash sudo systemctl status aws_agent.service aws_agent.service - Spyderbat AWS Agent Service Loaded: loaded (/etc/systemd/system/aws_agent.service; enabled; preset: disabled) Active: active (running) since Wed 2024-12-11 18:48:48 UTC; 3 weeks 6 days ago Main PID: 2146512 (aws_agent) Tasks: 8 (limit: 1112) Memory: 10.1M CPU: 4min 7.715s CGroup: /system.slice/aws_agent.service ├─2146512 /usr/bin/bash /opt/spyderbat/bin/aws_agent └─2146528 docker run --pull always -v /opt/spyderbat/etc:/etc/aws-config --name aws-agent public.ecr.aws/a6j2k0g1/aws-agent:latest --config /etc/aws-> Jan 08 12:24:30 ip-172-31-86-31.ec2.internal aws_agent[2146528]: 2025-01-08 12:24:30,479:INFO :poller eks/us-west-1 got 2 records Jan 08 12:24:30 ip-172-31-86-31.ec2.internal aws_agent[2146528]: 2025-01-08 12:24:30,755:INFO :Sending heartbeat and stat update Jan 08 12:24:31 ip-172-31-86-31.ec2.internal aws_agent[2146528]: 2025-01-08 12:24:31,346:INFO :Session(region_name=None) IAM Poller got 56 roles and their inl> ``` If there are any errors, they will be displayed in the status output. Ensure the YAML syntax is correct before restarting the service again. ### Checking agent logs The agent logs can be consulted based on the service journal: ```bash sudo journalctl -u aws_agent.service ``` *** ## 2. AWS Credentials Management The Spyderbat AWS Agent requires access to AWS services to collect data. The agent supports multiple methods for obtaining credentials, listed below in the order of precedence: ### 1. **IAM Instance Profile (Recommended)** If the agent is deployed on an EC2 instance with an IAM role assigned, it will automatically use the instance profile credentials. This is the most secure and recommended method. No additional configuration is required for this setup. ### 2. **Environment Variables** You can set the following environment variables to provide credentials explicitly: * `AWS_ACCESS_KEY_ID` * `AWS_SECRET_ACCESS_KEY` For example, add the variables to the environment: ```bash export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= ``` ### 3. **From Files** The agent can also read credentials from files. This is typically used when credentials are mounted as secrets in Kubernetes or other containerized environments. Place the credentials in the following files: * `/etc/aws-config/secrets/aws_access_key_id` * `/etc/aws-config/secrets/aws_secret_access_key` **Note**: This method is not recommended for standalone deployments. *** ## 3. Configuration Settings Below is a detailed explanation of each configuration setting available in the `aws-agent.yaml` file. ### `spyderbat_orc_url` * **Description**: The URL of the Spyderbat orchestration API endpoint. This is where the agent sends the collected data. * **Example**: ```yaml spyderbat_orc_url: https://orc.spyderbat.com ``` * **Default**: `https://orc.spyderbat.com` *** ### `outfile` * **Description**: Specifies a file where the agent writes the collected data instead of sending it to the Spyderbat backend. This is primarily for debugging purposes. * **Example**: ```yaml outfile: /tmp/out.json.gz ``` * **Default**: Not set. *** ### `cluster_name` * **Description**: The name of the Kubernetes cluster, used for identification in the Spyderbat UI. This is optional for standalone deployments. * **Example**: ```yaml cluster_name: staging-cluster-us-east-1 ``` * **Default**: Not set. *** ### `aws_account_id` * **Description**: Specifies the AWS account ID the agent monitors. Use `auto` for auto-discovery. * **Example**: ```yaml aws_account_id: auto ``` * **Default**: `auto` *** ### `role_arn` * **Description**: The ARN of the IAM role the agent assumes to gather information. This is useful when explicit AWS credentials are used. It should not be used if the correct role was already assumed through an EC2 IAM Instance Profile. * **Example**: ```yaml role_arn: arn:aws:iam::123456789012:role/SpyderbatRole ``` * **Default**: Not set. *** ### `send_buffer_size` * **Description**: The number of records accumulated before sending data to the Spyderbat backend. * **Example**: ```yaml send_buffer_size: 100 ``` * **Default**: `100` *** ### `send_buffer_records_bytes` * **Description**: The maximum size (in bytes) of accumulated records before sending to the backend. * **Example**: ```yaml send_buffer_records_bytes: 1000000 ``` * **Default**: `1000000` (1 MB) *** ### `send_buffer_max_delay` * **Description**: The maximum delay (in seconds) before sending accumulated records, even if the buffer is not full. * **Example**: ```yaml send_buffer_max_delay: 30 ``` * **Default**: `30` *** ### `log_level` * **Description**: Configures the logging level for the agent. * **Options**: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` * **Example**: ```yaml log_level: INFO ``` * **Default**: `INFO` *** ### `pollers` * **Description**: Configures the AWS services and regions to monitor. Each entry specifies a service, polling interval, and regions. * **Example**: ```yaml pollers: - service: ec2 polling_interval: 30 regions: - us-east-1 - us-west-2 - service: eks polling_interval: 30 regions: - us-east-1 - us-east-2 ``` * **Default**: Monitors all supported services and regions if not set explicitly. Per service in the pollers section, the following properties can be set: ### `polling_interval` * **Description**: The interval in seconds at which the agent will poll the service. * **Example**: ```yaml - service: eks polling_interval: 30 ``` * **Default**: 30 ### `regions` * **Description**: The regions that the agent will poll for the service. If not set, the agent will poll all regions. * **Example**: ```yaml - service: eks regions: - us-east-1 - us-east-2 ``` * **Default**: not set (all regions) ## Example configuration file You can find an example illustrated configuration yaml file [here](https://github.com/spyderbat/spyderbat-docs/blob/main/installation/spyderbat-aws-agent/aws-agent-example.yaml)