Logging Scenario - Send Local Logs to S3

Scenario: You want to save gateway/relay logs to an Amazon S3 bucket. This guide presents a simple method to send all gateway/relay logs to S3.

Set up the Export #

1. Enable relay logging in the Admin UI under Settings > Security within the Log Encryption & Storage tab. Ensure logging is set to STDOUT.

2. Create an admin token with only the Audit > Activities permission. Save this token to add to the script in step 5.

3. From the AWS IAM Dashboard, go to Access management > Users. On the Security credentials tab for the user, generate and AWS access key and AWS secret access key.

4. Ensure the gateway or relay has the aws-cli tools installed.

5. Save the following script as s3export.sh. This script exports in 15-minute intervals; if you prefer to do it more or less frequently change the FROMTIME and TOTIME variables.

   #!/bin/bash
   
   # day, hour, minute timestamp
   TIMESTAMP=`date +'%Y%m%d%H%M'`
   # to prevent overlapping records, do 16 min ago to 1 min ago
   FROMTIME=`date --date="16 minutes ago" +'%Y-%m-%d %H:%M:%S'`
   TOTIME=`date --date="1 minutes ago" +'%Y-%m-%d %H:%M:%S'`
   # this token needs only audit/activities permission
   export SDM_ADMIN_TOKEN=[token]
   S3NAME=strongdm-log-$TIMESTAMP.gz
   S3ACTIVITIESNAME=strongdm-activities-$TIMESTAMP.gz
   S3PATH=s3://bucket/path/to/logs # no trailing slash
   export AWS_ACCESS_KEY_ID=[token]
   export AWS_SECRET_ACCESS_KEY=[token]
   
   # ensure AWS environment variables are in place
   
   journalctl -q -o cat --since "$FROMTIME" --until "$TOTIME" -u sdm-proxy | \
   gzip | aws s3 cp - $S3PATH/$S3NAME
   
   sdm audit activities --from "$FROMTIME" --to "$TOTIME" | \
   gzip | aws s3 cp - $S3PATH/$S3ACTIVITIESNAME
   

6. Add the following line to /etc/crontab. If you changed the export interval above, change the cron interval here to match.

   0,15,30,45 * * * * root /home/ubuntu/s3export.sh
   

7. Verify that files are being generated every 15 minutes in your S3 bucket.

Extract SSH Captures From Exported Logs #

If your organization requires or is interested in extracting captured SSH sessions from your exported activity logs, see section Audit SSH for more information.