Log Stream

Last modified on October 31, 2024

This feature is part of the Enterprise plan. If it is not enabled for your organization, please contact StrongDM at the StrongDM Help Center.

Log Stream allows you to stream your StrongDM audit logs to a third-party object storage service, such as Amazon S3. Having logs streamed to an external storage location for ingestion into security information and event management (SIEM) applications can help provide insights and visibility to compliance, security, and other teams.

For more information about logging, please see Logs.

How to Configure Log Stream

Log Stream is enabled and configured in the Admin UI in Settings > Log Stream.

Amazon S3

The following steps show you how to configure Log Stream for streaming to an Amazon S3 bucket.

Under Log Stream, select Yes for Enable Log Stream?.
Select the Provider name (Amazon S3).
Enter the Region of your target bucket (for example, us-west-2).
Enter the Bucket Name (for example, unique-bucket-name).
Enter the Key Prefix (for example, sdm/logs).
Your region, bucket name, and key prefix values are then populated in a bucket policy for a general S3 bucket and in a bucket policy for a Customer Managed Key (CMK)-encrypted bucket. Click Copy to save the desired bucket policy to your clipboard.
The CMK bucket policy option includes an additional permission that allows S3 buckets to accept logs from StrongDM if the buckets use AWS Key Service Management (KMS) key encryption. Choose the CMK bucket policy option if you are using an AWS KMS CMK to encrypt S3 objects. After copying it, remember to replace any placeholder field values with your own.
Example bucket policy:
```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "StrongDMLogStream",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::910226215634:role/StrongDMLogStream"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::unique-bucket-name/sdm/logs/*"
    }
  ]
}
```
Example CMK-encrypted bucket policy to add to KMS:
```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "StrongDMLogStreamKMS",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::910226215634:role/StrongDMLogStream"
      },
      "Action": "kms:GenerateDataKey",
      "Resource": "arn:aws:kms:...<your key ARN>"
    }
  ]
}
```
In a separate browser tab or window, sign in to the AWS Management Console, open the Amazon S3 console, go to Buckets, and choose the name of the desired bucket.
Edit your bucket policy. If using the general bucket policy, on the bucket’s Permissions tab, click Edit bucket policy to add the bucket policy that you copied. If using the CMK-encrypted bucket policy option, add your key ARN to the bucket policy’s Resource field, add the bucket policy to KMS, and save. The key ARN allows encrypted data to be written to the bucket with the given key.
After saving changes to your bucket policy, return to the Admin UI Log Stream settings and click Test to test that data from StrongDM can be written to your S3 bucket location. The test uses the provided bucket name and key prefix values to write a test object to <BUCKET_NAME>/<KEY_PREFIX>/test.

A successful test displays the “Log Stream test successful” message. If the test fails, check that the region, bucket name, and key prefix values that you entered are correct, and ensure that the bucket policy is saved correctly in the Amazon S3 console.

When configuration is complete, your audit logs are streamed to the connected S3 bucket.

If an unrecoverable error occurs when streaming logs to the target storage location, such as if StrongDM is unable to stream some or all data to the target storage location, StrongDM administrators in your organization receive an email detailing the problem. In addition, the Log Stream settings provide more information about the stream status.

Moreover, if StrongDM detects a streaming failure, then logs are no longer streamed to your configured location. Once you have resolved the connection issue, logs automatically start streaming again. If you wish, you can retrieve logs from the duration of the failure using the StrongDM API or CLI.

Log storage in S3

Logs that are streamed to S3 are batched into multiple log entries and written to a file.

The location and format of each file is as follows:

Activities: /activities/YYYY/MM/DD/HH/MM/<randomUUID>.json
Queries: /queries/YYYY/MM/DD/HH/MM/<randomUUID>.json
Replays: /replays/YYYY/MM/DD/HH/MM/<randomUUID>.json

This storage format of logs includes multiple log entries in a single file, batching groups of writes together. It also can include log entries from multiple different activities or queries in the same file. One or more JSON log entries are separated by a new line.

Entries are written immediately, unless a write is already in progress, in which case up to 1000 entries or approximately 100 MiB of entries are accumulated for the subsequent write.

If you have an automation that parses the log files, please ensure that the scripts match the format and location of the data.

Examples of Streamed Logs

When Log Stream is configured, several varieties of logs are streamed to the target storage location, including activities, queries for all resource types, and replay data for interactive sessions. This section provides examples of each log format.

Test

{
    "formatVersion": "v1.0.0",
    "logType": "activities",
    "actorAccountID": "a-0000000000000000",
    "timestamp": "0001-01-01T00:00:00Z"
}

Activity log example

{
    "formatVersion": "v1.0.0",
    "id": "at-33faa6fb65df876a",
    "logType": "activities",
    "activity": "user logged into the Admin UI",
    "actorAccountID": "a-0abcdabcdab00000",
    "actorEmail": "alice.glick@strongdm.com",
    "actorFirstName": "Alice",
    "actorLastName": "Glick",
    "actorExternalID": "e-bca5454",
    "description": "Alice Glick (alice.glick@strongdm.com) logged into the Admin UI.",
    "ipAddress": "123.123.123.123",
    "userAgent": "strongdm-cli/1.2.3",
    "entities": [
        {
            "type": "user",
            "id": "a-0abcdabcdab00000",
            "email": "alice.glick@strongdm.com"
        }
    ],
    "timestamp": "2024-02-01T18:50:04.782960647Z",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}
{
    "formatVersion": "v1.0.0",
    "id": "at-44faa6fb65df876b",
    "logType": "activities",
    "activity": "User added to role",
    "actorAccountID": "a-0abcdabcdab00000",
    "actorEmail": "alice.glick@strongdm.com",
    "actorFirstName": "Alice",
    "actorLastName": "Glick",
    "actorExternalID": "e-bca5454",
    "description": "Alice Glick (alice.glick@strongdm.com) assigned Bob Belcher (bob.belcher@strongdm.com) to DEVROLE.",
    "ipAddress": "123.123.123.123",
    "userAgent": "strongdm-cli/1.2.3",
    "entities": [
        {
            "type": "user",
            "id": "a-0abcdabcdab00000",
            "email": "alice.glick@strongdm.com"
        }
    ],
    "timestamp": "2024-02-01T18:50:05.782960648Z",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}
{
    "formatVersion": "v1.0.0",
    "id": "at-55faa6fb65df876c",
    "logType": "activities",
    "activity": "user logged into the Admin UI",
    "actorAccountID": "a-0defgdefgde00001",
    "actorEmail": "bob.belcher@strongdm.com",
    "actorFirstName": "Bob",
    "actorLastName": "Belcher",
    "actorExternalID": "e-def1212",
    "description": "Bob Belcher (bob.belcher@strongdm.com) logged into the Admin UI.",
    "ipAddress": "456.456.456.456",
    "userAgent": "strongdm-cli/1.2.3",
    "entities": [
        {
            "type": "user",
            "id": "a-0defgdefgde00001",
            "email": "bob.belcher@strongdm.com"
        }
    ],
    "timestamp": "2024-02-02T18:50:04.782960646Z",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}

Query log example

{
    "formatVersion": "v1.0.0",
    "logType": "queries",
    "egressNodeID": "n-abcd1234",
    "actorAccountID": "a-0abcdabcdab00000",
    "actorFirstName": "Bob",
    "actorLastName": "Belcher",
    "actorEmail": "bob.belcher@strongdm.com",
    "actorExternalID": "e-def1212",
    "durationMs": "20",
    "hash": "0da22222ba9b212ecfed33a17147c466ae0929ca",
    "query": "select name from users",
    "queryCategory": "cloud",
    "resourceID": "rs-0cegcegcegceg00",
    "resourceName": "MySQL",
    "resourceTags": {
        "env": "dev"
    },
    "resourceType": "mysql",
    "sourceIp": "1.11.222.333",
    "timestamp": "2024-02-01T13:13:20.895597162Z",
    "uuid": "0CEGCEGCEGCEGCEGCEGCE1234ceg",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}
{
    "formatVersion": "v1.0.0",
    "logType": "queries",
    "egressNodeID": "n-abcd1234",
    "actorAccountID": "a-0defgdefgde00001",
    "actorFirstName": "Bob",
    "actorLastName": "Belcher",
    "actorEmail": "bob.belcher@strongdm.com",
    "actorExternalID": "e-bca5454",
    "durationMs": "20",
    "hash": "0da22222ba9b212ecfed33a17147c466ae0929fb",
    "query": "select name from users",
    "queryCategory": "cloud",
    "resourceID": "rs-0abcdabcdabcd00",
    "resourceName": "AWS Development",
    "resourceTags": {
        "env": "dev"
    },
    "resourceType": "aws",
    "sourceIp": "1.23.456.789",
    "timestamp": "2024-02-02T13:10:20.895597163Z",
    "uuid": "0ABCDABCDABCDABCDABCD1234abc",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}
{
    "formatVersion": "v1.0.0",
    "logType": "queries",
    "egressNodeID": "n-abcd1234",
    "actorAccountID": "a-0mnomnomnom00003",
    "actorFirstName": "Bob",
    "actorLastName": "Belcher",
    "actorEmail": "bob.belcher@strongdm.com",
    "actorExternalID": "e-bca5454",
    "durationMs": "42",
    "hash": "0da22222ba9b212ecfed44b17147c466ae0929fb",
    "query": "select name from users",
    "queryCategory": "cloud",
    "resourceID": "rs-0abcdabcdabcd00",
    "resourceName": "AWS Development",
    "resourceTags": {
        "env": "dev"
    },
    "resourceType": "aws",
    "sourceIp": "1.23.456.789",
    "timestamp": "2024-02-03T12:10:20.895597166Z",
    "uuid": "0ABCDABCDABCDABCDABCD1234abc",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}

Replay log example

{
    "formatVersion": "v1.0.0",
    "logType": "replays",
    "chunkID": "1",
    "events": [
        {
            "data": "base64binarydata",
            "durationMs": "46"
        }
    ],
    "hash": "0da22222ba9b212ecfed33a17147c466ae0929fb",
    "queryUUID": "sABCDABCDABCDABCDABCD1234abc",
    "timestamp": "2024-02-26T21:14:12.377835194Z",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}
{
    "formatVersion": "v1.0.0",
    "logType": "replays",
    "chunkID": "2",
    "events": [
        {
            "data": "base64binarydata",
            "durationMs": "50"
        }
    ],
    "hash": "0ab33333ba9b212ecfed33a17147c466ae0929fz",
    "queryUUID": "sDEFDEFDEFDEFDEFDEFDE1234def",
    "timestamp": "2024-02-26T21:15:12.377835194Z",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}
{
    "formatVersion": "v1.0.0",
    "logType": "replays",
    "chunkID": "3",
    "events": [
        {
            "data": "base64binarydata",
            "durationMs": "54"
        }
    ],
    "hash": "0lk44444ba9b212ecfed33a17147c466ae0929xy",
    "queryUUID": "sXYZXYZXYZXYZXYZXYZXYZ567xyz",
    "timestamp": "2024-02-26T21:16:12.377835194Z",
    "sdmOrgId":"o-6dce5b5663c12e6b"
}

Additional Information

The Log Encryption and Storage settings determine what data, if any, is stored with StrongDM, and in turn, what information is available via Log Stream. Activities are sent to Log Stream regardless of the Log Encryption and Storage settings.

Setting the Store with StrongDM? option to No prevents any data (replay or other queries) from being sent to or stored with StrongDM. This ensures that the data won’t leave the nodes (gateways and relays). Only metadata (such as user, resource, and total replay time) is sent and stored, even if Log Stream is enabled.
For the Remote encryption? setting, choosing to store logs with StrongDM with an encryption key causes the encryption of the log information or replay data using the key before it leaves the node but still store it (encrypted) with StrongDM and Log Stream.
Setting the Retain SSH, RDP, and Kubernetes Replay Data sent to StrongDM? setting to No prevents replay data from persisting in StrongDM. As long as Log Stream is enabled, replay data is sent to StrongDM and is then passed on to the external storage location that you configure. Only non-replay queries and replay metadata are stored in StrongDM with this setting on No.

Log References

To view a list of all available fields and their descriptions for each event type, please see the Log References section.

Log Stream

How to Configure Log Stream #

Amazon S3 #

Log storage in S3 #

Examples of Streamed Logs #

Test #

Activity log example #

Query log example #

Replay log example #

Additional Information #

Log References #