Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 1 addition & 18 deletions docs/integrations/amazon-aws/elasticache.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,24 +142,7 @@ account={{account}} region={{region}} namespace={{namespace}} "\"eventSource\":\

### Field Extraction Rule(s)

Create a Field Extraction Rule for CloudTrail Logs. Learn how to create a Field Extraction Rule [here](/docs/manage/field-extractions/create-field-extraction-rule).

```sql
Rule Name: AwsObservabilityElastiCacheCloudTrailLogsFER
Applied at: Ingest Time
Scope (Specific Data): account=* eventname eventsource "elasticache.amazonaws.com"
```

**Parse Expression**

```sumo
| json "eventSource", "awsRegion", "requestParameters.cacheClusterId", "responseElements.cacheClusterId", "recipientAccountId" as eventSource, region, req_cacheClusterId, res_cacheClusterId, accountid nodrop
| where eventSource = "elasticache.amazonaws.com"
| if (!isEmpty(req_cacheClusterId), req_cacheClusterId, res_cacheClusterId) as cacheclusterid
| "aws/elasticache" as namespace
| tolowercase(cacheclusterid) as cacheclusterid
| fields region, namespace, cacheclusterid, accountid
```
The FER `AwsObservabilityElastiCacheCloudTrailLogsFER` to extract fields `eventSource`, `region`, `req_cacheClusterId`, `res_cacheClusterId`, and `accountid` will be created as a part of app installation.

### Centralized AWS CloudTrail Log Collection

Expand Down
29 changes: 2 additions & 27 deletions docs/integrations/amazon-aws/lambda.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,23 +225,7 @@ Create a Field Extraction Rule for AWS Lambda. Learn how to create a Field Extra

### Cloud Trail FER

```sql
Rule Name: AwsObservabilityFieldExtractionRule
Applied at: Ingest Time
Scope (Specific Data): account=* eventname eventsource "lambda.amazonaws.com"
```

```sumo title="Parse Expression"
| json "eventSource", "awsRegion", "requestParameters", "recipientAccountId" as eventSource, region, requestParameters, accountid nodrop
| where eventSource = "lambda.amazonaws.com"
| json field=requestParameters "functionName", "resource" as functionname, resource nodrop
| parse regex field=functionname "\w+:\w+:\S+:[\w-]+:\S+:\S+:(?<functionname>[\S]+)$" nodrop
| parse field=resource "arn:aws:lambda:*:function:*" as f1, functionname2 nodrop
| if (isEmpty(functionname), functionname2, functionname) as functionname
| "aws/lambda" as namespace
| tolowercase(functionname) as functionname
| fields region, namespace, functionname, accountid
```
The FER `AwsObservabilityFieldExtractionRule` to extract fields `region`, `namespace`, `functionname`, and `accountid` will be created as a part of app installation.

### Centralized AWS CloudTrail Log Collection

Expand All @@ -268,16 +252,7 @@ Enter a parse expression to create an “account” field that maps to the alias

### Cloud Watch FER

```yml
Rule Name: AwsObservabilityLambdaCloudWatchLogsFER
Applied at: Ingest Time
Scope (Specific Data): account=* region* _sourceHost=/aws/lambda/*
Parse Expression:
| parse field=_sourceHost "/aws/lambda/*" as functionname
| tolowercase(functionname) as functionname
| "aws/lambda" as namespace
| fields functionname, namespace
```
The FER `AwsObservabilityLambdaCloudWatchLogsFER` to extract fields `functionname` and `namespace` will be created as a part of app installation.

## Installing the AWS Lambda App

Expand Down
32 changes: 2 additions & 30 deletions docs/integrations/amazon-aws/network-load-balancer.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,40 +58,12 @@ Namespace for AWS Network Load Balancer Service is AWS/NetworkELB.

## Field Extraction Rule(s)

Create a Field Extraction Rule for AWS Network Load Balancer Cloudtrail Logs. Learn how to create Field Extraction Rule [here](/docs/manage/field-extractions/create-field-extraction-rule).
The FER `AwsObservabilityNLBCloudTrailLogsFER` to extract fields `region`, `namespace`, `networkloadbalancer`, and `accountid` will be created as a part of app installation.

**AWS Network Load Balancer CloudTrail Logs**
```sql
Rule Name: AwsObservabilityNLBCloudTrailLogsFER
Applied at: Ingest Time
Scope (Specific Data): account=* eventSource eventName "elasticloadbalancing.amazonaws.com" "2015-12-01"
```

```sumo title="Parse Expression"
json "eventSource", "awsRegion", "recipientAccountId", "requestParameters.name", "requestParameters.type", "requestParameters.loadBalancerArn", "requestParameters.listenerArn", "apiVersion" as event_source, region, accountid, networkloadbalancer, loadbalancertype, loadbalancerarn, listenerarn, api_version nodrop
| where event_source = "elasticloadbalancing.amazonaws.com" and api_version matches "2015-12-01"
| "" as namespace
| parse field=loadbalancerarn ":loadbalancer/*/*/*" as balancertype1, networkloadbalancer1, f1 nodrop
| parse field=listenerarn ":listener/*/*/*/*" as balancertype2, networkloadbalancer2, f1, f2 nodrop
| if(loadbalancertype matches "network", "aws/networkelb", if(balancertype1 matches "net", "aws/networkelb", if(balancertype2 matches "net", "aws/networkelb", namespace))) as namespace
| if(loadbalancertype matches "application", "aws/applicationelb", if(balancertype1 matches "app", "aws/applicationelb", if(balancertype2 matches "app", "aws/applicationelb", namespace))) as namespace
| where namespace="aws/networkelb" or isEmpty(namespace)
| if (!isEmpty(networkloadbalancer), networkloadbalancer, if (!isEmpty(networkloadbalancer1), networkloadbalancer1, networkloadbalancer2)) as networkloadbalancer
| toLowerCase(networkloadbalancer) as networkloadbalancer
| fields region, namespace, networkloadbalancer, accountid
```

## Metric rules

Create the following Metric Rule for the AWS/NetworkELB namespace if not already created. Learn how to create a Metrics Rule [here](/docs/metrics/metric-rules-editor#create-a-metrics-rule).

```sql title="Rule 1*"
Rule name: AwsObservabilityNLBMetricsAddonEntityRule
Metric match expression: Namespace=AWS/NetworkELB LoadBalancer=*
Variable name: networkloadbalancer
Tag sequence: $LoadBalancer._1
Save it
```
The Metric Rule `AwsObservabilityNLBMetricsAddonEntityRule` for the AWS/NetworkELB namespace will be created as a part of app installation.

## Installing the AWS Network Load Balancer app

Expand Down
66 changes: 3 additions & 63 deletions docs/integrations/amazon-aws/rds.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,29 +432,7 @@ Sumo Logic supports several methods for collecting logs from Amazon CloudWatch.

### Field Extraction Rule(s)

Create a Field Extraction Rule for CloudTrail Logs. Learn how to create a Field Extraction Rule [here](/docs/manage/field-extractions/create-field-extraction-rule).

```sql
Rule Name: AwsObservabilityRdsCloudTrailLogsFER
Applied at: Ingest Time
Scope (Specific Data): account=* eventname eventsource "rds.amazonaws.com"
```

```sumo title="Parse Expression"
| json "eventSource", "awsRegion", "requestParameters", "responseElements", "recipientAccountId" as eventSource, region, requestParameters, responseElements, accountid nodrop
| where eventSource = "rds.amazonaws.com" | "aws/rds" as namespace
| json field=requestParameters "dBInstanceIdentifier", "resourceName", "dBClusterIdentifier", "dBProxyName" as dBInstanceIdentifier1, resourceName, dBClusterIdentifier1, dBProxyName1 nodrop
| json field=responseElements "dBInstanceIdentifier", "dBClusterIdentifier", "dBProxy.dBProxyName", "dBProxyTargetGroup.dBProxyName" as dBInstanceIdentifier3, dBClusterIdentifier3, dBProxyName2, dBProxyName3 nodrop
| parse field=resourceName "arn:aws:rds:*:db:*" as f1, dBInstanceIdentifier2 nodrop
| parse field=resourceName "arn:aws:rds:*:cluster:*" as f1, dBClusterIdentifier2 nodrop
| if (resourceName matches "arn:aws:rds:*:db:*", dBInstanceIdentifier2, if (!isEmpty(dBInstanceIdentifier1), dBInstanceIdentifier1, dBInstanceIdentifier3) ) as dBInstanceIdentifier
| if (resourceName matches "arn:aws:rds:*:cluster:*", dBClusterIdentifier2, if (!isEmpty(dBClusterIdentifier1), dBClusterIdentifier1, dBClusterIdentifier3) ) as dBClusterIdentifier
| if (isEmpty(dBInstanceIdentifier), dBClusterIdentifier, dBInstanceIdentifier) as dbidentifier
| tolowercase(dbidentifier) as dbidentifier
| if (!isEmpty(dBProxyName1), dBProxyName1, if (!isEmpty(dBProxyName2), dBProxyName2, dBProxyName3)) as proxyname
| tolowercase(proxyname) as proxyname
| fields region, namespace, dBInstanceIdentifier, dBClusterIdentifier, dbidentifier, proxyname, accountid
```
The FER `AwsObservabilityRdsCloudTrailLogsFER` to extract fields `region`, `namespace`, `dBInstanceIdentifier`, `dBClusterIdentifier`, `dbidentifier`, `proxyname`, and `accountid` will be created as a part of app installation.

### Centralized AWS CloudTrail log collection

Expand All @@ -481,49 +459,11 @@ Enter a parse expression to create an “account” field that maps to the alias

#### Create/Update Field Extraction Rule(s) for RDS CloudWatch logs


```
Rule Name: AwsObservabilityGenericCloudWatchLogsFER
Applied at: Ingest Time
Scope (Specific Data):
account=* region=* (_sourceHost=/aws/* or _sourceHost=API*Gateway*Execution*Logs*)
Parse Expression:
if (isEmpty(namespace),"unknown",namespace) as namespace
| if (_sourceHost matches "/aws/lambda/*", "aws/lambda", namespace) as namespace
| if (_sourceHost matches "/aws/rds/*", "aws/rds", namespace) as namespace
| if (_sourceHost matches "/aws/ecs/containerinsights/*", "aws/ecs", namespace) as namespace
| if (_sourceHost matches "/aws/kinesisfirehose/*", "aws/firehose", namespace) as namespace
| if (_sourceHost matches "/aws/apigateway/*", "aws/apigateway", namespace) as namespace
| if (_sourceHost matches "API-Gateway-Execution-Logs*", "aws/apigateway", namespace) as namespace
| parse field=_sourceHost "/aws/lambda/*" as functionname nodrop | tolowercase(functionname) as functionname
| parse field=_sourceHost "/aws/rds/proxy/*" as proxyname nodrop
| parse field=_sourceHost "/aws/rds/instance/*/" as dbidentifier nodrop
| parse field=_sourceHost "/aws/rds/cluster/*/" as dbidentifier nodrop
| parse field=_sourceHost "/aws/apigateway/*/*" as apiid, stage nodrop
| parse field=_sourceHost "API-Gateway-Execution-Logs_*/*" as apiid, stage nodrop | apiid as apiName
| tolowercase(dbidentifier) as dbidentifier
| fields namespace, functionname, proxyname, dbidentifier, apiid, apiName
```
The FER `AwsObservabilityGenericCloudWatchLogsFER` to extract fields `namespace`, `functionname`, `proxyname`, `dbidentifier`, `apiid`, and `apiName` will be created as a part of app installation.

### Metric Rules

Create the following two Metric Rules for the aws/rds namespace if not already created. Learn how to create a Metrics Rule [here](/docs/metrics/metric-rules-editor#create-a-metrics-rule).

```sql title="Rule 1"
Rule name: AwsObservabilityRDSClusterMetricsEntityRule
Metric match expression: Namespace=AWS/RDS DBClusterIdentifier=*
Variable name: dbidentifier
Tag sequence: $DBClusterIdentifier._1
Save it
```

```sql title="Rule 2"
Rule name: AwsObservabilityRDSInstanceMetricsEntityRule
Metric match expression: Namespace=AWS/RDS DBInstanceIdentifier=*
Variable name: dbidentifier
Tag sequence: $DBInstanceIdentifier._1
Save it
```
The Metric Rules `AwsObservabilityRDSClusterMetricsEntityRule` and `AwsObservabilityRDSInstanceMetricsEntityRule` for the aws/rds namespace will be created as a part of app installation.

## Installing the RDS app

Expand Down
28 changes: 1 addition & 27 deletions docs/integrations/amazon-aws/sns.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,33 +111,7 @@ account={{account}} region={{region}} namespace={{namespace}} TopicName={{topicn

### Field Extraction Rule(s)

Create a Field Extraction Rule for CloudTrail Logs. Learn how to create a Field Extraction Rule [here](/docs/manage/field-extractions/create-field-extraction-rule).

```sql
Rule Name: AwsObservabilitySNSCloudTrailLogsFER
Applied at: Ingest Time
Scope (Specific Data): account=* eventname eventsource \"sns.amazonaws.com\"
```

**Parse Expression**:

```sumo
| json "userIdentity", "eventSource", "eventName", "awsRegion", "recipientAccountId", "requestParameters", "responseElements" as userIdentity, event_source, event_name, region, recipient_account_id, requestParameters, responseElements nodrop
| where event_source = "sns.amazonaws.com"
| json field=userIdentity "accountId", "type", "arn", "userName" as accountid, type, arn, username nodrop
| parse field=arn ":assumed-role/*" as user nodrop
| parse field=arn "arn:aws:iam::*:*" as accountid, user nodrop
| json field=requestParameters "topicArn", "name", "resourceArn", "subscriptionArn" as req_topic_arn, req_topic_name, resource_arn, subscription_arn nodrop
| json field=responseElements "topicArn" as res_topic_arn nodrop
| if (isBlank(req_topic_arn), res_topic_arn, req_topic_arn) as topic_arn
| if (isBlank(topic_arn), resource_arn, topic_arn) as topic_arn
| parse field=topic_arn "arn:aws:sns:*:*:*" as region_temp, accountid_temp, topic_arn_name_temp nodrop
| parse field=subscription_arn "arn:aws:sns:*:*:*:*" as region_temp, accountid_temp, topic_arn_name_temp, arn_value_temp nodrop
| if (isBlank(req_topic_name), topic_arn_name_temp, req_topic_name) as topicname
| if (isBlank(accountid), recipient_account_id, accountid) as accountid
| "aws/sns" as namespace
| fields region, namespace, topicname, accountid
```
The FER `AwsObservabilitySNSCloudTrailLogsFER` to extract fields `region`, `namespace`, `topicname`, and `accountid` will be created as a part of app installation.

## Centralized AWS CloudTrail Log Collection
In case, you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following **Field Extraction Rule** to map a proper AWS account(s) friendly name/alias. Create it if not already present or update it as required.
Expand Down
20 changes: 1 addition & 19 deletions docs/integrations/amazon-aws/sqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,26 +133,8 @@ Sumo Logic supports collecting metrics using two source types:
1. If not present, create it. Learn how to create and manage fields [here](/docs/manage/fields/#manage-fields).

## Field Extraction Rule(s)
Create a Field Extraction Rule for CloudTrail Logs. Learn how to create a Field Extraction Rule [here](/docs/manage/field-extractions/create-field-extraction-rule).

* **Rule Name**: AwsObservabilitySQSCloudTrailLogsFER
* **Applied at**: Ingest Time
* **Scope (Specific Data)**: account=* eventname eventsource "sqs.amazonaws.com"
* **Parse Expression**:

```sumo
json "userIdentity", "eventSource", "eventName", "awsRegion", "recipientAccountId", "requestParameters", "responseElements", "sourceIPAddress" as userIdentity, event_source, event_name, region, recipient_account_id, requestParameters, responseElements, src_ip nodrop
| json field=userIdentity "accountId", "type", "arn", "userName" as accountid, type, arn, username nodrop
| json field=requestParameters "queueUrl" as queueUrlReq nodrop
| json field=responseElements "queueUrl" as queueUrlRes nodrop
| where event_source="sqs.amazonaws.com"
| if(event_name="CreateQueue", queueUrlRes, queueUrlReq) as queueUrl
| parse regex field=queueUrl "(?<queueName>[^\/]*$)"
| if (isBlank(recipient_account_id), accountid, recipient_account_id) as accountid
|! toLowerCase(queuename) as queuename
| "aws/sqs" as namespace
| fields region, namespace, queuename, accountid
```
The FER `AwsObservabilitySQSCloudTrailLogsFER` to extract fields `region`, `namespace`, `queuename`, and `accountid` will be created as a part of app installation.

## Centralized AWS CloudTrail Log Collection

Expand Down