--- title: Deploy Elastic Cloud Forwarder for AWS description: Deploy the Elastic Cloud Forwarder for AWS using CloudFormation CLI, AWS Console, or AWS Serverless Application Repository. url: https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws/deploy products: - Elastic Cloud Forwarder - Elastic Cloud Forwarder for AWS - Elastic Cloud Serverless - Elastic Observability applies_to: - Serverless Observability projects: Generally available - Elastic Cloud Hosted: Generally available - Self-managed Elastic deployments: Unavailable - Elastic Cloud Forwarder for AWS: Generally available since 1.0, Preview in 0.2 --- # Deploy Elastic Cloud Forwarder for AWS Elastic Cloud Forwarder for AWS can be deployed using any of the following methods: | Deployment method | Description | |---------------------------------------------|-----------------------------------------------------------------| | CloudFormation (AWS CLI) | Deploy using AWS CLI commands with CloudFormation templates. | | CloudFormation (AWS Console) | Deploy using the AWS Management Console UI. | | AWS Serverless Application Repository (SAR) | Deploy directly from the AWS Serverless Application Repository. | Each method achieves the same result and uses CloudFormation templates. Choose the method that best adapts to your workflow. ## Deploy using CloudFormation (AWS CLI) Use the AWS CLI to deploy the forwarder with CloudFormation templates. This method is ideal for automation and infrastructure-as-code workflows. ### Deployment examples The following examples show how to deploy Elastic Cloud Forwarder using AWS CloudFormation CLI. Copy and paste these commands after replacing the placeholder values with your actual configuration. - Use the `--template-url` flag to reference a template hosted on S3. - Use the `--region` flag to specify the AWS region where the CloudFormation stack will be deployed. The CloudFormation stack deployment region must match the region of your log sources. - To always use the most recent stable templates, use the `latest` path. For example, `v1/latest`. - To pin a specific version, replace `latest` with the desired version tag. For example, `v1/v{{version.edot-cf-aws}}`. Alternatively, if you have downloaded the template file, use the `--template-body file://` option with a local template file. - Elastic Cloud Forwarder for AWS: Preview since 1.8 This example deploys a single CloudFormation stack that collects VPC Flow logs and CloudTrail logs from two S3 buckets, and VPC Flow logs from a CloudWatch Log Group, as OTel-native logs. ```sh aws cloudformation create-stack \ --stack-name ecf \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/otel_logs-cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=S3Buckets,ParameterValue='"arn:aws:s3:::your-vpc-bucket,arn:aws:s3:::your-cloudtrail-bucket"' \ ParameterKey=CloudWatchLogGroups,ParameterValue='"arn:aws:logs:eu-central-1:123456789012:log-group:vpc-flow:*"' \ ParameterKey=LogTypes,ParameterValue='"vpcflow,cloudtrail"' ``` Wrap comma-separated list values in double quotes so the AWS CLI passes them as a single parameter value. Set `LogTypes` to `all` (the default) to forward every supported log type found in the configured sources. Provide at least one of `S3Buckets` or `CloudWatchLogGroups`. - Elastic Cloud Forwarder for AWS: Preview since 1.8 This example deploys a single CloudFormation stack in [ECS mode](/docs/reference/opentelemetry/edot-cloud-forwarder/aws#modes) that collects VPC Flow logs and GuardDuty findings from two S3 buckets and writes them to the data streams used by the Elastic AWS integration. ```sh aws cloudformation create-stack \ --stack-name ecf-ecs \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/ecs_logs-cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=S3Buckets,ParameterValue='"arn:aws:s3:::your-vpc-bucket,arn:aws:s3:::your-guardduty-bucket"' \ ParameterKey=LogTypes,ParameterValue='"vpcflow,guardduty"' \ ParameterKey=DataStreamNamespace,ParameterValue="default" ``` For GuardDuty buckets encrypted with customer-managed keys, also set `GuardDutyKMSKeyARNs`. Refer to [ECS mode parameters](/docs/reference/opentelemetry/edot-cloud-forwarder/aws/configure#ecs-mode-parameters) for the full parameter list. This example deploys a CloudFormation stack to collect CrowdStrike events from a configured Falcon Data Replicator(FDR) feed. Collected data is then written to the `logs-crowdstrike.fdr-default` data stream installed through [Elastic CrowdStrike Integration](https://www.elastic.co/docs/reference/integrations/crowdstrike). ```sh aws cloudformation create-stack \ --stack-name ecf-crowdstrike-fdr \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/crowdstrike_fdr_cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=FeedClientID,ParameterValue="" \ ParameterKey=FeedSecret,ParameterValue="" \ ParameterKey=FeedSQSURL,ParameterValue="" \ ParameterKey=FeedStorageRegion,ParameterValue="" ``` The `--capabilities CAPABILITY_NAMED_IAM` flag is required because this CloudFormation template creates AWS Identity and Access Management (IAM) resources. More specifically, it creates a named IAM role (`LambdaExecutionRole`) for the Lambda function. To acknowledge that AWS CloudFormation might create or modify IAM resources with custom names, you must specify the `CAPABILITY_NAMED_IAM` capability. The legacy single log type templates are deprecated in version 1.10.0 and will be removed in a future release. Existing stacks continue to work and are not affected. For new deployments, use the unified templates in the [deployment examples](#deployment-examples). The following examples use the legacy templates, where each stack handles a single log type from a single source. This example deploys a CloudFormation stack to collect VPC Flow logs stored in an S3 bucket. ```sh aws cloudformation create-stack \ --stack-name ecf-vpc \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/s3_logs-cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=SourceS3BucketARN,ParameterValue=your-s3-vpc-bucket-arn \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=EdotCloudForwarderS3LogsType,ParameterValue="vpcflow" ``` This example deploys a CloudFormation stack to collect ALB Access logs stored in an S3 bucket. ```sh aws cloudformation create-stack \ --stack-name ecf-alb \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/s3_logs-cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=SourceS3BucketARN,ParameterValue=your-s3-alb-bucket-arn \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=EdotCloudForwarderS3LogsType,ParameterValue="elbaccess" ``` This example deploys a CloudFormation stack to collect CloudTrail logs stored in an S3 bucket. ```sh aws cloudformation create-stack \ --stack-name ecf-cloudtrail \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/s3_logs-cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=SourceS3BucketARN,ParameterValue=your-cloudtrail-bucket-arn \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=EdotCloudForwarderS3LogsType,ParameterValue="cloudtrail" ``` This example deploys a CloudFormation stack to collect AWS WAF logs stored in an S3 bucket. ```sh aws cloudformation create-stack \ --stack-name ecf-waf \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/s3_logs-cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=SourceS3BucketARN,ParameterValue=arn:aws:s3:::aws-waf-logs-your-bucket-arn \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=EdotCloudForwarderS3LogsType,ParameterValue="waf" ``` Replace `aws-waf-logs-your-bucket-name` with your actual WAF logging bucket ARN. Remember that the bucket name must start with `aws-waf-logs-` as required by AWS WAF. This example deploys a CloudFormation stack to collect CloudWatch logs (VPC Flow Logs or CloudTrail) from a Log Group. ```sh aws cloudformation create-stack \ --stack-name ecf-cw-vpc \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/cloudwatch_logs-cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=SourceCloudWatchLogGroupARN,ParameterValue="arn:aws:logs:eu-central-1:123456789012:log-group:your-log-group:*" \ ParameterKey=EdotCloudForwarderCWLogType,ParameterValue="vpcflow" ``` Set `EdotCloudForwarderCWLogType` to `vpcflow` for VPC Flow Logs or `cloudtrail` for CloudTrail logs. The `SourceCloudWatchLogGroupARN` must include the trailing `:*`. - Elastic Cloud Forwarder for AWS: Preview since 1.5 This example deploys a CloudFormation stack in [ECS mode](/docs/reference/opentelemetry/edot-cloud-forwarder/aws#modes) to collect logs from an S3 bucket and write them to the data streams used by the Elastic AWS integration. ```sh aws cloudformation create-stack \ --stack-name ecf-vpc-ecs \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/s3_logs_ecs-cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=SourceS3BucketARN,ParameterValue=your-s3-vpc-bucket-arn \ ParameterKey=OTLPEndpoint,ParameterValue="" \ ParameterKey=ElasticAPIKey,ParameterValue="" \ ParameterKey=EdotCloudForwarderS3LogsType,ParameterValue="vpcflow" \ ParameterKey=DataStreamNamespace,ParameterValue="default" ``` `EdotCloudForwarderS3LogsType` accepts `vpcflow`, `elbaccess`, `cloudtrail`, `waf`, `guardduty`, `netskope_transaction`, or `netskope_alerts_events`. For GuardDuty buckets encrypted with a customer-managed key, also set `GuardDutyKMSKeyARN`. Refer to [Legacy template settings](/docs/reference/opentelemetry/edot-cloud-forwarder/aws/configure#legacy-template-settings) for details. ### Update an existing stack To update an existing CloudFormation stack while preserving some parameter values, follow these steps: Determine the name of the CloudFormation stack you want to modify. Use the following structure for your update command: - Include all required parameters. - Use `UsePreviousValue=true` for parameters that should remain unchanged. - Specify `ParameterValue=` for parameters that need to be updated. Run the command with the following parameters: ```sh aws cloudformation update-stack \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/.yaml \ --stack-name \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=Param1,UsePreviousValue=true \ ParameterKey=Param2,UsePreviousValue=true \ ParameterKey=Param3,UsePreviousValue=true \ ParameterKey=Param4,ParameterValue= ``` For example, to add a CloudTrail bucket and log type to the `ecf` stack while keeping other parameter values unchanged: ```sh aws cloudformation update-stack \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/otel_logs-cloudformation.yaml \ --stack-name ecf \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=OTLPEndpoint,UsePreviousValue=true \ ParameterKey=ElasticAPIKey,UsePreviousValue=true \ ParameterKey=CloudWatchLogGroups,UsePreviousValue=true \ ParameterKey=S3Buckets,ParameterValue='"arn:aws:s3:::your-vpc-bucket,arn:aws:s3:::your-cloudtrail-bucket"' \ ParameterKey=LogTypes,ParameterValue='"vpcflow,cloudtrail"' ``` The stack reconciles the S3 event notifications and subscription filters on update: sources removed from the lists are unwired, and new ones are wired automatically. For example, to increase reserved concurrency for the `ecf` stack while keeping other parameter values unchanged: ```sh aws cloudformation update-stack \ --template-url https://edot-cloud-forwarder.s3.amazonaws.com/v1/latest/cloudformation/otel_logs-cloudformation.yaml \ --stack-name ecf \ --capabilities CAPABILITY_NAMED_IAM \ --region eu-central-1 \ --parameters \ ParameterKey=OTLPEndpoint,UsePreviousValue=true \ ParameterKey=ElasticAPIKey,UsePreviousValue=true \ ParameterKey=S3Buckets,UsePreviousValue=true \ ParameterKey=CloudWatchLogGroups,UsePreviousValue=true \ ParameterKey=LogTypes,UsePreviousValue=true \ ParameterKey=ElasticCloudForwarderConcurrentExecutions,ParameterValue=100 ``` On the legacy templates the parameter is named `EdotCloudForwarderConcurrentExecutions`. After running the command, check the stack status in the AWS Management Console under **CloudFormation** → **Stacks**. Then, run this command to confirm the updated parameter values: ```sh aws cloudformation describe-stacks --stack-name ``` ## Deploy using CloudFormation (AWS Console) The AWS CloudFormation console supports a one-click flow that uses the **Launch Stack** button, or a manual flow where you paste the template URL. ### One-click deployment For the canonical one-click flow for each mode, refer to the [Quick start](/docs/reference/opentelemetry/edot-cloud-forwarder/aws#quick-start) on the product landing page. The **Launch Stack** buttons there cover OpenTelemetry mode, ECS mode, and CrowdStrike FDR. The CloudFormation stack deployment region must match the region of your log sources (S3 buckets or CloudWatch Log Groups). ### Manual deployment To manually specify the template, follow these steps: 1. Navigate to **CloudFormation** in the AWS Console. 2. Select **Create Stack** and choose **With new resources (standard)** to start a fresh deployment. 3. Select one of the following options under **Specify template**: - **Amazon S3 URL (Recommended)**: Paste the CloudFormation template URL from [CloudFormation templates](/docs/reference/opentelemetry/edot-cloud-forwarder/aws/configure#cloudformation-templates). - **Upload a template file**: Download the template from the S3 URL and upload it manually. 4. Select **Next** and configure all required parameters using the settings described in [Configure the template](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws/configure). 5. Select **Next** again and check **Acknowledge IAM capabilities**. This is required because the template creates named IAM roles with permissions to access the required resources. 6. Review your configuration and select **Submit** to deploy the stack. 7. Monitor the progress until the stack reaches the `CREATE_COMPLETE` state. ### Update an existing stack To modify parameters of an existing stack through the AWS Console: 1. Navigate to **CloudFormation** in the AWS Management Console. 2. Select the stack you want to update. 3. Click **Update stack** and select either **Make a direct update** or **Create a change set**. 4. Choose **Use existing template**. 5. Select **Next**. 6. Modify the parameter values as needed (refer to [Configure the template](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws/configure) for parameter descriptions). 7. Select **Next** and review your changes. 8. Select **Submit** to apply the updates. For a change set, select **Execute changeset** to apply it. 9. Monitor the stack update progress in the console. ## Deploy using AWS Serverless Application Repository In addition to deploying through CloudFormation templates, you can deploy Elastic Cloud Forwarder directly from the AWS Serverless Application Repository (SAR). ### Deploy from SAR To deploy from SAR, follow these steps: 1. Navigate to **AWS Serverless Application Repository** in the AWS Management Console. 2. Select **Available applications** and check the box **Show apps that create custom IAM roles or resource policies**. 3. Search for the appropriate application: - **OpenTelemetry mode**: Search for `elastic-cloud-forwarder-logs-otel`. - **ECS mode**: Search for `elastic-cloud-forwarder-logs-ecs`. Refer to [Modes](/docs/reference/opentelemetry/edot-cloud-forwarder/aws#modes) for context and to [ECS mode parameters](/docs/reference/opentelemetry/edot-cloud-forwarder/aws/configure#ecs-mode-parameters) for the additional template parameters. - **CrowdStrike FDR**: Search for `elastic-cloud-forwarder-fdr`. 4. Select the application. 5. **Configure the application settings**: Under **Application settings**, fill in the parameters described in the [Configure the template](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws/configure) section. Refer to that section for details on each parameter. 6. **Acknowledge IAM role creation**: At the bottom of the page, check the box to acknowledge that the application will create custom IAM roles. This is required for the forwarder to access your log source and send data to Elastic Observability. 7. Select **Deploy**. The deployment process will start, and a CloudFormation stack will be created with all the necessary resources. You can monitor the progress in the AWS CloudFormation console under **Stacks**. Elastic Cloud Forwarder for AWS: Deprecated since 1.10 The legacy single log type applications remain available but are deprecated and will be removed in a future release: `edot-cloud-forwarder-s3-logs` (S3, OpenTelemetry), `edot-cloud-forwarder-cloudwatch-logs` (CloudWatch, OpenTelemetry), and `cloud-forwarder-s3-logs-ecs` (S3, ECS). For new deployments, use the unified applications listed above. The same [deployment considerations](/docs/reference/opentelemetry/edot-cloud-forwarder/aws#deployment-considerations) apply to SAR deployments, including the requirement that the deployment region matches your log source region. ## CloudFormation stack resources The CloudFormation templates create a number of resources to process logs from the configured sources. ### Resources for the unified templates - Elastic Cloud Forwarder for AWS: Preview since 1.8 The unified templates (`otel_logs` and `ecs_logs`) create the following resources. S3 and CloudWatch wiring resources are only created when the corresponding source list is non-empty. | Resource name | Type | Description | |-----------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------| | `LambdaFunction` | `AWS::Lambda::Function` | Core Lambda function responsible for processing incoming logs. This is a key resource in the stack. | | `LambdaExecutionRole` | `AWS::IAM::Role` | IAM role granting permissions needed for the Lambda function to read source buckets and write to the failure bucket. | | `LambdaInvokeConfig` | `AWS::Lambda::EventInvokeConfig` | Configures error handling and invocation settings for the Lambda function, including the on-failure destination. | | `LambdaLogGroup` | `AWS::Logs::LogGroup` | CloudWatch log group storing logs for the main Lambda function. Useful for debugging and monitoring. | | `S3FailureBucket` | `AWS::S3::Bucket` | Bucket for storing failed invocations, preventing data loss. Only created when `S3FailureBucketARN` is not provided. | | `LambdaPermissionS3` | `AWS::Lambda::Permission` | Grants permission for S3 to invoke the Lambda function when new objects arrive. | | `S3NotificationManagerLambda`, `CustomS3NotificationManager` | `AWS::Lambda::Function`, `AWS::CloudFormation::CustomResource` | Helper that creates the S3 event notifications on every configured source bucket. | | `S3NotificationCleanupLambda`, `CustomS3NotificationCleanup` | `AWS::Lambda::Function`, `AWS::CloudFormation::CustomResource` | Helper that removes stale S3 event notifications when sources change or the stack is deleted. | | `LambdaPermissionCloudWatch` | `AWS::Lambda::Permission` | Grants permission for CloudWatch Logs to invoke the Lambda function. | | `SubscriptionFilterManagerLambda`, `CustomSubscriptionFilterManager` | `AWS::Lambda::Function`, `AWS::CloudFormation::CustomResource` | Helper that creates the subscription filters on every configured Log Group. | | `SubscriptionFilterCleanupLambda`, `CustomSubscriptionFilterCleanup` | `AWS::Lambda::Function`, `AWS::CloudFormation::CustomResource` | Helper that removes stale subscription filters when sources change or the stack is deleted. | | `S3NotificationManagerRole`, `S3NotificationCleanupRole`, `SubscriptionFilterManagerRole`, `SubscriptionFilterCleanupRole` and their policies | `AWS::IAM::Role`, `AWS::IAM::Policy` | IAM roles and policies granting the helper Lambda functions permission to manage S3 notifications and subscription filters on the configured sources. | The helper Lambda functions run only during stack create, update, and delete. They keep the S3 notifications and subscription filters in sync with the `S3Buckets` and `CloudWatchLogGroups` parameters. The unified stacks expose the following outputs: | Output | Description | |-------------------------------|-----------------------------------------------------------------------------| | `LambdaFunctionArn` | ARN of the forwarder Lambda function. | | `LambdaCloudWatchLogGroupArn` | ARN of the Lambda function's own log group. | | `S3FailureBucketARN` | ARN of the failure bucket in use, whether created by the stack or provided. | | `WiredS3Buckets` | The S3 buckets this stack wired to the forwarder. | | `WiredLogGroups` | The CloudWatch Log Groups this stack wired to the forwarder. | | `EnabledLogTypes` | The log types this stack is configured to forward. | ### Resources for CrowdStrike FDR This is a list of resources created by the stack when collecting CrowdStrike events from a Falcon Data Replicator (FDR) feed. | Resource name | Type | Description | |--------------------------|----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `LambdaExecutionRole` | `AWS::IAM::Role` | IAM role granting the Lambda function permissions to write to CloudWatch Logs, access the failure S3 bucket, and invoke itself to fan out FDR file processing. | | `LambdaFunction` | `AWS::Lambda::Function` | Core Lambda function that polls the FDR SQS queue on scheduler trigger, and processes SQS messages to extract CrowdStrike events. This is a key resource in the stack. | | `LambdaInvokeConfig` | `AWS::Lambda::EventInvokeConfig` | Configures error handling and invocation settings for the Lambda function, including the on-failure destination. | | `LambdaLogGroup` | `AWS::Logs::LogGroup` | CloudWatch log group storing execution logs for the Lambda function. Useful for debugging and monitoring. | | `SchedulerExecutionRole` | `AWS::IAM::Role` | IAM role assumed by EventBridge Scheduler to invoke the Lambda function on a schedule. | | `PollSchedule` | `AWS::Scheduler::Schedule` | EventBridge Scheduler that periodically triggers the Lambda function to poll the FDR SQS queue. | | `S3FailureBucketARN` | `AWS::S3::Bucket` | ARN of the bucket for storing failed Lambda invocations to prevent data loss, in the format `arn:aws:s3:::your-bucket-name`. | The `PollSchedule` (with `SchedulerExecutionRole`) drives the polling loop on a fixed interval, while `LambdaInvokeConfig` and the failure bucket, `S3FailureBucketARN`, capture unprocessed events to prevent data loss. ### Resources for S3 logs (legacy templates) - Elastic Cloud Forwarder for AWS: Deprecated since 1.10 This is a list of resources created by the legacy stacks when processing S3 logs. | Resource name | Type | Description | |-------------------------------------|---------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `CustomNotificationUpdater` | `AWS::CloudFormation::CustomResource` | Custom resource used to manage S3 event notifications dynamically. | | `LambdaExecutionRole` | `AWS::IAM::Role` | IAM role granting permissions needed for the Lambda function to interact with S3 and other AWS services. | | `LambdaFunction` | `AWS::Lambda::Function` | Core Lambda function responsible for processing incoming logs from S3. This is a key resource in the stack. | | `LambdaInvokeConfig` | `AWS::Lambda::EventInvokeConfig` | Configures error handling and invocation settings for the Lambda function. | | `LambdaLogGroup` | `AWS::Logs::LogGroup` | CloudWatch log group storing logs for the main Lambda function. Useful for debugging and monitoring. | | `LambdaPermissionS3Bucket` | `AWS::Lambda::Permission` | Grants permission for S3 to invoke the Lambda function when new logs arrive. | | `LambdaS3TriggerPolicy` | `AWS::IAM::Policy` | IAM policy allowing the Lambda function to process events triggered by S3. | | `NotificationUpdaterLambda` | `AWS::Lambda::Function` | Utility Lambda function handling S3 event notification updates dynamically. | | `NotificationUpdaterLambdaLogGroup` | `AWS::Logs::LogGroup` | CloudWatch log group storing logs for the `NotificationUpdaterLambda` function. | | `S3FailureBucketARN` | `AWS::S3::Bucket` | ARN of the bucket for storing failed invocations from the forwarder Lambda function, preventing data loss, in the format `arn:aws:s3:::your-bucket-name`. If not defined, the template creates a dedicated failure bucket automatically. | The main Lambda function, `LambdaFunction`, is the core component for processing S3 logs. S3 event notifications are handled dynamically using `CustomNotificationUpdater` and `NotificationUpdaterLambda`. CloudWatch logs ensure detailed monitoring of Lambda executions. IAM roles and permissions control access between S3 and Lambda functions, while `S3FailureBucketARN` prevents data loss by capturing unprocessed logs. ### Resources for CloudWatch Logs (legacy templates) - Elastic Cloud Forwarder for AWS: Deprecated since 1.10 This is a list of resources created by the legacy stacks when CloudWatch logs are the source. | Resource name | Type | Description | |-----------------------------------|----------------------------------|--------------------------------------------------------------------------------------------------------------------------| | `CloudWatchLogSubscriptionFilter` | `AWS::Logs::SubscriptionFilter` | Defines a filter that forwards logs from a CloudWatch Log Group to the Lambda function. Critical for log processing. | | `LambdaExecutionRole` | `AWS::IAM::Role` | IAM role granting necessary permissions for the Lambda function to interact with CloudWatch Logs and other AWS services. | | `LambdaFunction` | `AWS::Lambda::Function` | Core Lambda function responsible for processing incoming logs from CloudWatch. This is a key resource in the stack. | | `LambdaInvokeConfig` | `AWS::Lambda::EventInvokeConfig` | Configures event invocation settings, including error handling and retry behavior. | | `LambdaLogGroup` | `AWS::Logs::LogGroup` | CloudWatch log group that stores execution logs for the main Lambda function, aiding monitoring and debugging. | | `LambdaPermissionCloudWatch` | `AWS::Lambda::Permission` | Grants permission for CloudWatch Logs to invoke the Lambda function, enabling real-time log streaming. | | `S3FailureBucketARN` | `AWS::S3::Bucket` | ARN of the bucket for storing failed log events to prevent data loss, in the format `arn:aws:s3:::your-bucket-name`. | The CloudWatch Log Subscription Filter, `CloudWatchLogSubscriptionFilter`, ensures logs are correctly forwarded to the Lambda function. The Lambda function, `LambdaFunction`, serves as the core processing unit for CloudWatch logs. CloudWatch Log Groups help monitor execution performance and debug issues. IAM permissions (`LambdaExecutionRole`, `LambdaPermissionCloudWatch`) control interactions between CloudWatch and Lambda, while the failure bucket, `S3FailureBucketARN`, helps prevent data loss in case of processing errors. ## Remove a CloudFormation stack If you no longer need a deployed stack and want to clean up all associated resources, you can remove it using either the AWS CLI or the AWS Console. ### Important considerations Deleting a stack removes all AWS resources created by that stack. However: - If you allowed the stack to automatically create a dedicated S3 bucket for failed Lambda invocations, that bucket is not removed if it contains objects, because CloudFormation doesn't force-remove non-empty buckets. To remove the bucket entirely, you must empty it manually before deleting it. - If you specified an existing bucket through the `S3FailureBucketARN` parameter, that bucket is not removed because it is not managed by the stack. ### Remove using AWS CLI Use the following command to remove a stack: ```sh aws cloudformation delete-stack \ --stack-name \ --region ``` You can monitor the deletion progress through this command: ```sh aws cloudformation describe-stacks \ --stack-name \ --region ``` If the stack deletion fails and remains in a `DELETE_FAILED` state, you can retry the deletion with force mode: ```sh aws cloudformation delete-stack \ --stack-name \ --region \ --deletion-mode FORCE_DELETE_STACK ``` This forcibly removes the stack's resources, except any that cannot be removed, like the failure S3 bucket if it still contains objects. For a complete cleanup, empty the bucket manually before retrying deletion. The following command removes the `ecf` stack: ```sh aws cloudformation delete-stack \ --stack-name ecf \ --region eu-central-1 ``` Monitor the deletion progress: ```sh aws cloudformation describe-stacks \ --stack-name ecf \ --region eu-central-1 ``` ### Remove using AWS Console To remove a stack using the AWS Management Console: 1. Navigate to **CloudFormation** in the AWS Management Console. 2. Select the stack you want to remove from the list. 3. Click **Delete** at the top of the stack details page. 4. Monitor the deletion progress on the **Events** tab or wait until the stack disappears from the stack list (indicating deletion is complete). ## Next steps - [Configuration settings](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws/configure): Learn about all configuration options, including optional settings and sizing recommendations. - [Troubleshooting](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws/troubleshooting): Diagnose and resolve issues with log forwarding.