Amazon Data Firehose
Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability This functionality is generally available and ready for production usage. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Serverless Observability
Generally available
Stack
Generally available
You can use the monitoring tab in the Firehose console to ensure there are incoming records and the delivery success rate is 100%. By default Firehose also logs to a Cloudwatch log group with the name /aws/kinesisfirehose/<delivery stream name>, which is automatically created when the delivery stream is created. Two log streams, DestinationDelivery and BackupDelivery, are created in this log group.
The backup settings in the delivery stream specify how failed delivery requests are handled. For more details on how to configure backups to S3, refer to Step 3: Specify the destination settings for your Firehose stream.
Firehose can automatically scale to handle very high throughput. If your Elastic deployment is not properly configured for the data volume coming from Firehose, it could cause a bottleneck, which may lead to increased ingest times or indexing failures.
There are several facets to optimizing the underlying Elasticsearch performance, but Elastic Cloud provides several ready-to-use hardware profiles which can provide a good starting point. Other factors which can impact performance are shard sizing, indexing configuration, and index lifecycle management (ILM).
If you encounter further problems, contact us.