GCP Cloud Storage OpenTelemetry Assets
Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability We plan to add this functionality in a future Elastic Stack update. Subject to changes. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Serverless Observability
Preview
Preview
Serverless Security
Preview
Preview
Stack
Planned
Planned
| Version | 0.1.0 Availability Preview This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply. (View all) |
| Subscription level What's this? |
Basic |
| Developed by What's this? |
Elastic |
| Minimum Kibana version(s) | 9.6.0 |
To use pre-release integrations, go to the Integrations page in Kibana, scroll down, and toggle on the Display beta integrations option.
This package contains Kibana assets for monitoring Cloud Storage buckets with Google Cloud Monitoring metrics collected by the OpenTelemetry Collector.
The package is content only. It does not configure data collection. Use the Google Cloud Monitoring OpenTelemetry Input package (googlecloudmonitor_input_otel) to collect Cloud Storage metrics into Elasticsearch.
You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware.
Install the Google Cloud Monitoring OpenTelemetry Input package (googlecloudmonitor_input_otel) and configure it to collect Cloud Storage Cloud Monitoring metrics. This content package provides assets that visualize data collected by that input.
When configuring the input package, set the dataset name to gcp.cloudstorage.otel so that data is written to the metrics-gcp.cloudstorage.otel-default data stream, which these dashboards query.
The dashboards visualize the following Cloud Monitoring metric types:
| Metric type | Used for |
|---|---|
storage.googleapis.com/api/request_count |
Request volume, error rate, and fault attribution by method and response code. |
storage.googleapis.com/storage/v2/total_bytes |
Stored bytes by bucket, location, and storage class. |
storage.googleapis.com/storage/v2/total_count |
Stored object count by bucket and storage class. |
storage.googleapis.com/storage/total_byte_seconds |
Storage consumption over time, used for cost attribution. |
storage.googleapis.com/storage/total_bytes |
Stored bytes reported by the earlier metric, kept for buckets that still emit it. |
storage.googleapis.com/storage/object_count |
Object count reported by the earlier metric, kept for buckets that still emit it. |
storage.googleapis.com/authz/acl_based_object_access_count |
Object reads authorized by ACLs rather than IAM. |
storage.googleapis.com/authz/acl_operations_count |
ACL operations, which indicate buckets not yet using uniform bucket-level access. |
| Dashboard | Description |
|---|---|
| [GCP OTel] Cloud Storage Overview | Estate-wide health for Cloud Storage: request volume and error rate, whether failures are client-side or Google-side, a bucket hex map for spotting the worst offenders, and stored bytes, object counts, and storage consumption broken down by storage class and object type. |
| [GCP OTel] Cloud Storage Bucket Detail | Per-bucket deep-dive into request volume by fault class, method, and response code, availability, stored bytes and object counts by storage class, storage consumption, and ACL-based access that indicates buckets not yet using uniform bucket-level access. |
Open [GCP OTel] Cloud Storage Overview and click a bucket in the bucket hex map or the bucket ranking table to drill into [GCP OTel] Cloud Storage Bucket Detail for that bucket.
This integration includes one or more Kibana dashboards that visualizes the data collected by the integration. The screenshots below illustrate how the ingested data is displayed.
Changelog
| Version | Details | Minimum Kibana version |
|---|---|---|
| 0.1.0 | Enhancement (View pull request) Initial draft of the GCP Cloud Storage OpenTelemetry Assets package. |
9.6.0 |