STAN integration
Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability This functionality is generally available and ready for production usage. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability This functionality is generally available and ready for production usage. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Serverless Observability
Generally available
Serverless Security
Generally available
Stack
9.0+
Generally available since 9.0
| Version | 1.11.0 (View all) |
| Subscription level What's this? |
Basic |
| Developed by What's this? |
Elastic |
| Ingestion method(s) | API, File |
| Minimum Kibana version(s) | 9.0.0 8.13.0 |
This integration is used to collect logs and metrics from STAN servers. The integration collects metrics from STAN monitoring server APIs.
The STAN package is tested with Stan 0.15.1.
The log dataset collects the STAN logs.
Example
{
"@timestamp": "2024-06-18T10:41:22.515Z",
"agent": {
"ephemeral_id": "e925a178-f704-43e6-899e-b69b164425d8",
"id": "21652d23-59b4-4d65-a341-ede15f919642",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.13.0"
},
"data_stream": {
"dataset": "stan.log",
"namespace": "ep",
"type": "logs"
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "21652d23-59b4-4d65-a341-ede15f919642",
"snapshot": false,
"version": "8.13.0"
},
"event": {
"agent_id_status": "verified",
"created": "2024-06-18T10:41:36.382Z",
"dataset": "stan.log",
"ingested": "2024-06-18T10:41:38Z",
"kind": "event",
"type": [
"info"
]
},
"input": {
"type": "log"
},
"log": {
"file": {
"path": "/tmp/service_logs/stan.log"
},
"level": "info",
"offset": 0
},
"message": "STREAM: Starting nats-streaming-server[test-cluster] version 0.15.1",
"process": {
"pid": 7
},
"stan": {
"log": {
"msg": {}
}
},
"tags": [
"forwarded",
"stan-log"
]
}
ECS Field Reference
Please refer to the following document for detailed information on ECS fields.
Exported fields
| Field | Description | Type |
|---|---|---|
| @timestamp | Event timestamp. | date |
| data_stream.dataset | Data stream dataset. | constant_keyword |
| data_stream.namespace | Data stream namespace. | constant_keyword |
| data_stream.type | Data stream type. | constant_keyword |
| event.dataset | Event dataset | constant_keyword |
| event.module | Event module | constant_keyword |
| input.type | Type of Filebeat input. | keyword |
| log.offset | Offset of the entry in the log file. | long |
| stan.log.client.id | The id of the client | integer |
| stan.log.msg.bytes | Size of the payload in bytes | long |
| stan.log.msg.error.message | Details about the error occurred | text |
| stan.log.msg.max_messages | An optional number of messages to wait for before automatically unsubscribing | integer |
| stan.log.msg.queue_group | The queue group which subscriber will join | text |
| stan.log.msg.reply_to | The inbox subject on which the publisher is listening for responses | keyword |
| stan.log.msg.sid | The unique alphanumeric subscription ID of the subject | integer |
| stan.log.msg.subject | Subject name this message was received on | keyword |
| stan.log.msg.type | The protocol message type | keyword |
The default datasets are stats, channels, and subscriptions.
This is the stats dataset of the STAN package, in charge of retrieving generic
metrics from a STAN instance.
Example
{
"@timestamp": "2024-06-18T10:42:35.470Z",
"agent": {
"ephemeral_id": "b30d1b07-36f0-4e5f-9f80-56b6015fb855",
"id": "21652d23-59b4-4d65-a341-ede15f919642",
"name": "docker-fleet-agent",
"type": "metricbeat",
"version": "8.13.0"
},
"data_stream": {
"dataset": "stan.stats",
"namespace": "ep",
"type": "metrics"
},
"ecs": {
"version": "8.0.0"
},
"elastic_agent": {
"id": "21652d23-59b4-4d65-a341-ede15f919642",
"snapshot": false,
"version": "8.13.0"
},
"event": {
"agent_id_status": "verified",
"dataset": "stan.stats",
"duration": 1823495,
"ingested": "2024-06-18T10:42:47Z",
"module": "stan"
},
"host": {
"architecture": "x86_64",
"containerized": true,
"hostname": "docker-fleet-agent",
"id": "8259e024976a406e8a54cdbffeb84fec",
"ip": [
"192.168.252.5"
],
"mac": [
"02-42-C0-A8-FC-05"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "3.10.0-1160.102.1.el7.x86_64",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.6 LTS (Focal Fossa)"
}
},
"metricset": {
"name": "stats",
"period": 60000
},
"service": {
"address": "http://elastic-package-service-stan-1:8222/streaming/serverz",
"type": "stan"
},
"stan": {
"cluster": {
"id": "test-cluster"
},
"server": {
"id": "I9vNI3muOuNoem5vuoyo5z"
},
"stats": {
"bytes": 0,
"channels": 1,
"clients": 100,
"messages": 0,
"state": "STANDALONE",
"subscriptions": 100
}
}
}
ECS Field Reference
Please refer to the following document for detailed information on ECS fields.
Exported fields
| Field | Description | Type | Metric Type |
|---|---|---|---|
| @timestamp | Event timestamp. | date | |
| agent.id | Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id. | keyword | |
| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword | |
| cloud.availability_zone | Availability zone in which this host, resource, or service is located. | keyword | |
| cloud.instance.id | Instance ID of the host machine. | keyword | |
| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword | |
| cloud.region | Region in which this host, resource, or service is located. | keyword | |
| container.id | Unique container id. | keyword | |
| data_stream.dataset | Data stream dataset. | constant_keyword | |
| data_stream.namespace | Data stream namespace. | constant_keyword | |
| data_stream.type | Data stream type. | constant_keyword | |
| event.dataset | Event dataset | constant_keyword | |
| event.module | Event module | constant_keyword | |
| host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host. | keyword | |
| service.address | Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). | keyword | |
| stan.cluster.id | The cluster ID | keyword | |
| stan.server.id | The server ID | keyword | |
| stan.stats.bytes | Number of bytes consumed across all STAN queues | long | counter |
| stan.stats.channels | The number of STAN channels | integer | gauge |
| stan.stats.clients | The number of STAN clients | integer | gauge |
| stan.stats.messages | Number of messages across all STAN queues | long | counter |
| stan.stats.role | If clustered, role of this node in the cluster (Leader, Follower, Candidate) | keyword | |
| stan.stats.state | The cluster / streaming configuration state (STANDALONE, CLUSTERED) | keyword | |
| stan.stats.subscriptions | The number of STAN streaming subscriptions | integer | gauge |
This is the channels dataset of the STAN package, in charge of retrieving
metrics about channels from a STAN instance.
Example
{
"@timestamp": "2024-06-18T10:40:33.161Z",
"agent": {
"ephemeral_id": "2f2befc0-8f03-49d7-b5bf-8131709857b6",
"id": "21652d23-59b4-4d65-a341-ede15f919642",
"name": "docker-fleet-agent",
"type": "metricbeat",
"version": "8.13.0"
},
"data_stream": {
"dataset": "stan.channels",
"namespace": "ep",
"type": "metrics"
},
"ecs": {
"version": "8.0.0"
},
"elastic_agent": {
"id": "21652d23-59b4-4d65-a341-ede15f919642",
"snapshot": false,
"version": "8.13.0"
},
"event": {
"agent_id_status": "verified",
"dataset": "stan.channels",
"duration": 3556501,
"ingested": "2024-06-18T10:40:45Z",
"module": "stan"
},
"host": {
"architecture": "x86_64",
"containerized": true,
"hostname": "docker-fleet-agent",
"id": "8259e024976a406e8a54cdbffeb84fec",
"ip": [
"192.168.252.5"
],
"mac": [
"02-42-C0-A8-FC-05"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "3.10.0-1160.102.1.el7.x86_64",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.6 LTS (Focal Fossa)"
}
},
"metricset": {
"name": "channels",
"period": 60000
},
"service": {
"address": "http://elastic-package-service-stan-1:8222/streaming/channelsz?subs=1",
"type": "stan"
},
"stan": {
"channels": {
"bytes": 0,
"depth": 0,
"first_seq": 0,
"last_seq": 0,
"messages": 0,
"name": "foo"
},
"cluster": {
"id": "test-cluster"
},
"server": {
"id": "KJXiShxwxsi2oNAUH1Rlzn"
}
}
}
ECS Field Reference
Please refer to the following document for detailed information on ECS fields.
Exported fields
| Field | Description | Type | Metric Type |
|---|---|---|---|
| @timestamp | Event timestamp. | date | |
| agent.id | Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id. | keyword | |
| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword | |
| cloud.availability_zone | Availability zone in which this host, resource, or service is located. | keyword | |
| cloud.instance.id | Instance ID of the host machine. | keyword | |
| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword | |
| cloud.region | Region in which this host, resource, or service is located. | keyword | |
| container.id | Unique container id. | keyword | |
| data_stream.dataset | Data stream dataset. | constant_keyword | |
| data_stream.namespace | Data stream namespace. | constant_keyword | |
| data_stream.type | Data stream type. | constant_keyword | |
| event.dataset | Event dataset | constant_keyword | |
| event.module | Event module | constant_keyword | |
| host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host. | keyword | |
| service.address | Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). | keyword | |
| stan.channels.bytes | The number of STAN bytes in the channel | long | counter |
| stan.channels.depth | Queue depth based upon current sequence number and highest reported subscriber sequence number | long | gauge |
| stan.channels.first_seq | First sequence number stored in the channel. If first_seq > min([seq in subscriptions]) data loss has possibly occurred | long | |
| stan.channels.last_seq | Last sequence number stored in the channel | long | counter |
| stan.channels.messages | The number of STAN streaming messages | long | counter |
| stan.channels.name | The name of the STAN streaming channel | keyword | |
| stan.cluster.id | The cluster ID | keyword | |
| stan.server.id | The server ID | keyword |
This is the subscriptions dataset of the STAN package, in charge of retrieving
metrics about subscriptions from a STAN instance.
Example
{
"@timestamp": "2024-06-18T10:43:38.692Z",
"agent": {
"ephemeral_id": "871b52f0-2644-4638-811c-1b0befe0ee13",
"id": "21652d23-59b4-4d65-a341-ede15f919642",
"name": "docker-fleet-agent",
"type": "metricbeat",
"version": "8.13.0"
},
"data_stream": {
"dataset": "stan.subscriptions",
"namespace": "ep",
"type": "metrics"
},
"ecs": {
"version": "8.0.0"
},
"elastic_agent": {
"id": "21652d23-59b4-4d65-a341-ede15f919642",
"snapshot": false,
"version": "8.13.0"
},
"event": {
"agent_id_status": "verified",
"dataset": "stan.subscriptions",
"duration": 1947519984,
"ingested": "2024-06-18T10:43:50Z",
"module": "stan"
},
"host": {
"architecture": "x86_64",
"containerized": true,
"hostname": "docker-fleet-agent",
"id": "8259e024976a406e8a54cdbffeb84fec",
"ip": [
"192.168.252.5"
],
"mac": [
"02-42-C0-A8-FC-05"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "3.10.0-1160.102.1.el7.x86_64",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.6 LTS (Focal Fossa)"
}
},
"metricset": {
"name": "subscriptions",
"period": 60000
},
"service": {
"address": "http://elastic-package-service-stan-1:8222/streaming/channelsz?subs=1",
"type": "stan"
},
"stan": {
"cluster": {
"id": "test-cluster"
},
"server": {
"id": "SwiO2nzNV8CW27j45QPnFz"
},
"subscriptions": {
"channel": "foo",
"id": "benchmark-sub-23",
"last_sent": 0,
"offline": false,
"pending": 0,
"queue": "T",
"stalled": false
}
}
}
ECS Field Reference
Please refer to the following document for detailed information on ECS fields.
Exported fields
| Field | Description | Type | Metric Type |
|---|---|---|---|
| @timestamp | Event timestamp. | date | |
| agent.id | Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id. | keyword | |
| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword | |
| cloud.availability_zone | Availability zone in which this host, resource, or service is located. | keyword | |
| cloud.instance.id | Instance ID of the host machine. | keyword | |
| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword | |
| cloud.region | Region in which this host, resource, or service is located. | keyword | |
| container.id | Unique container id. | keyword | |
| data_stream.dataset | Data stream dataset. | constant_keyword | |
| data_stream.namespace | Data stream namespace. | constant_keyword | |
| data_stream.type | Data stream type. | constant_keyword | |
| event.dataset | Event dataset | constant_keyword | |
| event.module | Event module | constant_keyword | |
| host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host. | keyword | |
| service.address | Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). | keyword | |
| stan.cluster.id | The cluster ID | keyword | |
| stan.server.id | The server ID | keyword | |
| stan.subscriptions.channel | The name of the STAN channel the subscription is associated with | keyword | |
| stan.subscriptions.id | The name of the STAN channel subscription (client_id) | keyword | |
| stan.subscriptions.last_sent | Last known sequence number of the subscription that was acked | long | counter |
| stan.subscriptions.offline | Is the subscriber marked as offline? | boolean | |
| stan.subscriptions.pending | Number of pending messages from / to the subscriber | long | gauge |
| stan.subscriptions.queue | The name of the NATS queue that the STAN channel subscription is associated with, if any | keyword | |
| stan.subscriptions.stalled | Is the subscriber known to be stalled? | boolean |
This integration includes one or more Kibana dashboards that visualizes the data collected by the integration. The screenshots below illustrate how the ingested data is displayed.
Changelog
| Version | Details | Minimum Kibana version |
|---|---|---|
| 1.11.0 | Enhancement (View pull request) Allow @custom pipeline access to event.original without setting preserve_original_event. |
9.0.0 8.13.0 |
| 1.10.0 | Enhancement (View pull request) Update format to spec 3.0 |
9.0.0 8.13.0 |
| 1.9.0 | Enhancement (View pull request) Add support for Kibana 9.0.0. |
9.0.0 8.13.0 |
| 1.8.0 | Enhancement (View pull request) Allow the usage of deprecated log input and support for stack 9.0 |
8.13.0 |
| 1.7.0 | Enhancement (View pull request) ECS version updated to 8.11.0. Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template. |
8.13.0 |
| 1.6.0 | Enhancement (View pull request) Add global filter on data_stream.dataset to improve performance. |
8.10.2 |
| 1.5.0 | Enhancement (View pull request) Enable time series data for metrics data streams. This dramatically reduces storage for metrics and is expected to progressively improve query performance. For more details, see https://www.elastic.co/guide/en/elasticsearch/reference/current/tsds.html. |
8.10.2 |
| 1.4.2 | Enhancement (View pull request) Migrate Client IP Count Timeline visualization to lens. |
8.10.2 |
| 1.4.1 | Enhancement (View pull request) Add subscriptions data stream metric types mappings. |
8.10.2 |
| 1.4.0 | Enhancement (View pull request) Add subscriptions data stream dimensions mappings and ECS fields. |
8.10.2 |
| 1.3.9 | Enhancement (View pull request) Add stats data stream metric types mappings. |
8.10.2 |
| 1.3.8 | Enhancement (View pull request) Add stats data stream dimensions mappings and ECS fields. |
8.10.2 |
| 1.3.7 | Enhancement (View pull request) Add channels data stream metric types mappings. |
8.10.2 |
| 1.3.6 | Enhancement (View pull request) Add channels data stream dimensions mappings and ECS fields. |
8.10.2 |
| 1.3.5 | Enhancement (View pull request) Migrate Logs Overview dashboard visualizations to lens. |
8.10.2 |
| 1.3.4 | Enhancement (View pull request) Migrate Metrics Overview dashboard visualizations to lens. |
8.10.2 |
| 1.3.3 | Bug fix (View pull request) Fix reference error in Channel Overview dashboard. |
8.0.0 7.14.0 |
| 1.3.2 | Bug fix (View pull request) Add null check to the rename processor |
8.0.0 7.14.0 |
| 1.3.1 | Enhancement (View pull request) Added categories and/or subcategories. |
8.0.0 7.14.0 |
| 1.3.0 | Enhancement (View pull request) Update to ECS 8.0 |
8.0.0 7.14.0 |
| 1.2.0 | Enhancement (View pull request) Release stan package for v8.0.0 |
8.0.0 7.14.0 |
| 1.1.2 | Enhancement (View pull request) Uniform with guidelines |
— |
| 1.1.1 | Bug fix (View pull request) Fix logic that checks for the 'forwarded' tag |
— |
| 1.1.0 | Enhancement (View pull request) Update to ECS 1.12.0 |
7.14.0 |
| 1.0.0 | Enhancement (View pull request) Release Stan as GA |
— |
| 0.5.3 | Enhancement (View pull request) Convert to generated ECS fields |
— |
| 0.5.2 | Enhancement (View pull request) update to ECS 1.11.0 |
— |
| 0.5.1 | Enhancement (View pull request) Escape special characters in docs |
— |
| 0.5.0 | Enhancement (View pull request) Update integration description |
— |
| 0.4.0 | Enhancement (View pull request) Set "event.module" and "event.dataset" |
— |
| 0.3.0 | Enhancement (View pull request) update to ECS 1.10.0 and adding event.original options |
— |
| 0.2.0 | Bug fix (View pull request) Fix stack compatability |
7.13.0 |
| 0.1.3 | Enhancement (View pull request) update to ECS 1.9.0 |
7.11.0 |
| 0.1.2 | Bug fix (View pull request) Change kibana.version constraint to be more conservative. |
7.11.0 |
| 0.1.0 | Enhancement (View pull request) initial release |
7.11.0 |