Standalone Elastic Agent configuration examples on Elastic Cloud on Kubernetes

This section contains manifests that illustrate common use cases, and can be your starting point in exploring Elastic Agent deployed with ECK. These manifests are self-contained and work out-of-the-box on any non-secured Kubernetes cluster. They all contain a three-node Elasticsearch cluster and a single Kibana instance. Add the corresponding integration package to Kibana to install the dashboards, visualizations and other assets for each of these examples as described in the Elastic Agent documentation.

Warning

The examples in this section are for illustration purposes only and should not be considered to be production-ready. Some of these examples use the node.store.allow_mmap: false setting which has performance implications and should be tuned for production workloads, as described in Virtual memory.

System integration

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/3.5/config/recipes/elastic-agent/system-integration.yaml

Deploys Elastic Agent as a DaemonSet in standalone mode with system integration enabled. Collects syslog logs, auth logs and system metrics (for CPU, I/O, filesystem, memory, network, process and others).

Kubernetes integration

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/3.5/config/recipes/elastic-agent/kubernetes-integration.yaml

Deploys Elastic Agent as a DaemonSet in standalone mode with Kubernetes integration enabled. Collects API server, Container, Event, Node, Pod, Volume and system metrics.

Multiple Elasticsearch clusters output

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/3.5/config/recipes/elastic-agent/multi-output.yaml

Deploys two Elasticsearch clusters and two Kibana instances together with single Elastic Agent DaemonSet in standalone mode with System integration enabled. System metrics are sent to the elasticsearch cluster. Elastic Agent monitoring data is sent to elasticsearch-mon cluster.

Storing local state in host path volume

Elastic Agent managed by ECK stores local state in a host path volume by default. This ensures that Integrations run by the agent can continue their work without duplicating work that has already been done after the Pod has been recreated for example because of a Pod configuration change. Multiple replicas of an agent, for example Fleet Servers, can not be deployed on the same underlying Kubernetes node as they would try to use the same host path. Consider the following options for managing local state storage:

If local state storage in hostPath volumes is not desired this can be turned off by configuring an emptyDir volume instead.
The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 8.16 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. If local state storage is still desired but running the Agent container as root is not allowed, then you can give the Agent container the CHOWN and SETPCAP capabilities. At startup, Elastic Agent changes the owner of its state directory to its own user and group.
If you use an Elastic Agent version before 8.16 or cannot grant additional capabilities to the Agent container, you can use a DaemonSet to adjust the permissions of the Agent local state on each node before Elastic Agent starts. This DaemonSet must run as root (runAsUser: 0) and might also require privileged: true. Also note the Kibana changes required to trust the Elasticsearch CA when running in fleet mode.

Full configuration examples exist in Running as a non-root user.

If you deploy Elastic Agent as a Deployment, keep in mind that two Pods on the same node cannot share the host path. Refer to Rolling updates and multiple replicas with host path volumes for more information.