Type to search.

Configure a RHEL host

ECE

Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

Red Hat Enterprise Linux 8, 9, and 10, along with Rocky Linux 8 and 9, run Elastic Cloud Enterprise (ECE) on Podman rather than Docker. Use these steps to install and configure Podman, and to apply the SELinux, XFS, and kernel tuning that ECE expects on RHEL-compatible distributions.

  • Follow your internal guidelines to create a RHEL 8 (the version must be >= 8.5), RHEL 9, RHEL 10, Rocky Linux 8, or Rocky Linux 9 server or VM in your environment.

    The following Podman versions and network backends are supported:

    • RHEL 8 and Rocky Linux 8: Podman 4 with CNI
    • RHEL 9: Podman 4 or Podman 5 with CNI
    • Rocky Linux 9: Podman 5 with CNI
    • Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability Generally available since 4.2 This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. RHEL 10: Podman 5 with Netavark
  • Verify that required traffic is allowed. Check the Networking prerequisites for a list of ports that need to be open. The technical configuration depends on the underlying infrastructure. For example, for AWS, allowing traffic between hosts is implemented using security groups.

  • Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability Generally available since 4.2 This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. If you need IPv6 egress from ECE containers, ensure the host has working dual-stack (IPv4 and IPv6) connectivity, and complete the optional Podman dual-stack steps later in this guide.

Important

Make sure to use a supported combination of Linux distribution and container engine version as defined in our official Support matrix. Unsupported combinations can lead to various issues in your ECE environment, including failures when creating system deployments, upgrading workload deployments, proxy timeouts, and more.

FIPS (Federal Information Processing Standards) compliance is not officially supported in ECE. While ECE may function on FIPS-enabled systems, this configuration has not been validated through our testing processes and is not recommended for production environments.

  1. Install the OS packages lvm2, iptables, sysstat, and net-tools by executing:

    Shell
    sudo dnf install lvm2 iptables sysstat net-tools
    
    1. The ECE diagnostic script requires net-tools.
    Note

    For RHEL 9 and Rocky Linux 9, also install the containernetworking-plugins package using:

    Shell
    sudo dnf -y install containernetworking-plugins
    
  2. Remove Docker and previously installed Podman packages (if previously installed).

    Shell
    sudo dnf remove docker docker-ce podman podman-remote containerd.io
    
  3. As a sudoers user, edit the /etc/selinux/config file:

    1. If you are not using SELinux, set it to permissive mode:

      Text
      SELINUX=permissive
      
    2. If you are using SELinux, set it to enforcing mode:

      Note

      Avoid customizing the host Docker path /mnt/data/docker when using SELinux. Otherwise the ECE installer script needs to be adjusted.

      Text
      SELINUX=enforcing
      
  4. Install Podman:

    Note Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability Generally available since 4.2 This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

    RHEL 10 supports Podman 5 only. Use the Podman 5 steps in this section. Podman 4 is not a supported combination on RHEL 10.

    • For Podman 4:

      • Install the latest available version 4.* using dnf.

        Shell
        sudo dnf install podman-4.* podman-remote-4.*
        
      • To prevent automatic Podman major version updates, configure the Podman version to be locked at version 4.* while still allowing minor and patch updates.

        Shell
        ## Install versionlock
        sudo dnf install 'dnf-command(versionlock)'
        
        ## Lock major version
        sudo dnf versionlock add --raw 'podman-4.*'
        sudo dnf versionlock add --raw 'podman-remote-4.*'
        
        ## Verify that podman-4.* and podman-remote-4.* appear in the output
        sudo dnf versionlock list
        
    • For Podman 5:

      • Install the latest available version 5.* using dnf.

        Note

        Podman versions 5.2.2-11 and 5.2.2-13 are affected by a known memory leak issue. To avoid this bug, use a later version. Refer to the official Support matrix for more information.

        Shell
        sudo dnf install podman-5.* podman-remote-5.*
        
      • To prevent automatic Podman major version updates, configure the Podman version to be locked at version 5.* while still allowing minor and patch updates.

        Shell
        ## Install versionlock
        sudo dnf install 'dnf-command(versionlock)'
        
        ## Lock major version
        sudo dnf versionlock add --raw 'podman-5.*'
        sudo dnf versionlock add --raw 'podman-remote-5.*'
        
        ## Verify that podman-5.* and podman-remote-5.* appear in the output
        sudo dnf versionlock list
        
  5. Configure the Podman network backend. The required backend depends on the OS version:

    • For RHEL 8 and Rocky Linux 8: Ensure that Podman uses the CNI network backend. Although CNI is the default, set network_backend="cni" explicitly so the host does not depend on the existing configuration.

      1. If the /etc/containers/containers.conf file does not exist, copy the /usr/share/containers/containers.conf file to the /etc/containers/ directory (for example, using cp /usr/share/containers/containers.conf /etc/containers/).

      2. Open the /etc/containers/containers.conf file. Navigate to the network section and make sure that the network_backend setting is set to cni.

      3. Reboot the system (reboot).

      4. Check that the network stack is cni:

        Shell
        cat /etc/containers/containers.conf
        [...]
        [network]
        network_backend="cni"
        [...]
        
    • For RHEL 9 and Rocky Linux 9 only: Switch the network stack from Netavark to CNI.

      1. If the /etc/containers/containers.conf file does not exist, copy the /usr/share/containers/containers.conf file to the /etc/containers/ directory (for example, using cp /usr/share/containers/containers.conf /etc/containers/).

      2. Open the /etc/containers/containers.conf file. Navigate to the network section and make sure that the network_backend setting is set to cni.

      3. Reboot the system (reboot).

      4. Check that the network stack has changed to cni:

        Shell
        cat /etc/containers/containers.conf
        [...]
        [network]
        network_backend="cni"
        [...]
        
    • Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability Generally available since 4.2 This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. For RHEL 10: Keep the default Netavark backend. Do not install containernetworking-plugins and do not set network_backend=cni.

      1. If the /etc/containers/containers.conf file does not exist, copy the /usr/share/containers/containers.conf file to the /etc/containers/ directory (for example, using cp /usr/share/containers/containers.conf /etc/containers/).
      2. If the file contains a network_backend setting (for example a leftover cni pin), remove that line so Podman uses the RHEL 10 default (Netavark).
  6. If Podman requires a proxy in your infrastructure setup, modify the /usr/share/containers/containers.conf file and add the HTTP_PROXY and HTTPS_PROXY environment variables in the [engine] section. Note that multiple env variables in that configuration file exists — use the one in the [engine] section.

    Example:

    Text
    [engine]
    env = ["HTTP_PROXY=http://<PROXY_IP>:<PROXY_PORT>", "HTTPS_PROXY=http://<PROXY_IP>:<PROXY_PORT>"]
    
  7. Reload systemd configuration:

    Shell
    sudo systemctl daemon-reload
    
  8. Create OS groups, if they do not exist yet:

    Reference: Users and permissions

    Shell
    sudo groupadd elastic
    sudo groupadd podman
    
  9. Add user elastic to the podman group:

    Reference: Users and permissions

    Shell
    sudo useradd -g "elastic" -G "podman" elastic
    
  10. As a sudoers user, add the following line to /etc/sudoers.d/99-ece-users:

    Reference: Users and permissions

    Text
    elastic ALL=(ALL) NOPASSWD:ALL
    
  11. Add the required options to the kernel boot arguments:

    Shell
    sudo /sbin/grubby --update-kernel=ALL --args='cgroup_enable=memory cgroup.memory=nokmem swapaccount=1'
    
  12. Create the directory:

    Shell
    sudo mkdir -p /etc/systemd/system/podman.socket.d
    
  13. As a sudoers user, create the file /etc/systemd/system/podman.socket.d/podman.conf with the following content. Set the correct ownership and permission.

    Important

    Both ListenStream= and ListenStream=/var/run/docker.sock parameters are required!

    File content:

    Text
    [Socket]
    ListenStream=
    ListenStream=/var/run/docker.sock
    SocketMode=770
    SocketUser=elastic
    SocketGroup=podman
    

    File ownership and permission:

    Shell
    sudo chown root:root /etc/systemd/system/podman.socket.d/podman.conf
    sudo chmod 0644 /etc/systemd/system/podman.socket.d/podman.conf
    
  14. As a sudoers user, create the (text) file /usr/bin/docker with the following content. Verify that the regular double quotes in the text file are used (ASCII code Hex 22):

    Text
    #!/bin/bash
    podman-remote --url unix:///var/run/docker.sock "$@"
    
  15. Set the file permissions on /usr/bin/docker:

    Shell
    sudo chmod 0755 /usr/bin/docker
    
  16. As a sudoers user, add the following two lines to section [storage] in the file /etc/containers/storage.conf. Verify that those parameters are only defined once. Either remove or comment out potentially existing parameters.

    Note

    Avoid customizing the host Docker path /mnt/data/docker when using SELinux. Otherwise the ECE installer script needs to be adjusted.

    Text
    runroot = "/mnt/data/docker/runroot/"
    graphroot = "/mnt/data/docker"
    
  17. Enable Podman so that itself and running containers start automatically after a reboot:

    Shell
    sudo systemctl enable podman.service
    sudo systemctl enable podman-restart.service
    
  18. Enable the overlay kernel module (check Use the OverlayFS storage driver) that the Podman overlay storage driver uses (check Working with the Container Storage library and tools in Red Hat Enterprise Linux).

    In the Docker world there are two overlay drivers, overlay and overlay2. Today most users use the overlay2 driver, so we just use that one, and called it overlay. Refer also to Use the OverlayFS storage driver.

    Shell
    echo "overlay" | sudo tee -a /etc/modules-load.d/overlay.conf
    
  19. Format the additional data partition:

    Shell
    sudo mkfs.xfs /dev/nvme1n1
    
  20. Create the /mnt/data/ directory used as a mount point:

    Shell
    sudo install -o elastic -g elastic -d -m 700 /mnt/data
    
  21. As a sudoers user, modify the entry for the XFS volume in the /etc/fstab file to add pquota,prjquota. The default filesystem path used by Elastic Cloud Enterprise is /mnt/data.

    Note

    Replace /dev/nvme1n1 in the following example with the corresponding device on your host, and add this example configuration as a single line to /etc/fstab.

    Text
    /dev/nvme1n1	/mnt/data	xfs	defaults,nofail,x-systemd.automount,prjquota,pquota  0 2
    
  22. Restart the local-fs target:

    Shell
    sudo systemctl daemon-reload
    sudo systemctl restart local-fs.target
    
  23. Set the permissions on the newly mounted device:

    Shell
    ls /mnt/data
    sudo chown elastic:elastic /mnt/data
    
  24. Create the /mnt/data/docker directory for the Docker service storage:

    Note

    Avoid customizing the host Docker path /mnt/data/docker when using SELinux. Otherwise the ECE installer script needs to be adjusted.

    Shell
    sudo install -o elastic -g elastic -d -m 700 /mnt/data/docker
    
  25. If you want to use FirewallD, ensure you meet the networking prerequisites. Otherwise, you can disable it with:

    Shell
    sudo systemctl disable firewalld
    
    Note

    If FirewallD does not exist on your VM, you can skip this step.

  26. Configure kernel parameters:

    Shell
    cat <<EOF | sudo tee -a /etc/sysctl.conf
    vm.max_map_count=1048576
    net.ipv4.ip_forward=1
    net.ipv4.tcp_retries2=5
    net.netfilter.nf_conntrack_tcp_timeout_established=7200
    net.netfilter.nf_conntrack_max=262140
    vm.swappiness=1
    EOF
    
    1. This setting is required by Elasticsearch.
    2. Enable forwarding so the Docker networking works as expected.
    3. Decrease the maximum number of TCP retransmissions to 5 as recommended for Elasticsearch TCP retransmission timeout. Learn more.
    4. Set to 1 to prevent the host from swapping too early.
    Note Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability Generally available since 4.2 This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

    If you need IPv6 egress from containers, also add net.ipv6.conf.all.forwarding=1 to the same sysctl configuration.

    Note

    According to Elasticsearch networking settings, Elasticsearch overrides TCP keepalive settings at the socket level for its own connections:

    • If system-level values exceed 300 seconds, Elasticsearch automatically lowers them to 300 seconds.
    • Values below 300 seconds are used as-is.

    For non-Elasticsearch connections such as the proxy layer, consider reducing the following TCP keepalive parameters to detect stale network sessions and prevent firewalls from dropping silent connections:

    • net.ipv4.tcp_keepalive_time
    • net.ipv4.tcp_keepalive_intvl
    • net.ipv4.tcp_keepalive_probes
  27. Apply the new sysctl settings:

    Shell
    sudo sysctl -p
    sudo systemctl restart NetworkManager
    
  28. As a sudoers user, adjust the system limits. Add the following configuration values to the /etc/security/limits.conf file. These settings apply to host-level processes and interactive user sessions (for example, SSH).

    Text
    *                soft    nofile         1024000
    *                hard    nofile         1024000
    *                soft    memlock        unlimited
    *                hard    memlock        unlimited
    elastic          soft    nofile         1024000
    elastic          hard    nofile         1024000
    elastic          soft    memlock        unlimited
    elastic          hard    memlock        unlimited
    elastic          soft    nproc          unlimited
    elastic          hard    nproc          unlimited
    root             soft    nofile         1024000
    root             hard    nofile         1024000
    root             soft    memlock        unlimited
    
    Important

    The /etc/security/limits.conf settings are PAM-based and do not apply to processes running inside Podman containers. To ensure the same limits are enforced inside containers, you must also configure the Podman default ulimits as described in the next step.

  29. Configure the default container ulimits. Open the /etc/containers/containers.conf file and add the following under the [containers] section. If the file does not exist, copy it from /usr/share/containers/containers.conf first. These settings ensure that all containers created by Podman inherit the correct resource limits.

    Text
    [containers]
    default_ulimits = [
      "nofile=1024000:1024000",
      "memlock=-1:-1",
      "nproc=-1:-1",
    ]
    
    Note

    If the [containers] section already exists in the file, merge the default_ulimits setting into it rather than creating a duplicate section.

  30. Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability Generally available since 4.2 This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Optional: Enable dual-stack networking for IPv6 egress. Complete these steps only if ECE containers must reach IPv6 endpoints. Podman does not support configuring both IPv4 and IPv6 on the built-in default network through containers.conf, so create a dual-stack network and set it as the default for new containers.

    1. Create a dual-stack Podman network:

      Shell
      sudo podman network create \
        --subnet 10.89.0.0/24 \
        --subnet fd00:10:89::/64 \
        --ipv6 \
        ece-network
      
      Note

      Choose IPv4 and IPv6 subnets that do not overlap with other networks in your environment. These values are local to each host, so the same subnets can be reused across ECE hosts.

    2. Set the dual-stack network as the default for new containers. Open /etc/containers/containers.conf and, in the [network] section, set default_network. If the file or section does not exist yet, create it. Make sure not to create a duplicate [network] section if one already exists.

      Text
      [network]
      default_network = "ece-network"
      
  31. Restart the Podman service:

    Shell
    sudo systemctl daemon-reload
    sudo systemctl restart podman
    
  32. Reboot the RHEL host:

    Shell
    sudo reboot
    
  33. After rebooting, verify the host configuration.

    1. Confirm that Podman is running:

      Shell
      sudo systemctl status podman
      
    2. Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability Generally available since 4.2 This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Optional: If you enabled dual-stack networking for IPv6 egress, verify both the default network configuration and outbound IPv6 connectivity from a container:

      1. Confirm that the dual-stack network has both subnets:

        Shell
        sudo podman network inspect ece-network --format '{{json .Subnets}}'
        

        The output should include both the IPv4 (10.89.0.0/24) and IPv6 (fd00:10:89::/64) subnets.

      2. Run a short-lived container and test IPv6 egress:

        Shell
        sudo podman run --rm curlimages/curl:latest \
          -6 -s -o /dev/null -w "%{http_code}\n" https://ipv6.google.com
        

        A response of 200 confirms that containers can reach IPv6 endpoints.

Repeat these host configuration steps for every RHEL or Rocky Linux host that you want to use with Elastic Cloud Enterprise.

After configuring and verifying each host, continue to Installation of ECE using Podman to install Elastic Cloud Enterprise.