Type to search.

Variables and conditions in input configurations

Serverless

Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Serverless interfaces and procedures might differ from classic Elastic Stack deployments.

Stack

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

When running Elastic Agent in some environments, you might not know all the input configuration details up front. To solve this problem, the input configuration accepts variables and conditions that get evaluated at runtime using information from the running environment. Similar to autodiscovery, these capabilities allow you to apply configurations dynamically.

Let’s consider a unique agent policy that is deployed on two machines: a Linux machine named "linux-app" and a Windows machine named "winapp". Notice that the configuration has some variable references: ${host.name} and ${host.platform}:

YAML 
inputs:
 - id: unique-logfile-id
   type: logfile
   streams:
    - paths: /var/log/${host.name}/another.log
      condition: ${host.platform} == "linux"
    - path: c:/service/app.log
      condition: ${host.platform} == "windows"

At runtime, Elastic Agent resolves variables and evaluates the conditions based on values provided by the environment, generating two possible input configurations.

On the Windows machine:

YAML 
inputs:
 - id: unique-logfile-id
   type: logfile
   streams:
    - path: c:/service/app.log

On the Linux machine:

YAML 
inputs:
 - id: unique-logfile-id
   type: logfile
   streams:
    - paths: /var/log/linux-app/another.log

Using variable substitution along with conditions allows you to create concise, but flexible input configurations that adapt to their deployed environment.

The syntax for variable substitution is ${var}, where var is the name of a variable defined by a provider. A provider defines key/value pairs that are used for variable substitution and conditions.

Elastic Agent supports a variety of providers, such as host and local, that supply variables to Elastic Agent. For example, earlier you saw ${host.name} used to resolve the path to the host’s log file based on the {host.platform} value. Both of these values were provided by the host provider.

All providers are enabled by default when Elastic Agent starts. If a provider cannot be configured, its variables are ignored.

Refer to Providers for more detail.

The following agent policy uses a custom key named foo to resolve a value defined by a local provider:

YAML 
inputs:
 - id: unique-logfile-id
   type: logfile
   streams:
    - paths: /var/log/${foo}/another.log

providers:
  local:
    vars:
      foo: bar

The policy generated by this configuration looks like this:

YAML 
inputs:
 - id: unique-logfile-id
   type: logfile
   streams:
    - paths: /var/log/bar/another.log

When an input uses a variable substitution that is not present in the current key/value mappings being evaluated, the input is removed in the result.

For example, this agent policy uses an unknown key:

YAML 
inputs:
  - id: logfile-foo
    type: logfile
    path: "/var/log/foo"
  - id: logfile-unknown
    type: logfile
    path: "${ unknown.key }"

The policy generated by this configuration looks like this:

YAML 
inputs:
  - id: logfile-foo
    type: logfile
    path: "/var/log/foo"

Alternative variables and constants

Variable substitution can also define alternative variables or a constant.

To define a constant, use either ' or ". When a constant is reached during variable evaluation, any remaining variables are ignored, so a constant should be the last entry in the substitution.

To define alternatives, use | followed by the next variable or constant. The power comes from allowing the input to define the preference order of the substitution by chaining multiple variables together.

For example, the following agent policy chains together multiple variables to set the log path based on information provided by the running container environment. The constant /var/log/other is used to end of the path, which is common to both providers:

YAML 
inputs:
  - id: logfile-foo
    type: logfile
    path: "/var/log/foo"
  - id: logfile-container
    type: logfile
    path: "${docker.paths.log|kubernetes.container.paths.log|'/var/log/other'}"

Escaping variables

In some cases the ${var} syntax causes an issue with using a value where the actually wanted variable is ${var}. In this case double $$ can be provided for the variable.

The double $$ causes the variable to be ignored and the extra $ is removed from the beginning.

For example, the following agent policy uses the escaped variable so the actual value is used instead.

YAML 
inputs:
  - id: logfile-foo
    type: logfile
    path: "/var/log/foo"
    processors:
      - add_tags:
          tags: [$${development}]
          target: "environment"

The policy generated by this configuration looks like this:

YAML 
inputs:
  - id: logfile-foo
    type: logfile
    path: "/var/log/foo"
    processors:
      - add_tags:
          tags: [${development}]
          target: "environment"

A condition is a boolean expression that you can specify in your agent policy to control whether a configuration is applied to the running Elastic Agent. You can set a condition on inputs, streams, or even processors.

In this example, the input is applied if the host platform is Linux:

YAML 
inputs:
  - id: unique-logfile-id
    type: logfile
    streams:
      - paths:
         - /var/log/syslog
    condition: ${host.platform} == 'linux'

In this example, the stream is applied if the host platform is not Windows:

YAML 
inputs:
  - id: unique-system-metrics-id
    type: system/metrics
    streams:
      - metricsets:
          - load
        data_stream.dataset: system.cpu
        condition: ${host.platform} != 'windows'

In this example, the processor is applied if the host platform is not Windows:

YAML 
inputs:
  - id: unique-system-metrics-id
    type: system/metrics
    streams:
      - metricsets:
          - load
        data_stream.dataset: system.cpu
    processors:
      - add_fields:
          fields:
            platform: ${host.platform}
          target: host
        condition: ${host.platform} != 'windows'

The conditions supported by Elastic Agent are based on EQL's boolean syntax, but add support for variables from providers and functions to manipulate the values.

Supported operators:

  • Full PEMDAS math support for + - * / %.
  • Relational operators < <= >= > == !=
  • Logical operators and and or

Functions:

Types:

  • Booleans true and false

Run only when a specific label is included.

Eql 
arrayContains(${docker.labels}, 'monitor')

Skip on Linux platform or macOS.

Eql 
${host.platform} != "linux" and ${host.platform} != "darwin"

Run only for specific labels.

Eql 
arrayContains(${docker.labels}, 'monitor') or arrayContains(${docker.label}, 'production')

The condition syntax supports the following functions.

add(Number, Number) Number

Usage:

Eql 
add(1, 2) == 3
add(5, ${foo}) >= 5

arrayContains(Array, String) Boolean

Usage:

Eql 
arrayContains(${docker.labels}, 'monitor')

concat(String, String) String

Note

Parameters are coerced into strings before the concatenation.

Usage:

Eql 
concat("foo", "bar") == "foobar"
concat(${var1}, ${var2}) != "foobar"

divide(Number, Number) Number

Usage:

Eql 
divide(25, 5) > 0
divide(${var1}, ${var2}) > 7

endsWith(String, String) Boolean

Usage:

Eql 
endsWith("hello world", "hello") == true
endsWith(${var1}, "hello") != true

hasKey(Dictionary, String) Boolean

Usage:

Eql 
hasKey(${host}, "platform")

indexOf(String, String, Number?) Number

Note

Returns -1 if the string is not found.

Usage:

Eql 
indexOf("hello", "llo") == 2
indexOf(${var1}, "hello") >= 0

length(Array|Dictionary|string)

Usage:

Eql 
length("foobar") > 2
length(${docker.labels}) > 0
length(${host}) > 2

match(String, Regexp) boolean

Note

Regexp supports Go’s regular expression syntax. Conditions that use regular expressions are more expensive to run. If speed is critical, consider using endWiths or startsWith.

Usage:

Eql 
match("hello world", "^hello") == true
match(${var1}, "world$") == true

modulo(number, number) Number

Usage:

Eql 
modulo(25, 5) > 0
modulo(${var1}, ${var2}) == 0

multiply(Number, Number) Number

Usage:

Eql 
multiply(5, 5) == 25
multiple(${var1}, ${var2}) > x

number(String) Integer

Usage:

Eql 
number("42") == 42
number(${var1}) == 42

startsWith(String, String) Boolean

Usage:

Eql 
startsWith("hello world", "hello") == true
startsWith(${var1}, "hello") != true

string(Number) String

Usage:

Eql 
string(42) == "42"
string(${var1}) == "42"

stringContains(String, String) Boolean

Usage:

Eql 
stringContains("hello world", "hello") == true
stringContains(${var1}, "hello") != true

subtract(Number, Number) Number

Usage:

Eql 
subtract(5, 1) == 4
subtract(${foo}, 2) != 2

To debug configurations that include variable substitution and conditions, use the inspect command. This command shows the configuration that’s generated after variables are replaced and conditions are applied.

First run the Elastic Agent. For this example, we’ll use the following agent policy:

YAML 
outputs:
  default:
    type: elasticsearch
    hosts: [127.0.0.1:9200]
    apikey: <my-api-key>

providers:
  local_dynamic:
    items:
      - vars:
          key: value1
        processors:
          - add_fields:
              fields:
                custom: match1
              target: dynamic
      - vars:
          key: value2
        processors:
          - add_fields:
              fields:
                custom: match2
              target: dynamic
      - vars:
          key: value3
        processors:
          - add_fields:
              fields:
                custom: match3
              target: dynamic

inputs:
  - id: unique-logfile-id
    type: logfile
    enabled: true
    streams:
      - paths:
          - /var/log/${local_dynamic.key}

Then run elastic-agent inspect --variables to see the generated configuration. For example:

Shell 
$ ./elastic-agent inspect --variables
inputs:
- enabled: true
  id: unique-logfile-id-local_dynamic-0
  original_id: unique-logfile-id
  processors:
  - add_fields:
      fields:
        custom: match1
      target: dynamic
  streams:
  - paths:
    - /var/log/value1
  type: logfile
- enabled: true
  id: unique-logfile-id-local_dynamic-1
  original_id: unique-logfile-id
  processors:
  - add_fields:
      fields:
        custom: match2
      target: dynamic
  streams:
  - paths:
    - /var/log/value2
  type: logfile
- enabled: true
  id: unique-logfile-id-local_dynamic-2
  original_id: unique-logfile-id
  processors:
  - add_fields:
      fields:
        custom: match3
      target: dynamic
  streams:
  - paths:
    - /var/log/value3
  type: logfile
outputs:
  default:
    apikey: <my-api-key>
    hosts:
    - 127.0.0.1:9200
    type: elasticsearch
providers:
  local_dynamic:
    items:
    - processors:
      - add_fields:
          fields:
            custom: match1
          target: dynamic
      vars:
        key: value1
    - processors:
      - add_fields:
          fields:
            custom: match2
          target: dynamic
      vars:
        key: value2
    - processors:
      - add_fields:
          fields:
            custom: match3
          target: dynamic
      vars:
        key: value3

---