Update ingest stream settings
Base URL
https://{kibana_url}
Auth
Stack
Experimental
9.1+
The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.
Availability
Experimental since 9.1
This functionality is experimental and is not ready for production usage. Experimental features may change or be removed at any time. Elastic will work to fix any issues, but experimental features are not subject to the support SLA of official GA features. Specific Support terms apply.
Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.
This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Upserts the ingest settings of an ingest stream definition
Expand all
space_id
string optional
An identifier for the space. If space_id is not provided in the URL, the default space is used. Refer to Spaces for more information.
name
string · max length: 200 required
The name of the stream.
A required header to protect against CSRF attacks
{
}
ingest
union anyOf required
Any of:
−
Hide 5 properties
failure_store
union anyOf required
Any of:
{ inherit }
{ disabled }
object
object
−
Hide 1 property
lifecycle
object required
−
Hide 1 property
enabled
object required
−
Hide 1 property
data_retention
string · min length: 1
A non-empty string.
−
Hide 1 property
lifecycle
object required
−
Hide 1 property
lifecycle
union anyOf required
Any of:
−
Hide 1 property
dsl
object required
−
Hide 3 properties
data_retention
string · min length: 1
A non-empty string.
downsample
object []
+
Show 2 properties
after
string · min length: 1 required
A non-empty string.
fixed_interval
string · min length: 1 required
A non-empty string.
frozen_after
string · min length: 1
A non-empty string.
−
Hide 1 property
ilm
object required
−
Hide 1 property
policy
string · min length: 1 required
A non-empty string.
processing
object required
−
Hide 1 property
steps
union anyOf [] required
Any of:
Grok processor - Extract fields from text using grok patterns
−
Hide 9 properties
action
enum required
Values:
grok
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to parse with grok patterns
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
pattern_definitions
map string to string
patterns
string [] · min items: 1 required
Grok patterns applied in order to extract fields
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Any of:
FilterCondition
{ and }
{ or }
{ not }
{ never }
{ always }
A basic filter condition, either unary or binary.
−
Hide 2 properties
Any of:
contains, endsWith, eq, …
exists
A condition that compares a field to a value or range using an operator as the key.
−
Hide 12 properties
contains
boolean | number | string
Contains comparison value.
endsWith
boolean | number | string
Ends-with comparison value.
eq
boolean | number | string
Equality comparison value.
field
string · min length: 1 required
The document field to filter on.
gt
boolean | number | string
Greater-than comparison value.
gte
boolean | number | string
Greater-than-or-equal comparison value.
includes
boolean | number | string
Checks if multivalue field includes the value.
lt
boolean | number | string
Less-than comparison value.
lte
boolean | number | string
Less-than-or-equal comparison value.
neq
boolean | number | string
Inequality comparison value.
range
object
Range comparison values.
+
Show 4 properties
gt
boolean | number | string
A value that can be a string, number, or boolean.
gte
boolean | number | string
A value that can be a string, number, or boolean.
lt
boolean | number | string
A value that can be a string, number, or boolean.
lte
boolean | number | string
A value that can be a string, number, or boolean.
startsWith
boolean | number | string
Starts-with comparison value.
A condition that checks for the existence or non-existence of a field.
−
Hide 2 properties
exists
boolean
Indicates whether the field exists or not.
field
string · min length: 1 required
The document field to check.
A logical AND that groups multiple conditions.
−
Hide 1 property
and
union anyOf [] required
An array of conditions. All sub-conditions must be true for this condition to be true.
Same options as
wherein
action: grok
A logical OR that groups multiple conditions.
−
Hide 1 property
or
union anyOf [] required
An array of conditions. At least one sub-condition must be true for this condition to be true.
Same options as
wherein
action: grok
A logical NOT that negates a condition.
−
Hide 1 property
not
union anyOf required
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
A condition that always evaluates to false.
−
Hide 1 property
never
object required
An empty object. This condition never matches.
A condition that always evaluates to true. Useful for catch-all scenarios, but use with caution as partitions are ordered.
−
Hide 1 property
always
object required
An empty object. This condition always matches.
Dissect processor - Extract fields from text using a lightweight, delimiter-based parser
−
Hide 9 properties
action
enum required
Values:
dissect
append_separator
string · min length: 1
Separator inserted when target fields are concatenated
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to parse with dissect pattern
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
pattern
string · min length: 1 required
Dissect pattern describing field boundaries
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
URI parts processor - Parse a URI into components (scheme, domain, port, path, query, fragment, ...)
−
Hide 10 properties
action
enum required
Values:
uri_parts
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field holding the URI string to parse
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
keep_original
boolean
If true (default), preserve the original URI string alongside the extracted parts
remove_if_successful
boolean
If true, remove the source field after a successful parse. Source field is kept on failure.
to
string · min length: 1
Target field / column prefix for the extracted URI components (defaults to "url"). May equal from — the canonical ECS shape parses the url field in place. Note: combining to === from with remove_if_successful: true nulls the parsed object after writing it.
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Date processor - Parse dates from strings using one or more expected formats
−
Hide 11 properties
action
enum required
Values:
date
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
formats
string [] required
Accepted input date formats, tried in order
from
string · min length: 1 required
Source field containing the date/time text
ignore_failure
boolean
Continue pipeline execution if this processor fails
locale
string · min length: 1
Optional locale for date parsing
output_format
string · min length: 1
Optional output format for storing the parsed date as text
timezone
string · min length: 1
Optional timezone for date parsing
to
string · min length: 1
Target field for the parsed date (defaults to source)
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 5 properties
action
enum required
Values:
drop_document
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
math
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
expression
string · min length: 1 required
A non-empty string.
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1 required
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Rename processor - Change a field name and optionally its location
−
Hide 9 properties
action
enum required
Values:
rename
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Existing source field to rename or move
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip when source field is missing
override
boolean
Allow overwriting the target field if it already exists
to
string · min length: 1 required
New field name or destination path
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Set processor - Assign a literal or copied value to a field (mutually exclusive inputs)
−
Hide 9 properties
action
enum required
Values:
set
copy_from
string · min length: 1
Copy value from another field instead of providing a literal
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
override
boolean
Allow overwriting an existing target field
to
string · min length: 1 required
Target field to set or create
value
object
Literal value to assign to the target field
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Append processor - Append one or more values to an existing or new array field
−
Hide 8 properties
action
enum required
Values:
append
allow_duplicates
boolean
If true, do not deduplicate appended values
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1 required
Array field to append values to
value
object [] · min items: 1 required
Values to append (must be literal, no templates)
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Remove by prefix processor - Remove a field and all nested fields matching the prefix
−
Hide 5 properties
action
enum required
Values:
remove_by_prefix
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Field to remove along with all its nested fields
ignore_failure
boolean
Continue pipeline execution if this processor fails
Remove processor - Delete one or more fields from the document
−
Hide 7 properties
action
enum required
Values:
remove
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Field to remove from the document
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 10 properties
action
enum required
Values:
replace
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
pattern
string · min length: 1 required
replacement
string required
to
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Redact processor - Mask sensitive data using Grok patterns
−
Hide 11 properties
action
enum required
Values:
redact
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to redact sensitive data from
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing (defaults to true)
pattern_definitions
map string to string
Custom pattern definitions to use in the patterns
patterns
string [] · min items: 1 required
Grok patterns to match sensitive data (for example, "%{IP:client}", "%{EMAILADDRESS:email}")
prefix
string
Prefix to prepend to the redacted pattern name (defaults to "<")
suffix
string
Suffix to append to the redacted pattern name (defaults to ">")
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
uppercase
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
lowercase
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
trim
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 9 properties
action
enum required
Values:
join
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
delimiter
string required
description
string
Human-readable notes about this processor step
from
string [] · min items: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1 required
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Split processor - Split a field value into an array using a separator
−
Hide 10 properties
action
enum required
Values:
split
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to split into an array
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
preserve_trailing
boolean
Preserve empty trailing fields in the split result
separator
string · min length: 1 required
Regex separator used to split the field value into an array
to
string · min length: 1
Target field for the split array (defaults to source)
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 9 properties
action
enum required
Values:
sort
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Array field to sort
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
order
enum
Sort order - "asc" (ascending) or "desc" (descending). Defaults to "asc"
Values:
asc
desc
to
string · min length: 1
Target field for the sorted array (defaults to source)
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Convert processor - Change the data type of a field value (integer, long, double, boolean, or string)
−
Hide 9 properties
action
enum required
Values:
convert
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to convert to a different data type
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
to
string · min length: 1
Target field for the converted value (defaults to source)
type
enum required
Target data type: integer, long, double, boolean, or string
Values:
integer
long
double
boolean
string
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
concat
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
union anyOf [] · min items: 1 required
Any of:
−
Hide 2 properties
type
enum required
Values:
field
value
string · min length: 1 required
−
Hide 2 properties
type
enum required
Values:
literal
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1 required
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 10 properties
action
enum required
Values:
network_direction
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
destination_ip
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
internal_networks
string [] required
source_ip
string · min length: 1 required
target_field
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 10 properties
action
enum required
Values:
network_direction
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
destination_ip
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
internal_networks_field
string · min length: 1 required
source_ip
string · min length: 1 required
target_field
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
JsonExtract processor - Extract values from JSON strings using JSONPath-like selectors
−
Hide 8 properties
action
enum required
Values:
json_extract
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
List of extraction specifications
−
Hide 3 properties
field
string · min length: 1 required
Source field containing the JSON string to parse
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 9 properties
action
enum required
Values:
enrich
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
policy_name
string · min length: 1 required
A non-empty string.
to
string · min length: 1 required
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
User agent processor - Extract browser, OS, and device details from a user agent string
−
Hide 11 properties
action
enum required
Values:
user_agent
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
Extracts device type from the user agent string
from
string · min length: 1 required
The field containing the user agent string
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
properties
enum []
Specific properties to extract (defaults to all)
Values:
name
os
device
original
version
regex_file
string · min length: 1
Custom regex file name containing the regular expressions for parsing the user agent string
to
string · min length: 1
The field that will be filled with the user agent details
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Registered domain processor - extracts domain, registered_domain, top_level_domain, subdomain from a FQDN
−
Hide 8 properties
action
enum required
Values:
registered_domain
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
expression
string · min length: 1 required
The string expression containing the FQDN to parse
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when expression field is missing
prefix
string · min length: 1 required
The prefix for the output columns. The extracted parts are available as prefix.part_name
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Manual ingest pipeline wrapper around native Elasticsearch processors
−
Hide 8 properties
action
enum required
Manual ingest pipeline - executes raw Elasticsearch ingest processors
Values:
manual_ingest_pipeline
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
on_failure
string to object []
Fallback processors to run when a processor fails
processors
string to object [] required
List of raw Elasticsearch ingest processors to run
tag
string
Optional ingest processor tag for Elasticsearch
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 2 properties
condition
union anyOf required
Any of:
FilterCondition
{ and }
{ or }
{ not }
{ never }
{ always }
A basic filter condition, either unary or binary.
−
Hide 2 properties
else
union anyOf []
Same options as
stepsin
processing
steps
union anyOf [] required
Same options as
stepsin
processing
A logical AND that groups multiple conditions.
−
Hide 3 properties
else
union anyOf []
Same options as
stepsin
processing
steps
union anyOf [] required
Same options as
stepsin
processing
and
union anyOf [] required
An array of conditions. All sub-conditions must be true for this condition to be true.
Same options as
wherein
action: grok
A logical OR that groups multiple conditions.
−
Hide 3 properties
else
union anyOf []
Same options as
stepsin
processing
steps
union anyOf [] required
Same options as
stepsin
processing
or
union anyOf [] required
An array of conditions. At least one sub-condition must be true for this condition to be true.
Same options as
wherein
action: grok
A logical NOT that negates a condition.
−
Hide 3 properties
else
union anyOf []
Same options as
stepsin
processing
steps
union anyOf [] required
Same options as
stepsin
processing
not
union anyOf required
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
A condition that always evaluates to false.
−
Hide 3 properties
else
union anyOf []
Same options as
stepsin
processing
steps
union anyOf [] required
Same options as
stepsin
processing
never
object required
An empty object. This condition never matches.
A condition that always evaluates to true. Useful for catch-all scenarios, but use with caution as partitions are ordered.
−
Hide 3 properties
else
union anyOf []
Same options as
stepsin
processing
steps
union anyOf [] required
Same options as
stepsin
processing
always
object required
An empty object. This condition always matches.
settings
object required
−
Hide 3 properties
index.number_of_replicas
object
+
Show 1 property
index.number_of_shards
object
+
Show 1 property
index.refresh_interval
object
+
Show 1 property
value
union string | enum required
Values:
-1
wired
object required
−
Hide 3 properties
fields
map string to anyOf required
<string>
union anyOf
−
Hide 3 properties
Any of:
object
object
type: system
−
Hide 3 properties
format
string · min length: 1
A non-empty string.
type
enum required
Values:
keyword
match_only_text
long
double
date
boolean
ip
geo_point
integer
short
byte
float
half_float
text
wildcard
version
unsigned_long
date_nanos
−
Hide 3 properties
description
string required
−
Hide 2 properties
type
enum required
Values:
system
routing
object [] required
−
Hide 4 properties
destination
string · min length: 1 required
A non-empty string.
status
enum
Values:
enabled
disabled
where
union anyOf required
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 5 properties
classic
object required
−
Hide 1 property
field_overrides
map string to anyOf
<string>
union anyOf
+
Show 2 properties
Any of:
−
Hide 3 properties
format
string · min length: 1
A non-empty string.
type
enum required
Values:
keyword
match_only_text
long
double
date
boolean
ip
geo_point
integer
short
byte
float
half_float
text
wildcard
version
unsigned_long
date_nanos
−
Hide 2 properties
type
enum required
Values:
system
failure_store
union anyOf required
Any of:
{ inherit }
{ disabled }
object
object
−
Hide 1 property
lifecycle
object required
−
Hide 1 property
enabled
object required
−
Hide 1 property
data_retention
string · min length: 1
A non-empty string.
−
Hide 1 property
lifecycle
object required
−
Hide 1 property
lifecycle
union anyOf required
Any of:
−
Hide 1 property
dsl
object required
−
Hide 3 properties
data_retention
string · min length: 1
A non-empty string.
downsample
object []
+
Show 2 properties
after
string · min length: 1 required
A non-empty string.
fixed_interval
string · min length: 1 required
A non-empty string.
frozen_after
string · min length: 1
A non-empty string.
−
Hide 1 property
ilm
object required
−
Hide 1 property
policy
string · min length: 1 required
A non-empty string.
processing
union anyOf required
Any of:
−
Hide 1 property
steps
union anyOf [] required
Any of:
Grok processor - Extract fields from text using grok patterns
−
Hide 9 properties
action
enum required
Values:
grok
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to parse with grok patterns
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
pattern_definitions
map string to string
patterns
string [] · min items: 1 required
Grok patterns applied in order to extract fields
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Dissect processor - Extract fields from text using a lightweight, delimiter-based parser
−
Hide 9 properties
action
enum required
Values:
dissect
append_separator
string · min length: 1
Separator inserted when target fields are concatenated
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to parse with dissect pattern
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
pattern
string · min length: 1 required
Dissect pattern describing field boundaries
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
URI parts processor - Parse a URI into components (scheme, domain, port, path, query, fragment, ...)
−
Hide 10 properties
action
enum required
Values:
uri_parts
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field holding the URI string to parse
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
keep_original
boolean
If true (default), preserve the original URI string alongside the extracted parts
remove_if_successful
boolean
If true, remove the source field after a successful parse. Source field is kept on failure.
to
string · min length: 1
Target field / column prefix for the extracted URI components (defaults to "url"). May equal from — the canonical ECS shape parses the url field in place. Note: combining to === from with remove_if_successful: true nulls the parsed object after writing it.
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Date processor - Parse dates from strings using one or more expected formats
−
Hide 11 properties
action
enum required
Values:
date
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
formats
string [] required
Accepted input date formats, tried in order
from
string · min length: 1 required
Source field containing the date/time text
ignore_failure
boolean
Continue pipeline execution if this processor fails
locale
string · min length: 1
Optional locale for date parsing
output_format
string · min length: 1
Optional output format for storing the parsed date as text
timezone
string · min length: 1
Optional timezone for date parsing
to
string · min length: 1
Target field for the parsed date (defaults to source)
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 5 properties
action
enum required
Values:
drop_document
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
math
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
expression
string · min length: 1 required
A non-empty string.
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1 required
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Rename processor - Change a field name and optionally its location
−
Hide 9 properties
action
enum required
Values:
rename
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Existing source field to rename or move
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip when source field is missing
override
boolean
Allow overwriting the target field if it already exists
to
string · min length: 1 required
New field name or destination path
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Set processor - Assign a literal or copied value to a field (mutually exclusive inputs)
−
Hide 9 properties
action
enum required
Values:
set
copy_from
string · min length: 1
Copy value from another field instead of providing a literal
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
override
boolean
Allow overwriting an existing target field
to
string · min length: 1 required
Target field to set or create
value
object
Literal value to assign to the target field
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Append processor - Append one or more values to an existing or new array field
−
Hide 8 properties
action
enum required
Values:
append
allow_duplicates
boolean
If true, do not deduplicate appended values
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1 required
Array field to append values to
value
object [] · min items: 1 required
Values to append (must be literal, no templates)
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Remove by prefix processor - Remove a field and all nested fields matching the prefix
−
Hide 5 properties
action
enum required
Values:
remove_by_prefix
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Field to remove along with all its nested fields
ignore_failure
boolean
Continue pipeline execution if this processor fails
Remove processor - Delete one or more fields from the document
−
Hide 7 properties
action
enum required
Values:
remove
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Field to remove from the document
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 10 properties
action
enum required
Values:
replace
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
pattern
string · min length: 1 required
replacement
string required
to
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Redact processor - Mask sensitive data using Grok patterns
−
Hide 11 properties
action
enum required
Values:
redact
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to redact sensitive data from
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing (defaults to true)
pattern_definitions
map string to string
Custom pattern definitions to use in the patterns
patterns
string [] · min items: 1 required
Grok patterns to match sensitive data (for example, "%{IP:client}", "%{EMAILADDRESS:email}")
prefix
string
Prefix to prepend to the redacted pattern name (defaults to "<")
suffix
string
Suffix to append to the redacted pattern name (defaults to ">")
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
uppercase
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
lowercase
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
trim
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 9 properties
action
enum required
Values:
join
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
delimiter
string required
description
string
Human-readable notes about this processor step
from
string [] · min items: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1 required
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Split processor - Split a field value into an array using a separator
−
Hide 10 properties
action
enum required
Values:
split
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to split into an array
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
preserve_trailing
boolean
Preserve empty trailing fields in the split result
separator
string · min length: 1 required
Regex separator used to split the field value into an array
to
string · min length: 1
Target field for the split array (defaults to source)
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 9 properties
action
enum required
Values:
sort
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Array field to sort
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
order
enum
Sort order - "asc" (ascending) or "desc" (descending). Defaults to "asc"
Values:
asc
desc
to
string · min length: 1
Target field for the sorted array (defaults to source)
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Convert processor - Change the data type of a field value (integer, long, double, boolean, or string)
−
Hide 9 properties
action
enum required
Values:
convert
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
string · min length: 1 required
Source field to convert to a different data type
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
to
string · min length: 1
Target field for the converted value (defaults to source)
type
enum required
Target data type: integer, long, double, boolean, or string
Values:
integer
long
double
boolean
string
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 8 properties
action
enum required
Values:
concat
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
from
union anyOf [] · min items: 1 required
Any of:
−
Hide 2 properties
type
enum required
Values:
field
value
string · min length: 1 required
−
Hide 2 properties
type
enum required
Values:
literal
ignore_failure
boolean
Continue pipeline execution if this processor fails
to
string · min length: 1 required
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 10 properties
action
enum required
Values:
network_direction
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
destination_ip
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
internal_networks
string [] required
source_ip
string · min length: 1 required
target_field
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 10 properties
action
enum required
Values:
network_direction
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
destination_ip
string · min length: 1 required
ignore_failure
boolean
Continue pipeline execution if this processor fails
internal_networks_field
string · min length: 1 required
source_ip
string · min length: 1 required
target_field
string · min length: 1
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
JsonExtract processor - Extract values from JSON strings using JSONPath-like selectors
−
Hide 8 properties
action
enum required
Values:
json_extract
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
List of extraction specifications
−
Hide 3 properties
field
string · min length: 1 required
Source field containing the JSON string to parse
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 9 properties
action
enum required
Values:
enrich
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
policy_name
string · min length: 1 required
A non-empty string.
to
string · min length: 1 required
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
User agent processor - Extract browser, OS, and device details from a user agent string
−
Hide 11 properties
action
enum required
Values:
user_agent
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
Extracts device type from the user agent string
from
string · min length: 1 required
The field containing the user agent string
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when source field is missing
properties
enum []
Specific properties to extract (defaults to all)
Values:
name
os
device
original
version
regex_file
string · min length: 1
Custom regex file name containing the regular expressions for parsing the user agent string
to
string · min length: 1
The field that will be filled with the user agent details
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Registered domain processor - extracts domain, registered_domain, top_level_domain, subdomain from a FQDN
−
Hide 8 properties
action
enum required
Values:
registered_domain
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
expression
string · min length: 1 required
The string expression containing the FQDN to parse
ignore_failure
boolean
Continue pipeline execution if this processor fails
ignore_missing
boolean
Skip processing when expression field is missing
prefix
string · min length: 1 required
The prefix for the output columns. The extracted parts are available as prefix.part_name
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
Manual ingest pipeline wrapper around native Elasticsearch processors
−
Hide 8 properties
action
enum required
Manual ingest pipeline - executes raw Elasticsearch ingest processors
Values:
manual_ingest_pipeline
customIdentifier
string · min length: 1
Custom identifier to correlate this processor across outputs
description
string
Human-readable notes about this processor step
ignore_failure
boolean
Continue pipeline execution if this processor fails
on_failure
string to object []
Fallback processors to run when a processor fails
processors
string to object [] required
List of raw Elasticsearch ingest processors to run
tag
string
Optional ingest processor tag for Elasticsearch
where
union anyOf
The root condition object. It can be a simple filter or a combination of other conditions.
Same options as
wherein
action: grok
−
Hide 2 properties
condition
union anyOf required
Same options as
conditionin
StreamlangConditionBlock
−
Hide 1 property
processors
string to object [] required
settings
object required
−
Hide 3 properties
index.number_of_replicas
object
+
Show 1 property
index.number_of_shards
object
+
Show 1 property
index.refresh_interval
object
+
Show 1 property
value
union string | enum required
Values:
-1
200
The ingest settings were updated successfully.