Logstash pipelines

On the Logstash Pipelines management page, you can control multiple Logstash instances and pipeline configurations.

On the Logstash side, you must enable configuration management and register Logstash to use the centrally managed pipeline configurations.

1. Configure centralized pipeline management.

2. To add a new pipeline, go to the Logstash Pipelines management page using the navigation menu or the global search field.

3. Click Create pipeline.

4. Provide the following details, then click Create and deploy.

   Pipeline ID

   A name that uniquely identifies the pipeline. This is the ID that you used when you configured centralized pipeline management and specified a list of pipeline IDs in the xpack.management.pipeline.id setting.

   Description

   A description of the pipeline configuration. This information is for your use.

   Pipeline

   The pipeline configuration. You can treat the editor like any other editor. You don’t have to worry about whitespace or indentation.

   Pipeline workers

   The number of parallel workers used to run the filter and output stages of the pipeline.

   Pipeline batch size

   The maximum number of events an individual worker thread collects before executing filters and outputs.

   Pipeline batch delay

   Time in milliseconds to wait for each event before sending an undersized batch to pipeline workers.

   Queue type

   The internal queueing model for event buffering. Options are memory for in-memory queueing and persisted for disk-based acknowledged queueing.

   Queue max bytes

   The total capacity of the queue when persistent queues are enabled.

   Queue checkpoint writes

   The maximum number of events written before a checkpoint is forced when persistent queues are enabled.

To delete one or more pipelines, select their checkboxes then click Delete.

For more information about pipeline behavior, go to Centralized Pipeline Management.