Working with plugins
Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability This functionality is generally available and ready for production usage. Serverless interfaces and procedures might differ from classic Elastic Stack deployments. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Serverless
Generally available
Stack
Generally available
Apple’s rollout of stricter notarization requirements affected the notarization of Logstash artifacts. If macOS displays a dialog when you first run Logstash, you need to take an action to allow it to run. To prevent Gatekeeper checks on the Logstash files, run the following command on the downloaded .tar.gz archive or the directory to which it was extracted:
xattr -d -r com.apple.quarantine <archive-or-directory>
For example, if the .tar.gz file was extracted to the default logstash-<VERSION> directory, the command is:
xattr -d -r com.apple.quarantine logstash-9.5.6
- Replace the version with your installed Logstash version if it differs.
Alternatively, you can add a security override if a Gatekeeper popup appears by following the instructions in the How to open an app that hasn’t been notarized or is from an unidentified developer section of Safely open apps on your Mac.
Logstash has a rich collection of input, filter, codec, and output plugins. Check out the Elastic Support Matrix to see which plugins are supported at various levels.
Plugins are available in self-contained packages called gems and hosted on RubyGems.org. Use the plugin manager script--bin/logstash-plugin--to manage plugins:
- Listing plugins
- Adding plugins to your deployment
- Updating plugins
- Removing plugins
- Advanced: Adding a locally built plugin
- Advanced: Using
--path.plugins
If you don’t have an internet connection, check out Offline Plugin Management for information on building, installing, and updating offline plugin packs.
Most plugin manager commands require access to the internet to reach RubyGems.org. If your organization is behind a firewall, you can set these environments variables to configure Logstash to use your proxy.
export http_proxy=http://localhost:3128
export https_proxy=http://localhost:3128
Logstash release packages bundle common plugins. To list the plugins currently available in your deployment:
bin/logstash-plugin list
bin/logstash-plugin list --verbose
bin/logstash-plugin list '*namefragment*'
bin/logstash-plugin list --group output
- Lists all installed plugins
- Lists installed plugins with version information
- Lists all installed plugins containing a namefragment
- Lists all installed plugins for a particular group (input, filter, codec, output)
When you have access to internet, you can retrieve plugins hosted on the RubyGems.orgpublic repository and install them on top of your Logstash installation.
bin/logstash-plugin install logstash-input-github
After a plugin is successfully installed, you can use it in your configuration file.
Plugins have their own release cycles and are often released independently of Logstash’s core release cycle. Using the update subcommand you can get the latest version of the plugin.
bin/logstash-plugin update
bin/logstash-plugin update logstash-input-github
- updates all installed plugins
- updates only the plugin you specify
To avoid introducing breaking changes, the plugin manager updates only plugins for which newer minor or patch versions exist by default. If you wish to also include breaking changes, specify --level=major.
bin/logstash-plugin update --level=major
bin/logstash-plugin update --level=major logstash-input-github
- updates all installed plugins to latest, including major versions with breaking changes
- updates only the plugin you specify to latest, including major versions with breaking changes
If you need to remove plugins from your Logstash installation:
bin/logstash-plugin remove logstash-input-github
In some cases, you may want to install plugins which are not yet released and not hosted on RubyGems.org. Logstash provides you the option to install a locally built plugin which is packaged as a ruby gem. Using a file location:
bin/logstash-plugin install /path/to/logstash-output-kafka-1.0.0.gem
Using the Logstash --path.plugins flag, you can load a plugin source code located on your file system. Typically this is used by developers who are iterating on a custom plugin and want to test it before creating a ruby gem.
The path needs to be in a specific directory hierarchy: PATH/logstash/TYPE/NAME.rb, where TYPE is inputs filters, outputs or codecs and NAME is the name of the plugin.
# supposing the code is in /opt/shared/lib/logstash/inputs/my-custom-plugin-code.rb
bin/logstash --path.plugins /opt/shared/lib