Type to search.

Get started with system logs

Serverless

Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Serverless interfaces and procedures might differ from classic Elastic Stack deployments.

Stack

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

In this guide you can learn how to onboard system log data from a machine or server, then explore the data in Discover.

To follow the steps in this guide, you need an Elastic Stack deployment that includes:

  • Elasticsearch for storing and searching data
  • Kibana for visualizing and managing data
  • Kibana user with All privileges on Fleet and Integrations. For the minimum privileges and how they apply across spaces, refer to Fleet and Integrations roles and privileges.

To get started quickly, create an Elastic Cloud Hosted deployment and host it on AWS, GCP, or Azure. Try it out for free.

The Admin role or higher is required to onboard log data. To learn more, refer to Assign user roles and privileges.

Follow these steps to onboard system log data.

  1. Open your project

    Open an Elastic Observability Serverless project or Elastic Stack deployment.

  2. Select data collection method

    From the Observability UI, go to Add data. Under What do you want to monitor?, select Host, then select one of these options:

    Collect native OpenTelemetry metrics and logs using Elastic Agent.

    Recommended for: Users who want to collect native OpenTelemetry data or are already using OpenTelemetry in their environment.

    Bring data from Elastic integrations using the Elastic Agent.

    Recommended for: Users who want to leverage Elastic's pre-built integrations and centralized management through Fleet.

  3. Follow setup instructions

    Follow the in-product steps to auto-detect your logs and install and configure your chosen data collector.

  4. Verify data collection

    After the agent is installed and successfully streaming log data, you can view the data in the UI:

    1. From the navigation menu, go to Discover.
    2. In ES|QL mode, enter FROM logs-*. In classic mode, select All logs from the Data views menu. The view shows all log datasets. Notice you can add fields, change the view, expand a document to see details, and perform other actions to explore your data.

  5. Explore and analyze your data

    Now that you have logs flowing into Elasticsearch, you can start exploring and analyzing your data:

While the Elastic Agent and OpenTelemetry Collector are the recommended approaches for most users, Elastic provides additional tools for specific use cases:

Filebeat is a lightweight data shipper that sends log data to Elasticsearch. It's ideal for:

  • Simple log collection: When you need to collect logs from specific files or directories.
  • Custom parsing: When you need to parse logs using ingest pipelines before indexing.
  • Legacy systems: When you can't install the Elastic Agent or OpenTelemetry Collector.

For more information, refer to Collecting log data with Filebeat and Ingest logs from applications using Filebeat.

Winlogbeat is specifically designed for collecting Windows event logs. It's ideal for:

  • Windows environments: When you need to collect Windows security, application, and system event logs.
  • Security monitoring: When you need detailed Windows security event data.
  • Compliance requirements: When you need to capture specific Windows event IDs.

For more information, refer to the Winlogbeat documentation.

Logstash is a powerful data processing pipeline that can collect, transform, and enrich log data before sending it to Elasticsearch. It's ideal for:

  • Complex data processing: When you need to parse, filter, and transform logs before indexing.
  • Multiple data sources: When you need to collect logs from various sources and normalize them.
  • Advanced use cases: When you need data enrichment, aggregation, or routing to multiple destinations.
  • Extending Elastic integrations: When you want to add custom processing to data collected by Elastic Agent or Beats.

For more information, refer to Logstash and Using Logstash with Elastic integrations.

You can use Elasticsearch REST APIs to send log data directly to Elasticsearch. This approach is ideal for:

  • Custom applications: When you want to send logs directly from your application code.
  • Programmatic collection: When you need to collect logs using custom scripts or tools.
  • Real-time streaming: When you need to send logs as they're generated.

For more information, refer to Elasticsearch REST APIs.

Now that you've added logs and explored your data, learn how to onboard other types of data:

To onboard other types of data, select Add Data from the main menu.