Build your first workflow

In this tutorial, you'll create a workflow that indexes and searches through national parks data. Along the way, you'll learn the core concepts and capabilities of workflows.

Prerequisites

To use workflows, turn on the Elastic Workflows (workflows:ui:enabled) advanced setting.
You must have the appropriate subscription. Refer to the subscription page for Elastic Cloud and Elastic Stack/self-managed for the breakdown of available features and their associated subscription tiers.
Access to workflows is controlled by Kibana privileges. Ensure your role has All privileges for Analytics → Workflows, which allows you to create, edit, run, and manage workflows.

Tutorial

Go to Workflows

To access the Workflows page, find Workflows in the navigation menu or using the global search field.
Create a new workflow

Click Create a new workflow. The YAML editor opens.

Define your workflow

Remove the placeholder content and copy and paste the following YAML into the editor:

name: 🏔️ National Parks Demo
description: Creates an Elasticsearch index, loads sample national park data using bulk operations, searches for parks by category, and displays the results.
enabled: true
tags: ["demo", "getting-started"]
consts:
  indexName: national-parks
triggers:
  - type: manual
steps:
  - name: get_index
    type: elasticsearch.indices.exists
    with:
      index: "{{ consts.indexName }}"
  - name: check_if_index_exists
    type: if
    condition: 'steps.get_index.output : true'
    steps:
      - name: index_already_exists
        type: console
        with:
          message: "index: {{ consts.indexName }} already exists. Will proceed to delete it and re-create"
      - name: delete_index
        type: elasticsearch.indices.delete
        with:
          index: "{{ consts.indexName }}"
    else:
      - name: no_index_found
        type: console
        with:
          message: "index: {{ consts.indexName }} Not found. Will proceed to create"

  - name: create_parks_index
    type: elasticsearch.indices.create
    with:
      index: "{{ consts.indexName }}"
      mappings:
        properties:
          name: { type: text }
          category: { type: keyword }
          description: { type: text }
  - name: bulk_index_park_data
    type: elasticsearch.request
    with:
      method: POST
      path: /{{ consts.indexName }}/_bulk?refresh=wait_for
      headers:
        Content-Type: application/x-ndjson
      body: |
        {"index":{}}
        {"name": "Yellowstone National Park", "category": "geothermal", "description": "America's first national park, established in 1872, famous for Old Faithful geyser and diverse wildlife including grizzly bears, wolves, and herds of bison and elk."}
        {"index":{}}
        {"name": "Grand Canyon National Park", "category": "canyon", "description": "Home to the immense Grand Canyon, a mile deep gorge carved by the Colorado River, revealing millions of years of geological history in its colorful rock layers."}
        {"index":{}}
        {"name": "Yosemite National Park", "category": "mountain", "description": "Known for its granite cliffs, waterfalls, clear streams, giant sequoia groves, and biological diversity. El Capitan and Half Dome are iconic rock formations."}
        {"index":{}}
        {"name": "Zion National Park", "category": "canyon", "description": "Utah's first national park featuring cream, pink, and red sandstone cliffs soaring into a blue sky. Famous for the Narrows wade through the Virgin River."}
        {"index":{}}
        {"name": "Rocky Mountain National Park", "category": "mountain", "description": "Features mountain environments, from wooded forests to mountain tundra, with over 150 riparian lakes and diverse wildlife at various elevations."}
  - name: search_park_data
    type: elasticsearch.search
    with:
      index: "{{ consts.indexName }}"
      size: 5
      query:
        term:
          category: "canyon"
  - name: log_results
    type: console
    with:
      message: |-
        Found {{ steps.search_park_data.output.hits.total.value }} parks in category "canyon".
  - name: loop_over_results
    type: foreach
    foreach: "{{steps.search_park_data.output.hits.hits | json}}"
    steps:
      - name: process-item
        type: console
        with:
          message: "{{foreach.item._source.name}}"

Save your workflow

Click Save. Your workflow is now ready to run.
Run your workflow

Click the Run icon (next to Save) to execute your workflow.
Monitor execution

As your workflow runs, execution logs display in a panel next to your workflow. In the panel, you can find:
- Real-time execution logs: Each step appears as it executes.
- Workflow status indicators: Green for success, red for failures, and timestamps for duration.
- Expandable step details: Click any step to see input, output, and timeline.
View execution history

To examine past executions:
1. Click the Executions tab.
2. View a list of all workflow runs (including pending and in progress runs), along with their status and completion time.
3. Click any execution to see its detailed logs.

Understand what happened

Let's examine each part of the workflow to understand how it works.

Workflow metadata

name: 🏔️ National Parks Demo
description: Creates an Elasticsearch index, loads sample national park data using bulk operations, searches for parks by category, and displays the results.
enabled: true
tags: ["demo", "getting-started"]

name: A unique identifier for your workflow.
description: Explains the workflow's purpose.
enabled: Controls whether the workflow can be run.
tags: Labels for organizing and finding workflows.

Constants
YAML
```
consts:
  indexName: national-parks-data
```
- consts: Defines reusable values that can be referenced throughout the workflow. Use constants for fixed values that don't change between runs. For values that vary per execution, use inputs instead.
- Accessed using template syntax: {{ consts.indexName }}. This promotes consistency and makes the workflow easier to maintain.
Triggers
YAML
```
triggers:
  - type: manual
```
- triggers: Defines how the workflow starts.
- type: Specifies the trigger type. Manual triggers require explicit user action (clicking the Run icon ) to start a workflow.
Create index
YAML
```
- name: create_parks_index
  type: elasticsearch.indices.create
  with:
    index: "{{ consts.indexName }}"
    settings:
      number_of_shards: 1
      number_of_replicas: 0
    mappings:
      properties:
        name: { type: text }
        category: { type: keyword }
        description: { type: text }
```
- Step type: This is an action step that directly interacts with Elasticsearch.
- Step purpose: Establishes the data structure for the park information, ensuring fields are properly typed for searching and aggregation.
- Key elements:
  - Uses elasticsearch.indices.create, which is a built-in action that maps to the Elasticsearch Create Index API.
  - Defines mappings to control how data is indexed (text for full-text search, keyword for exact matching).
  - References the constant indexName for consistency.
  - Sets index settings for optimal performance in this demo.
Bulk index documents
YAML
```
- name: bulk_index_park_data
  type: elasticsearch.request
  with:
    method: POST
    path: /{{ consts.indexName }}/_bulk?refresh=wait_for
    headers:
      Content-Type: application/x-ndjson
    body: |
      {"index":{}}
      {"name": "Yellowstone National Park", "category": "geothermal", "description": "America's first national park, established in 1872, famous for Old Faithful geyser and diverse wildlife including grizzly bears, wolves, and herds of bison and elk."}
      {"index":{}}
      {"name": "Grand Canyon National Park", "category": "canyon", "description": "Home to the immense Grand Canyon, a mile deep gorge carved by the Colorado River, revealing millions of years of geological history in its colorful rock layers."}
    # ... additional parks
```
- Step type: A generic Elasticsearch request action that calls the bulk API directly.
- Step purpose: Efficiently loads multiple documents in a single operation, populating the index with sample data.
- Key elements:
  - Uses elasticsearch.request with method: POST to call the /<index>/_bulk endpoint.
  - The refresh=wait_for query parameter ensures documents are searchable immediately after indexing.
  - The headers parameter sets Content-Type: application/x-ndjson, which is required for bulk operations.
  - The body uses the pipe (|) syntax to define a multiline string in YAML.
  - Each pair of lines represents an action (such as {"index":{}}) followed by the document to index.
  - Uses the field names (name, category, description) from the mappings defined in the create_parks_index step.
  - This step demonstrates how to handle batch operations using the generic request action.
Search parks
YAML
```
- name: search_park_data
  type: elasticsearch.search
  with:
    index: "{{ consts.indexName }}"
    size: 5
    query:
      term:
        category: "canyon"
```
- Step type: Internal action step for querying Elasticsearch.
- Step purpose: Retrieves specific data based on criteria, demonstrating how workflows can make decisions based on data.
- Key elements:
  - Searches for parks with category "canyon" (will find Grand Canyon and Zion).
  - Results from steps.search_park_data.output are automatically available to subsequent steps.
  - Limits results to 5 documents for manageable output.
  - Shows how workflows can filter and process data dynamically.
Log results
YAML
```
- name: log_results
  type: console
  with:
    message: |-
      Found {{ steps.search_park_data.output.hits.total.value }} parks in category "canyon".
      Top results: {{ steps.search_park_data.output.hits.hits | json:2 }}
```
- Step type: A console step for output and debugging.
- Step purpose: Presents the results in a human-readable format, demonstrating how to access and format data from previous steps.
- Key elements:
  - Template variables access the search results: {{ steps.search_park_data.output }}.
  - The | json:2 filter formats JSON output with indentation.
  - Uses the exact step name search_park_data to reference previous step output.
  - Shows how data flows through the workflow and can be transformed.

Key concepts demonstrated

This workflow introduces several fundamental concepts:

Action steps: Built-in steps that interact with Elasticsearch APIs.
Data flow: How information moves from step to step using outputs and template variables.
Constants: Reusable values that make workflows maintainable.
Template syntax: The {{ }} notation for dynamic values.
Step chaining: How each step builds on previous ones to create a complete process.

What's next

Learn more about the workflow framework:

Triggers: Control when workflows run.
Steps: Define how a workflow operates and the outcomes it can produce.
Pass data and handle errors: Make the workflow resilient to failures and understand mechanisms for controlling data flow.