Automate root cause analysis for an observability alert
Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability This functionality is generally available and ready for production usage. Serverless interfaces and procedures might differ from classic Elastic Stack deployments. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Serverless
Generally available
Stack
9.4+
Generally available since 9.4
This guide walks through building an observability workflow that responds to an alert by running an Elastic Agent Builder agent for root-cause analysis, generating a case title and description from the agent's output, opening the case, and attaching both the alert and the agent's reasoning trace as comments.
The workflow is adapted from root-cause-analysis-rca-workflow.yaml in the elastic/workflows library.
If you're new to workflows, complete Build your first workflow first.
- Permissions.
Allon Analytics → Workflows, Observability → Cases, and whatever Agent Builder privilege is required to invoke agents in your space. Refer to Kibana privileges. - Alerting rule. A configured observability alerting rule that fires on the conditions you want to auto-investigate (metric thresholds, SLO burn rate, anomaly detection, or custom query).
- SRE agent. An Elastic Agent Builder agent configured to investigate observability signals. The examples use
elastic-ai-agentas a default. Substitute your agent ID. - Attach the workflow to the rule. After saving the workflow, attach it as an action on the alerting rule. Refer to Alert triggers.
The workflow runs in a single pass when an alert fires:
- An alert trigger starts the workflow with the alert payload at
event. - An
ai.agentstep runs an initial analysis of the alert and returns the agent's conversation ID so follow-up steps can continue the same conversation. - Two more
ai.agentcalls reuse the conversation to generate a case title and a case description. - A
cases.createCasestep opens the case with the agent-generated title and description. cases.addAlertsattaches the triggering alert.- A
kibana.requeststep fetches the agent's conversation transcript. cases.addCommentsteps append the agent's reasoning trace and the raw analysis as comments for auditability.
-
Trigger on observability alerts
YAML triggers: - type: alertAttach the workflow to the alerting rule you want to investigate.
-
Run initial RCA
Call the agent with the alert payload. Keep
create-conversation: trueso follow-up steps can continue the conversation and the agent has context when generating the title and description:YAML steps: - name: rca_analysis type: ai.agent agent-id: elastic-ai-agent create-conversation: true with: message: | Investigate the following alert and propose root causes. Keep your analysis and data exploration brief to preserve context. <alert> {{ event | json }} </alert>The agent's response is at
steps.rca_analysis.output.message, and the conversation ID is atsteps.rca_analysis.output.conversation_id. -
Generate a case title and description
Reuse the conversation (so the agent remembers its analysis) and ask it for a title and description:
YAML - name: case_title type: ai.agent agent-id: elastic-ai-agent with: conversation_id: "{{ steps.rca_analysis.output.conversation_id }}" message: "Based on your analysis, produce a clear case title. Output only the title." - name: case_description type: ai.agent agent-id: elastic-ai-agent with: conversation_id: "{{ steps.rca_analysis.output.conversation_id }}" message: "Based on your analysis, produce a clear case description. Output only the description."Using a conversation ID keeps tokens cheap and ensures the title and description match the earlier analysis.
-
Open the case
Create the case with the agent-generated title and description:
YAML - name: create_case type: cases.createCase with: title: "{{ steps.case_title.output.message }}" description: "{{ steps.case_description.output.message }}" owner: "observability" severity: "medium" tags: ["auto-rca", "ai-generated"]ownerisobservabilityfor observability cases. -
Attach the alert
YAML - name: attach_alert type: cases.addAlerts with: case_id: "{{ steps.create_case.output.case.id }}" alerts: - alertId: "{{ event.alerts[0]._id }}" index: "{{ event.alerts[0]._index }}" rule: id: "{{ event.rule.id }}" name: "{{ event.rule.name }}" -
Attach the agent's analysis and reasoning
Append the raw analysis as one comment and the reasoning trace as another. Fetch the reasoning trace with a
kibana.requestagainst the Agent Builder conversations API:YAML - name: add_analysis type: cases.addComment with: case_id: "{{ steps.create_case.output.case.id }}" comment: "{{ steps.rca_analysis.output.message }}" - name: get_conversation type: kibana.request with: method: GET path: /api/agent_builder/conversations/{{ steps.rca_analysis.output.conversation_id }} - name: add_reasoning type: cases.addComment with: case_id: "{{ steps.create_case.output.case.id }}" comment: | ## AI investigation summary [View full conversation]({{ kibanaUrl }}/app/agent_builder/conversations/{{ steps.rca_analysis.output.conversation_id }}) {%- for round in steps.get_conversation.output.rounds %} {%- for step in round.steps %} {%- if step.type == "reasoning" %} - **Reasoning:** {{ step.reasoning }} {%- elsif step.type == "tool_call" %} - **Action:** `{{ step.tool_id }}` {%- endif %} {%- endfor %} {%- endfor %}The Liquid loop walks the conversation's rounds and formats each reasoning step and tool call as a bullet. The comment becomes an auditable record of how the agent reached its conclusion.
Full workflow YAML
name: observability--root-cause-analysis
description: Investigate an observability alert with an AI agent, then open a case populated with the analysis and reasoning trace.
enabled: true
tags: ["rca", "ai", "observability"]
triggers:
- type: alert
steps:
- name: rca_analysis
type: ai.agent
agent-id: elastic-ai-agent
create-conversation: true
with:
message: |
Investigate the following alert and propose root causes.
Keep your analysis and data exploration brief to preserve context.
<alert>
{{ event | json }}
</alert>
- name: case_title
type: ai.agent
agent-id: elastic-ai-agent
with:
conversation_id: "{{ steps.rca_analysis.output.conversation_id }}"
message: "Based on your analysis, produce a clear case title. Output only the title."
- name: case_description
type: ai.agent
agent-id: elastic-ai-agent
with:
conversation_id: "{{ steps.rca_analysis.output.conversation_id }}"
message: "Based on your analysis, produce a clear case description. Output only the description."
- name: create_case
type: cases.createCase
with:
title: "{{ steps.case_title.output.message }}"
description: "{{ steps.case_description.output.message }}"
owner: "observability"
severity: "medium"
tags: ["auto-rca", "ai-generated"]
- name: attach_alert
type: cases.addAlerts
with:
case_id: "{{ steps.create_case.output.case.id }}"
alerts:
- alertId: "{{ event.alerts[0]._id }}"
index: "{{ event.alerts[0]._index }}"
rule:
id: "{{ event.rule.id }}"
name: "{{ event.rule.name }}"
- name: add_analysis
type: cases.addComment
with:
case_id: "{{ steps.create_case.output.case.id }}"
comment: "{{ steps.rca_analysis.output.message }}"
- name: get_conversation
type: kibana.request
with:
method: GET
path: /api/agent_builder/conversations/{{ steps.rca_analysis.output.conversation_id }}
- name: add_reasoning
type: cases.addComment
with:
case_id: "{{ steps.create_case.output.case.id }}"
comment: |
## AI investigation summary
[View full conversation]({{ kibanaUrl }}/app/agent_builder/conversations/{{ steps.rca_analysis.output.conversation_id }})
{%- for round in steps.get_conversation.output.rounds %}
{%- for step in round.steps %}
{%- if step.type == "reasoning" %}
- **Reasoning:** {{ step.reasoning }}
{%- elsif step.type == "tool_call" %}
- **Action:** `{{ step.tool_id }}`
{%- endif %}
{%- endfor %}
{%- endfor %}
- Route by service. Use a
switchstep onevent.alerts[0].service.nameto pick different agents for different services (a database-focused agent for DB alerts, a frontend-focused agent for RUM alerts, and so on). - Summarize before paging. Add an
ai.summarizestep that turns the analysis into a one-liner and post it to the on-call Slack channel. - Gate destructive remediation. If you want the workflow to trigger remediation, add an
ifstep that only runs when the agent's confidence is high, and invoke a child workflow that handles the remediation in isolation. - Correlate across signals. Add
elasticsearch.esql.querysteps before the agent call to pull metric and log context in the alert's time window, and feed them into the agent's prompt.
- Observability workflows: The outcome this workflow supports.
- AI steps reference: Parameters for
ai.agentand related AI steps. - Elastic Agent Builder for Observability: How Agent Builder integrates with observability workflows.
- Observability incident management: The product surface this workflow automates.
- Observability alerting: How alert triggers fire on Observability rules.
- Cases action steps: Full reference for
cases.*steps. elastic/workflowsexamples folder: More end-to-end examples.