Journald Input


Version	1.3.0 (View all)
Subscription level What's this?	Basic
Developed by What's this?	Elastic
Minimum Kibana version(s)	9.0.0 8.8.0

The journald input integration reads logs from the journald system service. The journald input reads the log data and the metadata associated with it.

The journald input is available on Linux systems with systemd installed.

To read Journald logs from within a container, you need to use a Docker image variant that contains journalctl binary. The variant supporting Journald is elastic-agent-complete.

Journal files can have breaking changes making it impossible to read files generated by a newer versions of Journald. Ensure the journal files you are reading were generated by a version equal to or older than the journalctl shipped with the Docker image.

To check the version of journalctl shipped with an Elastic-Agent Docker image, run the following command:

docker run --rm -it --entrypoint journalctl docker.elastic.co/elastic-agent/elastic-agent-complete:<VERSION>  --version

An example event looks as follows:

{
    "@timestamp": "2020-07-22T13:17:10.012Z",
    "agent": {
        "ephemeral_id": "f7858fe6-ce04-46d6-83c3-f45a4e019395",
        "id": "26693255-8a33-48c9-87cc-3d5f846c4bcd",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.11.0"
    },
    "data_stream": {
        "dataset": "journald.logs",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "8.0.0"
    },
    "elastic_agent": {
        "id": "26693255-8a33-48c9-87cc-3d5f846c4bcd",
        "snapshot": true,
        "version": "8.11.0"
    },
    "event": {
        "agent_id_status": "verified",
        "code": "ec387f577b844b8fa948f33cad9a75e6",
        "created": "2023-10-02T18:19:38.048Z",
        "dataset": "journald.logs",
        "ingested": "2023-10-02T18:19:41Z",
        "kind": "event"
    },
    "host": {
        "hostname": "sleipnir",
        "id": "505afdafda3b4f33a63749ae39284742"
    },
    "input": {
        "type": "journald"
    },
    "journald": {
        "custom": {
            "available": "0",
            "available_pretty": "0B",
            "current_use": "1023455232",
            "current_use_pretty": "976.0M",
            "disk_available": "6866636800",
            "disk_available_pretty": "6.3G",
            "disk_keep_free": "1466253312",
            "disk_keep_free_pretty": "1.3G",
            "journal_name": "System journal",
            "journal_path": "/var/log/journal/505afdafda3b4f33a63749ae39284742",
            "limit": "977502208",
            "limit_pretty": "932.2M",
            "max_use": "977502208",
            "max_use_pretty": "932.2M"
        },
        "gid": 0,
        "host": {
            "boot_id": "fa3c2e3080dc4cd5be5cb5a43e140d51"
        },
        "pid": 19317,
        "process": {
            "capabilities": "25402800cf",
            "command_line": "/lib/systemd/systemd-journald",
            "executable": "/lib/systemd/systemd-journald",
            "name": "systemd-journal"
        },
        "uid": 0
    },
    "log": {
        "syslog": {
            "appname": "systemd-journald",
            "facility": {
                "code": 3
            },
            "priority": 6
        }
    },
    "message": "System journal (/var/log/journal/505afdafda3b4f33a63749ae39284742) is 976.0M, max 932.2M, 0B free.",
    "process": {
        "args": [
            "/lib/systemd/systemd-journald"
        ],
        "args_count": 1,
        "command_line": "/lib/systemd/systemd-journald",
        "pid": 19317,
        "thread": {
            "capabilities": {
                "effective": [
                    "CAP_CHOWN",
                    "CAP_DAC_OVERRIDE",
                    "CAP_DAC_READ_SEARCH",
                    "CAP_FOWNER",
                    "CAP_SETGID",
                    "CAP_SETUID",
                    "CAP_SYS_PTRACE",
                    "CAP_SYS_ADMIN",
                    "CAP_AUDIT_CONTROL",
                    "CAP_MAC_OVERRIDE",
                    "CAP_SYSLOG",
                    "CAP_AUDIT_READ"
                ]
            }
        }
    },
    "systemd": {
        "cgroup": "/system.slice/systemd-journald.service",
        "invocation_id": "7c11cda63635437bafe21c92851618a8",
        "slice": "system.slice",
        "transport": "driver",
        "unit": "systemd-journald.service"
    },
    "tags": [
        "forwarded"
    ],
    "user": {
        "group": {
            "id": "0"
        },
        "id": "0"
    }
}

Exported fields

Field	Description	Type
@timestamp	Event timestamp.	date
container.log.tag	User defined tag of a container. Originates from the Docker journald logging driver.	keyword
container.partial	A field that flags log integrity when a message is split. The docker journald logging driver splits long message into multiple events.	boolean
data_stream.dataset	Data stream dataset.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
ecs.version	ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events.	keyword
event.code	Identification code for this event, if one exists. Some event sources use event codes to identify messages unambiguously, regardless of message language or wording adjustments over time. An example of this is the Windows Event ID.	keyword
host.hostname	Hostname of the host. It normally contains what the `hostname` command returns on the host machine.	keyword
host.id	Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`.	keyword
host.ip	Host ip address.	ip
input.type		keyword
journald.audit.login_uid	The login UID of the process the journal entry originates from, as maintained by the kernel audit subsystem.	long
journald.audit.session	The session of the process the journal entry originates from, as maintained by the kernel audit subsystem.	keyword
journald.code.file	The code location generating this message, if known. Contains the source filename.	keyword
journald.code.func	The code location generating this message, if known. Contains the function name.	keyword
journald.code.line	The code location generating this message, if known. Contains the line number.	long
journald.coredump.unit	Used to annotate messages containing coredumps from system units.	keyword
journald.coredump.user_unit	Used to annotate messages containing coredumps from user units.	keyword
journald.custom	Structured fields added to the log message by the caller.	flattened
journald.gid	The group ID of the process the journal entry originates from formatted as a decimal string. Note that entries obtained via "stdout" or "stderr" of forked processes will contain credentials valid for a parent process.	long
journald.host.boot_id	The kernel boot ID for the boot the message was generated in, formatted as a 128-bit hexadecimal string.	keyword
journald.kernel.device	The kernel device name. If the entry is associated to a block device, contains the major and minor numbers of the device node, separated by ":" and prefixed by "b". Similarly for character devices, but prefixed by "c". For network devices, this is the interface index prefixed by "n". For all other devices, this is the subsystem name prefixed by "+", followed by ":", followed by the kernel device name.	keyword
journald.kernel.device_name	The kernel device name as it shows up in the device tree below `/sys/`.	keyword
journald.kernel.device_node_path	The device node path of this device in `/dev/`.	keyword
journald.kernel.device_symlinks	Additional symlink names pointing to the device node in `/dev/`. This field is frequently set more than once per entry.	keyword
journald.kernel.subsystem	The kernel subsystem name.	keyword
journald.object.audit.login_uid		long
journald.object.audit.session		long
journald.object.gid		long
journald.object.pid	Privileged programs (currently UID 0) may attach OBJECT_PID= to a message. This will instruct systemd-journald to attach additional `journald.object.\` on behalf of the caller. These additional fields added automatically by systemd-journald. These additional `journald.object.\` fields are the same as the equivalent `journald.\*` field except that the process identified by PID is described, instead of the process which logged the message.	long
journald.object.process.command_line		keyword
journald.object.process.executable		keyword
journald.object.process.name		keyword
journald.object.systemd.owner_uid		long
journald.object.systemd.session		keyword
journald.object.systemd.unit		keyword
journald.object.systemd.user_unit		keyword
journald.object.uid		long
journald.pid	The process ID of the process the journal entry originates from formatted as a decimal string. Note that entries obtained via "stdout" or "stderr" of forked processes will contain credentials valid for a parent process.	long
journald.process.capabilities	The effective capabilities(7) of the process the journal entry originates from.	keyword
journald.process.command_line	The command line of the process the journal entry originates from.	keyword
journald.process.executable	The executable path of the process the journal entry originates from.	keyword
journald.process.name	The name of the process the journal entry originates from.	keyword
journald.uid	The user ID of the process the journal entry originates from formatted as a decimal string. Note that entries obtained via "stdout" or "stderr" of forked processes will contain credentials valid for a parent process.	long
log.syslog.facility.code	The Syslog numeric facility of the log event, if available. According to RFCs 5424 and 3164, this value should be an integer between 0 and 23.	long
log.syslog.identifier	Identifier (usually process) contained in the syslog header.	keyword
log.syslog.pid	PID contained in the syslog header.	long
log.syslog.priority	Syslog numeric priority of the event, if available. According to RFCs 5424 and 3164, the priority is 8 * facility + severity. This number is therefore expected to contain a value between 0 and 191.	long
message	For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message.	match_only_text
process.args	Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information.	keyword
process.args_count	Length of the process.args array. This field can be useful for querying or performing bucket analysis on how many arguments were provided to start a process. More arguments may be an indication of suspicious activity.	long
process.command_line	Full command line that started the process, including the absolute path to the executable, and all arguments. Some arguments may be filtered to protect sensitive information.	wildcard
process.command_line.text	Multi-field of `process.command_line`.	match_only_text
process.pid	Process id.	long
process.thread.capabilities.effective	This is the set of capabilities used by the kernel to perform permission checks for the thread.	keyword
source.ip	IP address of the source.	ip
systemd.cgroup	The control group path in the systemd hierarchy.	keyword
systemd.invocation_id	The invocation ID for the runtime cycle of the unit the message was generated in, as available to processes of the unit in $INVOCATION_ID.	keyword
systemd.owner_uid	The owner UID of the systemd user unit or systemd session (if any) of the process the journal entry originates from.	long
systemd.session	The systemd session ID (if any).	keyword
systemd.slice	The systemd slice unit name.	keyword
systemd.transport	How the entry was received by the journal service.	keyword
systemd.unit	The systemd unit name.	keyword
systemd.user_slice	The systemd user slice name.	keyword
systemd.user_unit	The unit name in the systemd user manager (if any).	keyword
tags	List of keywords used to tag each event.	keyword
user.group.id	Unique identifier for the group on the system/platform.	keyword
user.id	Unique identifier of the user.	keyword

Changelog

Version	Details	Minimum Kibana version
1.3.0	Enhancement (View pull request) Add missing journald input options.	9.0.0 8.8.0
1.2.1	Enhancement (View pull request) Update documentation to mention the requirements for reading Journald logs.	9.0.0 8.8.0
1.2.0	Enhancement (View pull request) Add support for defining Conditions	9.0.0 8.8.0
1.1.2	Enhancement (View pull request) Update Kibana requirement to support 9.x.	9.0.0 8.8.0
1.1.1	Bug fix (View pull request) Add missing source.ip and host.ip ECS fields.	8.8.0
1.1.0	Enhancement (View pull request) Upgrade journald to ECS 8.10. Enhancement (View pull request) Added ECS mapping process.thread.capabilities.effective for Elastic Agent 8.11 and newer.	8.8.0
1.0.1	Bug fix (View pull request) Align journald syslog fields with ECS.	8.8.0
1.0.0	Enhancement (View pull request) Convert Journald to input type.	8.8.0
0.0.5	Enhancement (View pull request) Added categories and/or subcategories.	—
0.0.4	Enhancement (View pull request) Update Kibana requirement to support 8.x. Enhancement (View pull request) Map `message_id` to ECS `event.code`.	—
0.0.3	Enhancement (View pull request) Add documentation for multi-fields	—
0.0.2	Enhancement (View pull request) Update package description for consistency with other packages.	—
0.0.1	Enhancement (View pull request) Initial release of generic journald input package.	—