CEF processor

Extracts fields from Common Event Format (CEF) message document.

Name	Required	Default	Description
`field`	yes	-	The field to be parsed.
`target_field`	no	`field`	The field that the parsed structured object will be written into. Any existing content in this field will be overwritten.
`ignore_missing`	no	`false`	If `true` and `field` does not exist or is `null`, the processor quietly exits without modifying the document
`ignore_empty_values`	no	`true`	If `true` then keys with empty values are quietly ignored in the document
`timezone`	no	UTC	The default timezone used by the processor. Supports template snippets.
`description`	no	-	Description of the processor. Useful for describing the purpose of the processor or its configuration.
`if`	no	-	Conditionally execute the processor. See Conditionally run a processor.
`ignore_failure`	no	`false`	Ignore failures for the processor. See Handling pipeline failures.
`on_failure`	no	-	Handle failures for the processor. See Handling pipeline failures.
`tag`	no	-	Identifier for the processor. Useful for debugging and metrics.

Examples

Simple example

{
  "cef": {
    "field": "message",
    "target_fields": "my_cef",
  }
}

Full Example

Here is a cef processor config

{
  "description" : "...",
  "processors" : [
    {
      "cef" : {
        "field" : "message",
        "target_field" : "my_cef",
      }
    }
  ]
}

When the above processor executes against the following message

CEF:0|Elastic|Vaporware|1.0.0-alpha|18|Web request|low|eventId=3457 requestMethod=POST slat=38.915 slong=-77.511 proto=TCP sourceServiceName=httpd requestContext=https://www.google.com src=1.2.3.4 spt=33876 dst=192.168.10.1 dpt=443 request=https://www.example.com/cart

it produces this result

{
  "my_cef": {
    "severity": "low",
    "name": "Web request",
    "device": {
      "product": "Vaporware",
      "event_class_id": 18,
      "vendor": "Elastic",
      "version": "1.0.0-alpha"
    },
    "version": 0
  },
  "observer": {
    "product": "Vaporware",
    "vendor": "Elastic",
    "version": "1.0.0-alpha"
  },
  "destination": {
    "port": 443,
    "ip": "192.168.10.1"
  },
  "http": {
    "request": {
      "referrer": "https://example.com",
      "method": "POST"
    }
  },
  "source": {
    "geo": {
      "location": {
        "lon": -77.511,
        "lat": 38.915
      }
    },
    "port": 33876,
    "service": {
      "name": "httpd"
    },
    "ip": "1.2.3.4"
  },
  "event": {
    "code": 18,
    "id": 3457
  },
  "url": {
    "original": "https://example.com"
  },
  "network": {
    "transport": "TCP"
  }
}

Example using `ignore_empty_values`

{
  "cef": {
    "field": "message",
    "target_fields": "my_cef",
    "ignore_empty_values": false
  }
}

The final document will have fields with empty values when the corresponding CEF key has no value.

{
  "my_cef": {
    "severity": "low",
    "name": "Web request",
    "device": {
      "product": "Vaporware",
      "event_class_id": 18,
      "vendor": "Elastic",
      "version": "1.0.0-alpha"
    },
    "version": 0
  },
  "http": {
    "request": {
      "referrer": "",
      "method": "POST"
    }
  }
}

Exception scenarios

If the CEF message is invalid according to the spec then an exception is thrown by the processor. Various scenarios include:

CEF header does not start with "CEF:"
Escaped pipe in extensions (moo=this\|has an escaped pipe)
Equals symbol in message (moo=this =has = equals\= )
Malformed escape sequences (moo='Foo-Bar/2018.1.7; =Email:user@example.com;)
When CEF header is truncated (CEF:0|Elastic|Mgmt|activityID=1111111111111111111)
If there are invalid timestamps, mac addresses, or ip addresses