---
title: Keyword marker token filter
description: Marks specified tokens as keywords, which are not stemmed. The keyword_marker filter assigns specified tokens a keyword attribute of true. Stemmer token...
url: https://www.elastic.co/docs/reference/text-analysis/analysis-keyword-marker-tokenfilter
products:
  - Elasticsearch
---

# Keyword marker token filter
Marks specified tokens as keywords, which are not stemmed.
The `keyword_marker` filter assigns specified tokens a `keyword` attribute of `true`. Stemmer token filters, such as [`stemmer`](https://www.elastic.co/docs/reference/text-analysis/analysis-stemmer-tokenfilter) or [`porter_stem`](https://www.elastic.co/docs/reference/text-analysis/analysis-porterstem-tokenfilter), skip tokens with a `keyword` attribute of `true`.
<important>
  To work properly, the `keyword_marker` filter must be listed before any stemmer token filters in the [analyzer configuration](https://docs-demo.elastic.dev/docs/manage-data/data-store/text-analysis/create-custom-analyzer).
</important>

The `keyword_marker` filter uses Lucene’s [KeywordMarkerFilter](https://lucene.apache.org/core/10_0_0/analysis/common/org/apache/lucene/analysis/miscellaneous/KeywordMarkerFilter.html).

## Example

To see how the `keyword_marker` filter works, you first need to produce a token stream containing stemmed tokens.
The following [analyze API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-analyze) request uses the [`stemmer`](https://www.elastic.co/docs/reference/text-analysis/analysis-stemmer-tokenfilter) filter to create stemmed tokens for `fox running and jumping`.
```json

{
  "tokenizer": "whitespace",
  "filter": [ "stemmer" ],
  "text": "fox running and jumping"
}
```

The request produces the following tokens. Note that `running` was stemmed to `run` and `jumping` was stemmed to `jump`.
```text
[ fox, run, and, jump ]
```

To prevent `jumping` from being stemmed, add the `keyword_marker` filter before the `stemmer` filter in the previous analyze API request. Specify `jumping` in the `keywords` parameter of the `keyword_marker` filter.
```json

{
  "tokenizer": "whitespace",
  "filter": [
    {
      "type": "keyword_marker",
      "keywords": [ "jumping" ]
    },
    "stemmer"
  ],
  "text": "fox running and jumping"
}
```

The request produces the following tokens. `running` is still stemmed to `run`, but `jumping` is not stemmed.
```text
[ fox, run, and, jumping ]
```

To see the `keyword` attribute for these tokens, add the following arguments to the analyze API request:
- `explain`: `true`
- `attributes`: `keyword`

```json

{
  "tokenizer": "whitespace",
  "filter": [
    {
      "type": "keyword_marker",
      "keywords": [ "jumping" ]
    },
    "stemmer"
  ],
  "text": "fox running and jumping",
  "explain": true,
  "attributes": "keyword"
}
```

The API returns the following response. Note the `jumping` token has a `keyword` attribute of `true`.
```json
{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "whitespace",
      "tokens": [
        {
          "token": "fox",
          "start_offset": 0,
          "end_offset": 3,
          "type": "word",
          "position": 0
        },
        {
          "token": "running",
          "start_offset": 4,
          "end_offset": 11,
          "type": "word",
          "position": 1
        },
        {
          "token": "and",
          "start_offset": 12,
          "end_offset": 15,
          "type": "word",
          "position": 2
        },
        {
          "token": "jumping",
          "start_offset": 16,
          "end_offset": 23,
          "type": "word",
          "position": 3
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "__anonymous__keyword_marker",
        "tokens": [
          {
            "token": "fox",
            "start_offset": 0,
            "end_offset": 3,
            "type": "word",
            "position": 0,
            "keyword": false
          },
          {
            "token": "running",
            "start_offset": 4,
            "end_offset": 11,
            "type": "word",
            "position": 1,
            "keyword": false
          },
          {
            "token": "and",
            "start_offset": 12,
            "end_offset": 15,
            "type": "word",
            "position": 2,
            "keyword": false
          },
          {
            "token": "jumping",
            "start_offset": 16,
            "end_offset": 23,
            "type": "word",
            "position": 3,
            "keyword": true
          }
        ]
      },
      {
        "name": "stemmer",
        "tokens": [
          {
            "token": "fox",
            "start_offset": 0,
            "end_offset": 3,
            "type": "word",
            "position": 0,
            "keyword": false
          },
          {
            "token": "run",
            "start_offset": 4,
            "end_offset": 11,
            "type": "word",
            "position": 1,
            "keyword": false
          },
          {
            "token": "and",
            "start_offset": 12,
            "end_offset": 15,
            "type": "word",
            "position": 2,
            "keyword": false
          },
          {
            "token": "jumping",
            "start_offset": 16,
            "end_offset": 23,
            "type": "word",
            "position": 3,
            "keyword": true
          }
        ]
      }
    ]
  }
}
```


## Configurable parameters

<definitions>
  <definition term="ignore_case">
    (Optional, Boolean) If `true`, matching for the `keywords` and `keywords_path` parameters ignores letter case. Defaults to `false`.
  </definition>
  <definition term="keywords">
    (Required*, array of strings) Array of keywords. Tokens that match these keywords are not stemmed.
    This parameter, `keywords_path`, or `keywords_pattern` must be specified. You cannot specify this parameter and `keywords_pattern`.
  </definition>
  <definition term="keywords_path">
    (Required*, string) Path to a file that contains a list of keywords. Tokens that match these keywords are not stemmed.
  </definition>
</definitions>

This path must be absolute or relative to the `config` location, and the file must be UTF-8 encoded. Each word in the file must be separated by a line break. The file must be available on every node in the cluster. How you add it depends on where you run Elasticsearch: refer to the [custom configuration file instructions for your deployment type](https://docs-demo.elastic.dev/docs/deploy-manage/plugins-and-custom-configuration-files#plugins-by-deployment-type).
This parameter, `keywords`, or `keywords_pattern` must be specified. You cannot specify this parameter and `keywords_pattern`.
<definitions>
  <definition term="keywords_pattern">
    (Required*, string) [Java regular expression](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.md) used to match tokens. Tokens that match this expression are marked as keywords and not stemmed.
  </definition>
</definitions>

This parameter, `keywords`, or `keywords_path` must be specified. You cannot specify this parameter and `keywords` or `keywords_pattern`.
<warning>
  Poorly written regular expressions can cause Elasticsearch to run slowly or result in stack overflow errors, causing the running node to suddenly exit.
</warning>


## Customize and add to an analyzer

To customize the `keyword_marker` filter, duplicate it to create the basis for a new custom token filter. You can modify the filter using its configurable parameters.
For example, the following [create index API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-create) request uses a custom `keyword_marker` filter and the `porter_stem` filter to configure a new [custom analyzer](https://docs-demo.elastic.dev/docs/manage-data/data-store/text-analysis/create-custom-analyzer).
The custom `keyword_marker` filter marks tokens specified in the `analysis/example_word_list.txt` file as keywords. The `porter_stem` filter does not stem these tokens.
```json

{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_custom_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [
            "my_custom_keyword_marker_filter",
            "porter_stem"
          ]
        }
      },
      "filter": {
        "my_custom_keyword_marker_filter": {
          "type": "keyword_marker",
          "keywords_path": "analysis/example_word_list.txt"
        }
      }
    }
  }
}
```