Import the trained model and vocabulary

Warning

PyTorch models can execute code on your Elasticsearch server, exposing your cluster to potential security vulnerabilities.

Only use models from trusted sources and never use models from unverified or unknown providers.

Important

If you want to install a trained model in a restricted or closed network, refer to these instructions.

After you choose a model, you must import it and its tokenizer vocabulary to your cluster. When you import the model, it must be chunked and imported one chunk at a time for storage in parts due to its size.

Note

Trained models must be in a TorchScript representation for use with Elastic Stack machine learning features.

Eland is an Elasticsearch Python client that provides a simple script to perform the conversion of Hugging Face transformer models to their TorchScript representations, the chunking process, and upload to Elasticsearch; it is therefore the recommended import method. You can either install the Python Eland client on your machine or use a Docker image to build Eland and run the model import script.

Import with the Eland client installed

Install the Eland Python client with PyTorch extra dependencies.
Shell
```
python -m pip install 'eland[pytorch]'
```
Run the eland_import_hub_model script to download the model from Hugging Face, convert it to TorchScript format, and upload to the Elasticsearch cluster. For example:
Text
```
eland_import_hub_model \
--url <url> \
--es-api-key <api-key> \
--hub-model-id elastic/distilbert-base-cased-finetuned-conll03-english \
--task-type ner
```
1. Specify your Elasticsearch cluster URL.
2. Specify an API key to securely access your cluster. Refer to Authentication methods to learn more.
3. Specify the identifier for the model in the Hugging Face model hub.
4. Specify the type of NLP task. Supported values are fill_mask, ner, question_answering, text_classification, text_embedding, text_expansion, text_similarity, and zero_shot_classification.

For more details, refer to the Eland documentation.

Import with Docker

If you want to use Eland without installing it, run the following command:

$ docker run -it --rm --network host docker.elastic.co/eland/eland

The eland_import_hub_model script can be run directly in the docker command:

docker run -it --rm docker.elastic.co/eland/eland \
    eland_import_hub_model \
      --url $ELASTICSEARCH_URL \
      --hub-model-id elastic/distilbert-base-uncased-finetuned-conll03-english \
      --start

Replace the $ELASTICSEARCH_URL with the URL for your Elasticsearch cluster. Refer to Authentication methods to learn more.

Authentication methods

The following authentication options are available when using the import script:

username/password authentication (specified with the -u and -p options):

eland_import_hub_model --url https://<hostname>:<port> -u <username> -p <password> ...

username/password authentication (embedded in the URL):

eland_import_hub_model --url https://<user>:<password>@<hostname>:<port> ...

API key authentication:

eland_import_hub_model --url https://<hostname>:<port> --es-api-key <api-key> ...