Elasticsearch for Apache Hadoop
Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability This functionality is not available in Elastic Cloud Serverless. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Serverless
Unavailable
Unavailable
Stack
Generally available
Elasticsearch for Apache Hadoop is an umbrella project consisting of two similar, yet independent sub-projects: elasticsearch-hadoop and repository-hdfs.
This documentation pertains to elasticsearch-hadoop. For information about repository-hdfs and using HDFS as a back-end repository for doing snapshot or restore from or to Elasticsearch, go to Hadoop HDFS repository plugin.
Elasticsearch for Apache Hadoop is an open-source, stand-alone, self-contained, small library that allows big data processing frameworks (specifically Apache Hadoop Map/Reduce and Apache Spark) to interact with Elasticsearch. One can think of it as a connector that allows data to flow bi-directionaly so that applications can leverage transparently the Elasticsearch engine capabilities to significantly enrich their capabilities and improve performance.
Elasticsearch for Apache Hadoop provides native integration for Map/Reduce, Spark, and Hive, making Elasticsearch accessible as if it were a native resource within your data processing cluster. As such, Elasticsearch for Apache Hadoop operates as a library that processing jobs import and use through its APIs to read from and write to Elasticsearch.
While the official name of the project is Elasticsearch for Apache Hadoop throughout the documentation the term elasticsearch-hadoop will be used instead to increase readability.
This document assumes the reader already has a basic familiarity with Elasticsearch, and Hadoop and/or Spark concepts. For more information, refer to Elasticsearch for Apache Hadoop resources.