--- title: EclecticIQ Integration description: The EclecticIQ integration allows you to ingest threat intelligence observables from an outgoing feeds on your EclecticIQ Intelligence Center instance... url: https://www.elastic.co/docs/reference/integrations/ti_eclecticiq products: - Elastic integrations applies_to: - Serverless Observability projects: Generally available - Serverless Security projects: Generally available - Elastic Stack: Generally available since 9.1 --- # EclecticIQ Integration | | | |-------------------------------------------------------------------------------------------------|---------------------------------| | **Version** | 1.11.0 ([View all](#changelog)) | | **Managed integration release status** | GA | | **Subscription level**[What's this?](https://www.elastic.co/subscriptions) | Basic | | **Developed by**[What's this?](https://www.elastic.co/docs/reference/integrations/developed-by) | Partner | | **Ingestion method(s)** | API | | **Minimum Kibana version(s)** | 9.1.08.19.0 | You can use this integration as an Elastic Managed integration on Elastic Cloud Hosted deployments running this version or later. The EclecticIQ integration allows you to ingest threat intelligence [observables](https://docs.eclecticiq.com/ic/current/work-with-intelligence/observables/) from an outgoing feeds on your [EclecticIQ Intelligence Center](https://docs.eclecticiq.com/ic/current/) instance. Observables ingested from an EclecticIQ Intelligence Center outgoing feed can be monitored and explored on [Intelligence → Indicators](https://www.elastic.co/guide/en/security/current/indicators-of-compromise.html) in Kibana. ## Elastic Managed enabled integration Elastic Managed integrations are only supported on Elastic Cloud Serverless and Elastic Cloud Hosted deployments. An Elastic Managed integration lets you ingest data from a cloud source while avoiding the orchestration, management, and maintenance associated with standard ingest infrastructure. Elastic runs the collector for you, so you can focus on your data instead of the infrastructure that collects it. For more information, refer to [Elastic Managed integrations](https://www.elastic.co/docs/manage-data/ingest/managed-integrations/managed-integrations) and the [Elastic Managed integrations FAQ](https://www.elastic.co/docs/manage-data/ingest/managed-integrations/managed-integrations-faq). ## Data streams The EclecticIQ integration collects one type of data streams: logs. **Logs** collected from this integration are collections of threat intelligence observables ingested from the connected EclecticIQ Intelligence Center outgoing feed. ## Requirements You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware. You must also set up your EclecticIQ Intelligence Center for Elasticsearch to connect to it. See *Set up EclecticIQ Intelligence Center*. ## Setup For step-by-step instructions on how to set up an integration, see the [Getting started](https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current/getting-started-observability.html) guide. You must create one integration instance per EclecticIQ Intelligence Center outgoing feed you want to retrieve intelligence from. ### Set up EclecticIQ Intelligence Center Before using the integration, you must: - Set up outgoing feeds on EclecticIQ Intelligence Center. - Connect the integration to the EclectiCIQ Intelligence Center instance. ### Set up outgoing feeds on EclecticIQ Intelligence Center Set up an outgoing feed on EclecticIQ Intelligence Center: [Create and configure outgoing feeds](https://docs.eclecticiq.com/ic/current/integrations/extensions/outgoing-feeds/configure-outgoing-feeds-general-options/). These outgoing feeds must have these properties: - **Transport type:** *HTTP download* - **Content type:** *EclecticIQ Observables CSV* - **Update strategy:** *Append*, *Diff* or *Replace*. This must match the update strategy set for the integration instance. See *Update strategies*. - **Authorized groups:** Must set one or more groups. Feed must be authenticated. See [EclecticIQ Intelligence Center permissions](https://docs.eclecticiq.com/ic/current/get-to-know-the-ic/permissions/ic-permissions/). Only observables packed by this outgoing feed are fetched.> To find the ID of an EclecticIQ Intelligence Center outgoing feed:> 1. Log in to EclecticIQ Intelligence Center. 2. Navigate to **Data configuration > Outgoing feeds**. 3. Select an outgoing feed to open it. 4. Inspect the address bar of your browser. 5. The ID of this outgoing feed is the value of the `?detail=` query parameter. > For example: For an outgoing feed that displays `https://ic-playground.eclecticiq.com/main/configuration/outgoing-feeds?detail=6` in the address bar, its ID is `6`. ### Index name This integration retrieves and makes available the latest version of the threat intelligence retrieved from EclecticIQ Intelligence Center in the following index: `logs-ti_eclecticiq_latest.observables-1` When threat intelligence is deleted from datasets used by the configured outgoing feed, these are removed from that index. In the Intelligence dashboard, to see only the latest threat intelligence from EclecticIQ Intelligence Center, filter results with: ``` _index : logs-ti_eclecticiq_latest.observables-1 and threat.indicator.type : * ``` Or ``` NOT labels.is_ioc_transform_source: * AND and threat.feed.name: "EclecticIQ" ``` ### Update strategies You must set the **same** *Update strategy* for both the EclecticIQ Integration instance and the EclecticIQ Intelligence Center outgoing feed it retrieves data from. Update strategies are how a feed decides to pack data from its configured datasets when it runs: - **(Recommended)** *Diff* only packs data that has been deleted from or added to the feed's datasets since the last run. - *Append* only packs data that has been added to the feed's datasets since the last run. - **(Not recommended)** *Replace* packs *all* the data currently in the feed's datasets each time it runs. Records that already exist on Elasticsearch are de-duplicated, but records that are outdated or removed from the feeds' datasets will not be correspondingly removed from Elasticsearch. **Known issue with *Replace*:** *Replace* usually removes *all* the data from a given destination before replacing it with all the data packed from a given feed's datasets. Currently, this is not supported by the integration. ### Supported EclecticIQ observables The following is a list of EclecticIQ observables supported by this integration. For information about how these observables are mapped, see *Exported fields*. - `asn` - `domain` - `email` - `file` - `file-size` - `hash-md5` - `hash-md5` - `hash-sha1` - `hash-sha256` - `hash-sha384` - `hash-sha512` - `hash-ssdeep` - `ipv4` - `ipv4-cidr` - `ipv6` - `ipv6-cidr` - `mac-48` - `mutex` - `port` - `process` - `process-name` - `uri` - `winregistry` - `certificate-serial-number` - `malware` - `rule` - `user-agent` - `organization` - `email-subject` - `host` - `cve` ### Known issues Certain threat intelligence observables in the Elastic Indicator Intelligence dashboard are displayed with a `-`. That data is not displayed, but retained in the JSON body of the event. ## Example ```json { "@timestamp": "2023-01-01T00:00:00.000Z", "agent": { "ephemeral_id": "cf201e4c-c043-4a07-baa4-2227c8fbb4c3", "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb", "name": "docker-fleet-agent", "type": "filebeat", "version": "8.13.0" }, "data_stream": { "dataset": "ti_eclecticiq.threat", "namespace": "14085", "type": "logs" }, "eclecticiq": { "threat": { "observable_id": "OwWGOybxVeL+USaXvDQSNonD5eU=" } }, "ecs": { "version": "8.11.0" }, "elastic_agent": { "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb", "snapshot": false, "version": "8.13.0" }, "event": { "agent_id_status": "verified", "category": [ "threat" ], "created": "2023-06-08T12:00:30.028Z", "dataset": "ti_eclecticiq.threat", "id": "ZgAq/IXlrjc2J5AdLsDMWhENshI=", "ingested": "2024-08-02T04:24:34Z", "kind": "enrichment", "provider": "Test", "start": "2021-12-19T00:27:19.108Z", "type": [ "indicator" ], "url": "https://www.example.com/" }, "host": { "architecture": "aarch64", "containerized": false, "hostname": "docker-fleet-agent", "id": "8269eab9370b4429947d2a16c3058fcb", "ip": [ "172.29.0.7" ], "mac": [ "02-42-AC-1D-00-07" ], "name": "docker-fleet-agent", "os": { "codename": "focal", "family": "debian", "kernel": "6.4.16-linuxkit", "name": "Ubuntu", "platform": "ubuntu", "type": "linux", "version": "20.04.6 LTS (Focal Fossa)" } }, "input": { "type": "cel" }, "threat": { "indicator": { "first_seen": "2021-12-19T00:27:19.108Z", "marking": { "tlp": "GREEN" }, "name": "example.com", "type": "domain-name", "url": { "domain": "example.com" } } } } ``` ## Exported fields | Field | Description | Type | |---------------------------------|------------------------------------------------------------------------------------------------|------------------| | @timestamp | Event timestamp. | date | | data_stream.dataset | Data stream dataset. | constant_keyword | | data_stream.namespace | Data stream namespace. | constant_keyword | | data_stream.type | Data stream type. | constant_keyword | | eclecticiq.threat.deleted_at | Date when observable was removed from dataset | date | | eclecticiq.threat.observable_id | The ID of the observable, based on kind and value. | keyword | | event.dataset | Event dataset | constant_keyword | | event.module | Event module | constant_keyword | | input.type | Input type | keyword | | labels.is_ioc_transform_source | Indicates whether an IOC is in the raw source data stream, or the in latest destination index. | constant_keyword | | threat.feed.name | Display friendly feed name | constant_keyword | | threat.indicator.first_seen | The date and time when intelligence source first reported sighting this indicator. | date | | threat.indicator.last_seen | The date and time when intelligence source last reported sighting this indicator. | date | | threat.indicator.modified_at | The date and time when intelligence source last modified information for this indicator. | date | ## Changelog | Version | Details | Minimum Kibana version | |------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------| | **1.11.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/20987))Update Elastic Managed integration documentation to the current wording and docs links. | 9.1.08.19.0 | | **1.10.1** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/22123))Statically map `event.ingested` in transform destination indices so detection rules using it as the timestamp override do not report empty destination indices as missing the field. | 9.1.08.19.0 | | **1.10.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/21585))Remove redundant ECS field definitions and the catch-all dynamic template from the transforms, relying on the `ecs@mappings` component template. | 9.1.08.19.0 | | **1.9.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/20927))Enable request trace log removal. | 9.0.58.19.2 | | **1.8.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/20574))Add tags to ingest pipeline processors and add `preserve_original_event` to pipeline-level `on_failure` handlers. | 9.0.58.19.2 | | **1.7.2** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/20421))Set agentless deployment mode `release` field to `ga`. | 9.0.58.19.2 | | **1.7.1** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/20002))Fix transform destination to use keyword for data_stream.namespace, allowing multi-namespace deployments. | 9.0.58.19.2 | | **1.7.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/19716))Enable Agentless deployment. | 9.0.58.19.2 | | **1.6.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/18404))Use num_failure_retries instead of unattended mode for transform failure recovery. | 9.0.08.13.0 | | **1.5.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/16535))Allow transforms to run in unattended mode. | 9.0.08.13.0 | | **1.4.2** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/16180))Remove duplicated field definitions in transform. | 9.0.08.13.0 | | **1.4.1** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/14290))Update `error.message` field mapping as per ECS inside transform destination indices. | 9.0.08.13.0 | | **1.4.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/12919))Update Kibana constraint to support 9.0.0. | 9.0.08.13.0 | | **1.3.2** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/12781))Updated SSL description in package manifest.yml to be uniform and to include links to documentation. | 8.13.0 | | **1.3.1** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/12145))Update links to getting started docs | 8.13.0 | | **1.3.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/12046))Add "preserve_original_event" tag to documents with `event.kind` set to "pipeline_error". | 8.13.0 | | **1.2.4** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/11284))Use triple-brace Mustache templating when referencing variables in ingest pipelines. | 8.13.0 | | **1.2.3** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/11231))Fix labels.is_ioc_transform_source values | 8.13.0 | | **1.2.2** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/11008))Add missing fields in transform | 8.13.0 | | **1.2.1** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/10674))Fix ECS date mapping on threat fields. | 8.13.0 | | **1.2.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/10232))Increase CEL resource.tracer.maxsize to prevent loss of trace responses. | 8.13.0 | | **1.1.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/10135))ECS version updated to 8.11.0. Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template. | 8.13.0 | | **1.0.1** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/10062))Adjust field mappings for transform destination index. | 8.12.0 | | **1.0.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/9779))Release package as GA. | 8.12.0 | | **0.3.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/9238))Set sensitive values as secret, upgrade to package spec 3.0.3. | 8.12.0 | | **0.2.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/8911))Enable SSL configuration for requests**Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/8911))Fix parsing incoming data from "Poll Outgoing feed" stream | 8.8.2 | | **0.1.1** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/8943))Changed owners | 8.8.2 | | **0.1.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/8465))Initial version of the package | 8.8.2 |